sqlgrey-users — Users of SQLgrey

Re: [Sqlgrey-users] #Error Messages in Log

[<da...@ha...>]

On 2014-01-02 15:50, Regan Yelcich wrote:
> Try compiling from source it seems to fix the cluster bugs. 
>
AFAIK there are no "cluster bugs" and there is no code to compile (since
sqlgrey is written in perl).
Perhaps you are thinking of another product?


- Dan

>
>> On 2/01/2014, at 6:58 pm, "Grooz, Marc (regio iT)" <Mar...@re...> wrote:
>>
>> Hi Dan,
>>
>> the error still appear after switching from cluster to dbi support:
>>
>> sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 246.
>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO domain_awl (sender_domain, src, first_seen, last_seen) VALUES('example.com','1.2.3','2013-12-11 03:20:02',NOW()):#012, reconnecting to DB
>>
>> kind regards
>>
>> Marc
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Grooz, Marc (regio iT)
>> Gesendet: Freitag, 20. Dezember 2013 13:21
>> An: 'SQLgrey users mailing-list'
>> Betreff: AW: [Sqlgrey-users] #Error Messages in Log
>>
>> Hi Dan,
>>
>> #(i assume you put the "x" into the triplet, to hide the ip?)
>>
>> Yes, you are right.
>>
>> # First, if i understand you correctly, you ONLY have one database-server?
>> Yes, again
>>
>> # So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
>> I try that and come later on to you again and report if the error is still there.
>>
>> Kind regards
>>
>> Marc
>>
>> -----Ursprüngliche Nachricht-----
>> Von: da...@ha... [mailto:da...@ha...]
>> Gesendet: Donnerstag, 19. Dezember 2013 20:35
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>
>> Hi Marc.
>>
>> Well the error message your'e getting, specifically says "Duplicate entry 'x--undef---undef-'".
>> The from AWL table has a primary-key that is made up of the fields src + sender_domain + sender_name. Lets call this a "triplet".
>>
>> (i assume you put the "x" into the triplet, to hide the ip?)
>>
>> Primary keys must be unique. So inserting with a key value with the triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time, but inserting the exact same values will yield a "Duplicate entry" error.
>>
>> Now, sqlgrey shouldnt never have to insert the same keys twice, but i can think of a couple of scenarios where they in theory could happen..
>>
>> First, if i understand you correctly, you ONLY have one database-server?
>> If this is correct, you do not need to enable db-clustingering. I wrote the db-clustering code specifically for having multiple server, where you write to one "master" and read from a lot of slaves. (slaves as in MySQL replication slaves).
>> Also i never tested this code on PostGRES and im unsure how that would work.
>>
>> So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
>>
>>
>> If you ARE running a mysql setup with a master and a slave, this error could in theory occur if your MySQL replication has stopped, failed, is slow or way behind. Because, if the new triplets arent copied to the slave, it wont be there when you read from your read_host. Thus, sqlgrey thinks this is a new triplet and gladly tries to insert into the master, where it actually already existed and boom-goes-the-dynamite.... you get the Duplicate entry error.
>>
>> Regards
>> - Dan
>>
>>
>>
>>> On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
>>> Hi Karl,
>>>
>>> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>>>
>>> Kind regards
>>>
>>> Marc
>>>
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Karl O. Pinc [mailto:ko...@me...]
>>> Gesendet: Mittwoch, 18. Dezember 2013 17:32
>>> An: SQLgrey users mailing-list
>>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>>
>>> Hi Marc,
>>>
>>> Then there's a db connectivity problem of some sort.
>>> Something that prevents the db from even seeing the connection
>>> attempt.  (Unless your db is so lame it's not logging something.)
>>>
>>> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
>>> They are notifications of an unexpected condition.
>>>
>>> Perhaps your db has concurrency problems?  (I always use Postgres
>>> because it's always been most featurefull and caused the fewest
>>> headaches.)
>>>
>>>> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>>>> Hi Karl,
>>>>
>>>> the syslog, db log and db error log are empty of error messages in
>>>> that time.
>>>>
>>>> Kind regards
>>>>
>>>> Marc
>>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Karl O. Pinc [mailto:ko...@me...]
>>>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>>>> An: SQLgrey users mailing-list
>>>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>>>
>>>>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>>>> Hi,
>>>>>
>>>>> i configure tree servers with dbcluster support. They only use one
>>>>> Database as readhost and dbhost. Sometimes I get these error
>>>>> messages:
>>>>>
>>>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>>>> from_awl
>>>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>>>> VALUES('-undef-','-undef-','x','2013-12-16
>>>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key
>>>>> 'PRIMARY', reconnecting to DB
>>>>>
>>>>> any idea?
>>>> Always the usual to check first.  (And always easy to forget.) What
>>>> do the db logs and the os logs say?
>>>>
>>>>
>>>> Karl <ko...@me...>
>>>> Free Software:  "You don't pay back, you pay forward."
>>>>                 -- Robert A. Heinlein
>>>>
>>>> ---------------------------------------------------------------------
>>>> -
>>>> -------- Rapidly troubleshoot problems before they affect your
>>>> business. Most IT organizations don't have a clear picture of how
>>>> application performance affects their revenue. With AppDynamics, you
>>>> get 100% visibility into your Java,.NET, & PHP application. Start
>>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>>> ostg.clktrk
>>>> _______________________________________________
>>>> Sqlgrey-users mailing list
>>>> Sql...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>>>
>>>> ---------------------------------------------------------------------
>>>> -
>>>> -------- Rapidly troubleshoot problems before they affect your
>>>> business. Most IT organizations don't have a clear picture of how
>>>> application performance affects their revenue. With AppDynamics, you
>>>> get 100% visibility into your Java,.NET, & PHP application. Start
>>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>>> ostg.clktrk
>>>> _______________________________________________
>>>> Sqlgrey-users mailing list
>>>> Sql...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>>
>>>
>>> Karl <ko...@me...>
>>> Free Software:  "You don't pay back, you pay forward."
>>>                 -- Robert A. Heinlein
>>>
>>> ----------------------------------------------------------------------
>>> -------- Rapidly troubleshoot problems before they affect your
>>> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
>>> lktrk _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>>
>>> ----------------------------------------------------------------------
>>> -------- Rapidly troubleshoot problems before they affect your
>>> business. Most IT organizations don't have a clear picture of how
>>> application performance affects their revenue. With AppDynamics, you
>>> get 100% visibility into your Java,.NET, & PHP application. Start your
>>> 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
>>> lktrk _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT 
>> organizations don't have a clear picture of how application performance 
>> affects their revenue. With AppDynamics, you get 100% visibility into your 
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT 
> organizations don't have a clear picture of how application performance 
> affects their revenue. With AppDynamics, you get 100% visibility into your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

I didn't use the Distributionpackage. I installed the latest Version from the sqlgrey Website and I don't use the clustercode anymore.

Regards Marc


-----Ursprüngliche Nachricht-----
Von: Regan Yelcich [mailto:reg...@gm...] 
Gesendet: Donnerstag, 2. Januar 2014 15:51
An: SQLgrey users mailing-list
Betreff: Re: [Sqlgrey-users] #Error Messages in Log

Try compiling from source it seems to fix the cluster bugs. 



> On 2/01/2014, at 6:58 pm, "Grooz, Marc (regio iT)" <Mar...@re...> wrote:
> 
> Hi Dan,
> 
> the error still appear after switching from cluster to dbi support:
> 
> sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 246.
> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO 
> domain_awl (sender_domain, src, first_seen, last_seen) 
> VALUES('example.com','1.2.3','2013-12-11 03:20:02',NOW()):#012, 
> reconnecting to DB
> 
> kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: Grooz, Marc (regio iT)
> Gesendet: Freitag, 20. Dezember 2013 13:21
> An: 'SQLgrey users mailing-list'
> Betreff: AW: [Sqlgrey-users] #Error Messages in Log
> 
> Hi Dan,
> 
> #(i assume you put the "x" into the triplet, to hide the ip?)
> 
> Yes, you are right.
> 
> # First, if i understand you correctly, you ONLY have one database-server?
> Yes, again
> 
> # So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
> I try that and come later on to you again and report if the error is still there.
> 
> Kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: da...@ha... [mailto:da...@ha...]
> Gesendet: Donnerstag, 19. Dezember 2013 20:35
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
> 
> Hi Marc.
> 
> Well the error message your'e getting, specifically says "Duplicate entry 'x--undef---undef-'".
> The from AWL table has a primary-key that is made up of the fields src + sender_domain + sender_name. Lets call this a "triplet".
> 
> (i assume you put the "x" into the triplet, to hide the ip?)
> 
> Primary keys must be unique. So inserting with a key value with the triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time, but inserting the exact same values will yield a "Duplicate entry" error.
> 
> Now, sqlgrey shouldnt never have to insert the same keys twice, but i can think of a couple of scenarios where they in theory could happen..
> 
> First, if i understand you correctly, you ONLY have one database-server?
> If this is correct, you do not need to enable db-clustingering. I wrote the db-clustering code specifically for having multiple server, where you write to one "master" and read from a lot of slaves. (slaves as in MySQL replication slaves).
> Also i never tested this code on PostGRES and im unsure how that would work.
> 
> So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
> 
> 
> If you ARE running a mysql setup with a master and a slave, this error could in theory occur if your MySQL replication has stopped, failed, is slow or way behind. Because, if the new triplets arent copied to the slave, it wont be there when you read from your read_host. Thus, sqlgrey thinks this is a new triplet and gladly tries to insert into the master, where it actually already existed and boom-goes-the-dynamite.... you get the Duplicate entry error.
> 
> Regards
> - Dan
> 
> 
> 
>> On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
>> Hi Karl,
>> 
>> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>> 
>> Kind regards
>> 
>> Marc
>> 
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: Karl O. Pinc [mailto:ko...@me...]
>> Gesendet: Mittwoch, 18. Dezember 2013 17:32
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>> 
>> Hi Marc,
>> 
>> Then there's a db connectivity problem of some sort.
>> Something that prevents the db from even seeing the connection 
>> attempt.  (Unless your db is so lame it's not logging something.)
>> 
>> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
>> They are notifications of an unexpected condition.
>> 
>> Perhaps your db has concurrency problems?  (I always use Postgres 
>> because it's always been most featurefull and caused the fewest
>> headaches.)
>> 
>>> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>>> Hi Karl,
>>> 
>>> the syslog, db log and db error log are empty of error messages in 
>>> that time.
>>> 
>>> Kind regards
>>> 
>>> Marc
>>> 
>>> -----Ursprüngliche Nachricht-----
>>> Von: Karl O. Pinc [mailto:ko...@me...]
>>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>>> An: SQLgrey users mailing-list
>>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>> 
>>>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>>> Hi,
>>>> 
>>>> i configure tree servers with dbcluster support. They only use one 
>>>> Database as readhost and dbhost. Sometimes I get these error
>>>> messages:
>>>> 
>>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>>> from_awl
>>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>>> VALUES('-undef-','-undef-','x','2013-12-16
>>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
>>>> 'PRIMARY', reconnecting to DB
>>>> 
>>>> any idea?
>>> Always the usual to check first.  (And always easy to forget.) What 
>>> do the db logs and the os logs say?
>>> 
>>> 
>>> Karl <ko...@me...>
>>> Free Software:  "You don't pay back, you pay forward."
>>>                 -- Robert A. Heinlein
>>> 
>>> --------------------------------------------------------------------
>>> -
>>> -
>>> -------- Rapidly troubleshoot problems before they affect your 
>>> business. Most IT organizations don't have a clear picture of how 
>>> application performance affects their revenue. With AppDynamics, you 
>>> get 100% visibility into your Java,.NET, & PHP application. Start 
>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>> ostg.clktrk
>>> _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>> 
>>> --------------------------------------------------------------------
>>> -
>>> -
>>> -------- Rapidly troubleshoot problems before they affect your 
>>> business. Most IT organizations don't have a clear picture of how 
>>> application performance affects their revenue. With AppDynamics, you 
>>> get 100% visibility into your Java,.NET, & PHP application. Start 
>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>> ostg.clktrk
>>> _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>> 
>> 
>> 
>> Karl <ko...@me...>
>> Free Software:  "You don't pay back, you pay forward."
>>                 -- Robert A. Heinlein
>> 
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.
>> c lktrk _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>> 
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how 
>> application performance affects their revenue. With AppDynamics, you 
>> get 100% visibility into your Java,.NET, & PHP application. Start 
>> your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.
>> c lktrk _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> 
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how 
> application performance affects their revenue. With AppDynamics, you 
> get 100% visibility into your Java,.NET, & PHP application. Start your 
> 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Sqlgrey-users mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Regan Y. <reg...@gm...>]

Try compiling from source it seems to fix the cluster bugs. 



> On 2/01/2014, at 6:58 pm, "Grooz, Marc (regio iT)" <Mar...@re...> wrote:
> 
> Hi Dan,
> 
> the error still appear after switching from cluster to dbi support:
> 
> sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 246.
> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO domain_awl (sender_domain, src, first_seen, last_seen) VALUES('example.com','1.2.3','2013-12-11 03:20:02',NOW()):#012, reconnecting to DB
> 
> kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: Grooz, Marc (regio iT)
> Gesendet: Freitag, 20. Dezember 2013 13:21
> An: 'SQLgrey users mailing-list'
> Betreff: AW: [Sqlgrey-users] #Error Messages in Log
> 
> Hi Dan,
> 
> #(i assume you put the "x" into the triplet, to hide the ip?)
> 
> Yes, you are right.
> 
> # First, if i understand you correctly, you ONLY have one database-server?
> Yes, again
> 
> # So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
> I try that and come later on to you again and report if the error is still there.
> 
> Kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: da...@ha... [mailto:da...@ha...]
> Gesendet: Donnerstag, 19. Dezember 2013 20:35
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
> 
> Hi Marc.
> 
> Well the error message your'e getting, specifically says "Duplicate entry 'x--undef---undef-'".
> The from AWL table has a primary-key that is made up of the fields src + sender_domain + sender_name. Lets call this a "triplet".
> 
> (i assume you put the "x" into the triplet, to hide the ip?)
> 
> Primary keys must be unique. So inserting with a key value with the triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time, but inserting the exact same values will yield a "Duplicate entry" error.
> 
> Now, sqlgrey shouldnt never have to insert the same keys twice, but i can think of a couple of scenarios where they in theory could happen..
> 
> First, if i understand you correctly, you ONLY have one database-server?
> If this is correct, you do not need to enable db-clustingering. I wrote the db-clustering code specifically for having multiple server, where you write to one "master" and read from a lot of slaves. (slaves as in MySQL replication slaves).
> Also i never tested this code on PostGRES and im unsure how that would work.
> 
> So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
> 
> 
> If you ARE running a mysql setup with a master and a slave, this error could in theory occur if your MySQL replication has stopped, failed, is slow or way behind. Because, if the new triplets arent copied to the slave, it wont be there when you read from your read_host. Thus, sqlgrey thinks this is a new triplet and gladly tries to insert into the master, where it actually already existed and boom-goes-the-dynamite.... you get the Duplicate entry error.
> 
> Regards
> - Dan
> 
> 
> 
>> On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
>> Hi Karl,
>> 
>> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>> 
>> Kind regards
>> 
>> Marc
>> 
>> 
>> -----Ursprüngliche Nachricht-----
>> Von: Karl O. Pinc [mailto:ko...@me...]
>> Gesendet: Mittwoch, 18. Dezember 2013 17:32
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>> 
>> Hi Marc,
>> 
>> Then there's a db connectivity problem of some sort.
>> Something that prevents the db from even seeing the connection
>> attempt.  (Unless your db is so lame it's not logging something.)
>> 
>> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
>> They are notifications of an unexpected condition.
>> 
>> Perhaps your db has concurrency problems?  (I always use Postgres
>> because it's always been most featurefull and caused the fewest
>> headaches.)
>> 
>>> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>>> Hi Karl,
>>> 
>>> the syslog, db log and db error log are empty of error messages in
>>> that time.
>>> 
>>> Kind regards
>>> 
>>> Marc
>>> 
>>> -----Ursprüngliche Nachricht-----
>>> Von: Karl O. Pinc [mailto:ko...@me...]
>>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>>> An: SQLgrey users mailing-list
>>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>> 
>>>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>>> Hi,
>>>> 
>>>> i configure tree servers with dbcluster support. They only use one
>>>> Database as readhost and dbhost. Sometimes I get these error
>>>> messages:
>>>> 
>>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>>> from_awl
>>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>>> VALUES('-undef-','-undef-','x','2013-12-16
>>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key
>>>> 'PRIMARY', reconnecting to DB
>>>> 
>>>> any idea?
>>> Always the usual to check first.  (And always easy to forget.) What
>>> do the db logs and the os logs say?
>>> 
>>> 
>>> Karl <ko...@me...>
>>> Free Software:  "You don't pay back, you pay forward."
>>>                 -- Robert A. Heinlein
>>> 
>>> ---------------------------------------------------------------------
>>> -
>>> -------- Rapidly troubleshoot problems before they affect your
>>> business. Most IT organizations don't have a clear picture of how
>>> application performance affects their revenue. With AppDynamics, you
>>> get 100% visibility into your Java,.NET, & PHP application. Start
>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>> ostg.clktrk
>>> _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>> 
>>> ---------------------------------------------------------------------
>>> -
>>> -------- Rapidly troubleshoot problems before they affect your
>>> business. Most IT organizations don't have a clear picture of how
>>> application performance affects their revenue. With AppDynamics, you
>>> get 100% visibility into your Java,.NET, & PHP application. Start
>>> your 15-day FREE TRIAL of AppDynamics Pro!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>>> ostg.clktrk
>>> _______________________________________________
>>> Sqlgrey-users mailing list
>>> Sql...@li...
>>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>> 
>> 
>> 
>> Karl <ko...@me...>
>> Free Software:  "You don't pay back, you pay forward."
>>                 -- Robert A. Heinlein
>> 
>> ----------------------------------------------------------------------
>> -------- Rapidly troubleshoot problems before they affect your
>> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
>> lktrk _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>> 
>> ----------------------------------------------------------------------
>> -------- Rapidly troubleshoot problems before they affect your
>> business. Most IT organizations don't have a clear picture of how
>> application performance affects their revenue. With AppDynamics, you
>> get 100% visibility into your Java,.NET, & PHP application. Start your
>> 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
>> lktrk _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> 
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT 
> organizations don't have a clear picture of how application performance 
> affects their revenue. With AppDynamics, you get 100% visibility into your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

Hi Dan,

the error still appear after switching from cluster to dbi support:

sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 246.
sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO domain_awl (sender_domain, src, first_seen, last_seen) VALUES('example.com','1.2.3','2013-12-11 03:20:02',NOW()):#012, reconnecting to DB

kind regards

Marc

-----Ursprüngliche Nachricht-----
Von: Grooz, Marc (regio iT)
Gesendet: Freitag, 20. Dezember 2013 13:21
An: 'SQLgrey users mailing-list'
Betreff: AW: [Sqlgrey-users] #Error Messages in Log

Hi Dan,

#(i assume you put the "x" into the triplet, to hide the ip?)

Yes, you are right.

# First, if i understand you correctly, you ONLY have one database-server?
Yes, again

# So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
I try that and come later on to you again and report if the error is still there.

Kind regards

Marc

-----Ursprüngliche Nachricht-----
Von: da...@ha... [mailto:da...@ha...]
Gesendet: Donnerstag, 19. Dezember 2013 20:35
An: SQLgrey users mailing-list
Betreff: Re: [Sqlgrey-users] #Error Messages in Log

Hi Marc.

Well the error message your'e getting, specifically says "Duplicate entry 'x--undef---undef-'".
The from AWL table has a primary-key that is made up of the fields src + sender_domain + sender_name. Lets call this a "triplet".

(i assume you put the "x" into the triplet, to hide the ip?)

Primary keys must be unique. So inserting with a key value with the triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time, but inserting the exact same values will yield a "Duplicate entry" error.

Now, sqlgrey shouldnt never have to insert the same keys twice, but i can think of a couple of scenarios where they in theory could happen..

First, if i understand you correctly, you ONLY have one database-server?
If this is correct, you do not need to enable db-clustingering. I wrote the db-clustering code specifically for having multiple server, where you write to one "master" and read from a lot of slaves. (slaves as in MySQL replication slaves).
Also i never tested this code on PostGRES and im unsure how that would work.

So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.


If you ARE running a mysql setup with a master and a slave, this error could in theory occur if your MySQL replication has stopped, failed, is slow or way behind. Because, if the new triplets arent copied to the slave, it wont be there when you read from your read_host. Thus, sqlgrey thinks this is a new triplet and gladly tries to insert into the master, where it actually already existed and boom-goes-the-dynamite.... you get the Duplicate entry error.

Regards
- Dan



On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
> Hi Karl,
>
> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>
> Kind regards
>
> Marc
>
>
> -----Ursprüngliche Nachricht-----
> Von: Karl O. Pinc [mailto:ko...@me...]
> Gesendet: Mittwoch, 18. Dezember 2013 17:32
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>
> Hi Marc,
>
> Then there's a db connectivity problem of some sort.
> Something that prevents the db from even seeing the connection
> attempt.  (Unless your db is so lame it's not logging something.)
>
> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
> They are notifications of an unexpected condition.
>
> Perhaps your db has concurrency problems?  (I always use Postgres
> because it's always been most featurefull and caused the fewest
> headaches.)
>
> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>> Hi Karl,
>>
>> the syslog, db log and db error log are empty of error messages in
>> that time.
>>
>> Kind regards
>>
>> Marc
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Karl O. Pinc [mailto:ko...@me...]
>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>
>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>> Hi,
>>>
>>> i configure tree servers with dbcluster support. They only use one
>>> Database as readhost and dbhost. Sometimes I get these error
>>> messages:
>>>
>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>> from_awl
>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>> VALUES('-undef-','-undef-','x','2013-12-16
>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key
>>> 'PRIMARY', reconnecting to DB
>>>
>>> any idea?
>> Always the usual to check first.  (And always easy to forget.) What
>> do the db logs and the os logs say?
>>
>>
>> Karl <ko...@me...>
>> Free Software:  "You don't pay back, you pay forward."
>>                  -- Robert A. Heinlein
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your
>> business. Most IT organizations don't have a clear picture of how
>> application performance affects their revenue. With AppDynamics, you
>> get 100% visibility into your Java,.NET, & PHP application. Start
>> your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your
>> business. Most IT organizations don't have a clear picture of how
>> application performance affects their revenue. With AppDynamics, you
>> get 100% visibility into your Java,.NET, & PHP application. Start
>> your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>>
>
>
>
> Karl <ko...@me...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
>
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your
> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your
> business. Most IT organizations don't have a clear picture of how
> application performance affects their revenue. With AppDynamics, you
> get 100% visibility into your Java,.NET, & PHP application. Start your
> 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Sqlgrey-users mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

Hi Dan,

#(i assume you put the "x" into the triplet, to hide the ip?)

Yes, you are right.

# First, if i understand you correctly, you ONLY have one database-server?
Yes, again

# So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.
I try that and come later on to you again and report if the error is still there.

Kind regards

Marc

-----Ursprüngliche Nachricht-----
Von: da...@ha... [mailto:da...@ha...] 
Gesendet: Donnerstag, 19. Dezember 2013 20:35
An: SQLgrey users mailing-list
Betreff: Re: [Sqlgrey-users] #Error Messages in Log

Hi Marc.

Well the error message your'e getting, specifically says "Duplicate entry 'x--undef---undef-'".
The from AWL table has a primary-key that is made up of the fields src + sender_domain + sender_name. Lets call this a "triplet".

(i assume you put the "x" into the triplet, to hide the ip?)

Primary keys must be unique. So inserting with a key value with the triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time, but inserting the exact same values will yield a "Duplicate entry" error.

Now, sqlgrey shouldnt never have to insert the same keys twice, but i can think of a couple of scenarios where they in theory could happen..

First, if i understand you correctly, you ONLY have one database-server?
If this is correct, you do not need to enable db-clustingering. I wrote the db-clustering code specifically for having multiple server, where you write to one "master" and read from a lot of slaves. (slaves as in MySQL replication slaves).
Also i never tested this code on PostGRES and im unsure how that would work.

So with only one mysql-server, i would completly disable the dbcluster option, just to lower the complexity of debugging.


If you ARE running a mysql setup with a master and a slave, this error could in theory occur if your MySQL replication has stopped, failed, is slow or way behind. Because, if the new triplets arent copied to the slave, it wont be there when you read from your read_host. Thus, sqlgrey thinks this is a new triplet and gladly tries to insert into the master, where it actually already existed and boom-goes-the-dynamite.... you get the Duplicate entry error.

Regards
- Dan



On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
> Hi Karl,
>
> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>
> Kind regards
>
> Marc
>
>
> -----Ursprüngliche Nachricht-----
> Von: Karl O. Pinc [mailto:ko...@me...]
> Gesendet: Mittwoch, 18. Dezember 2013 17:32
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>
> Hi Marc,
>
> Then there's a db connectivity problem of some sort.
> Something that prevents the db from even seeing the connection 
> attempt.  (Unless your db is so lame it's not logging something.)
>
> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
> They are notifications of an unexpected condition.
>
> Perhaps your db has concurrency problems?  (I always use Postgres 
> because it's always been most featurefull and caused the fewest 
> headaches.)
>
> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>> Hi Karl,
>>
>> the syslog, db log and db error log are empty of error messages in 
>> that time.
>>
>> Kind regards
>>
>> Marc
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Karl O. Pinc [mailto:ko...@me...]
>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>
>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>> Hi,
>>>
>>> i configure tree servers with dbcluster support. They only use one 
>>> Database as readhost and dbhost. Sometimes I get these error
>>> messages:
>>>
>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>> from_awl
>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>> VALUES('-undef-','-undef-','x','2013-12-16
>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
>>> 'PRIMARY', reconnecting to DB
>>>
>>> any idea?
>> Always the usual to check first.  (And always easy to forget.) What 
>> do the db logs and the os logs say?
>>
>>
>> Karl <ko...@me...>
>> Free Software:  "You don't pay back, you pay forward."
>>                  -- Robert A. Heinlein
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how 
>> application performance affects their revenue. With AppDynamics, you 
>> get 100% visibility into your Java,.NET, & PHP application. Start 
>> your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>> ---------------------------------------------------------------------
>> -
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how 
>> application performance affects their revenue. With AppDynamics, you 
>> get 100% visibility into your Java,.NET, & PHP application. Start 
>> your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>>
>
>
>
> Karl <ko...@me...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
>
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how 
> application performance affects their revenue. With AppDynamics, you 
> get 100% visibility into your Java,.NET, & PHP application. Start your 
> 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Sqlgrey-users mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[<da...@ha...>]

On 2013-12-20 00:12, Michael Ludvig wrote:
> Why don't we do INSERT IGNORE ... to prevent the duplicate key error?
> It's probably only a timing issue, the pre-existing record would
> eventually propagate from the master to the slave over time so it's not
> a problem if we can't insert it again.
Beside Karls point, of it being a MySQL specific option, theres another
good reason not to do this;
It doesn't happen in my setup and this is the first time anyone has
reported this. So it would seem that this isnt a normal thing and the
error might actually mean that something is wrong in the mysql setup.
Using ignore would cause one to miss a possible serious problem.
Also, the error simply causes sqlgrey to reconnect to the sqlserver
which is very low cost, even if it happened more than once per minute.

If it turns out that this is normal behavior that we need to account
for, we could simply have sqlgrey ignore 'duplicate key' errors on
insert, when db_host != read_host. But i doubt this is the case.



> One other thing - could the index be corrupted? The index may think the
> record is there while in fact it isn't.
Thats actually a really good suggestion.. I've seen all sorts of odd
behavior from broken myisam indicies..
@Marc - http://dev.mysql.com/doc/refman/5.1/en/myisam-repair.html -
perhaps something will show up :).





> Michael
>
>
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT 
> organizations don't have a clear picture of how application performance 
> affects their revenue. With AppDynamics, you get 100% visibility into your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Karl O. P. <ko...@me...>]

On 12/19/2013 05:12:21 PM, Michael Ludvig wrote:

> Why don't we do INSERT IGNORE ... to prevent the duplicate key error?

That would prevent sqlgrey from working with non-MySQL databases
.
> It's probably only a timing issue, the pre-existing record would
> eventually propagate from the master to the slave over time so it's
> not
> a problem if we can't insert it again.
> 
> One other thing - could the index be corrupted? The index may think
> the
> record is there while in fact it isn't.

If it's a MySQL problem then the problem should be fixed on
the MySQL side, IMO.



Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [Sqlgrey-users] #Error Messages in Log

[Michael L. <ml...@lo...>]

On 20/12/13 08:35, da...@ha... wrote:
> If you ARE running a mysql setup with a master and a slave, this error
> could in theory occur if your MySQL replication has stopped, failed, is
> slow or way behind. Because, if the new triplets arent copied to the
> slave, it wont be there when you read from your read_host. Thus, sqlgrey
> thinks this is a new triplet and gladly tries to insert into the master,
> where it actually already existed and boom-goes-the-dynamite.... you get
> the Duplicate entry error.

Why don't we do INSERT IGNORE ... to prevent the duplicate key error?
It's probably only a timing issue, the pre-existing record would
eventually propagate from the master to the slave over time so it's not
a problem if we can't insert it again.

One other thing - could the index be corrupted? The index may think the
record is there while in fact it isn't.

Michael

Re: [Sqlgrey-users] #Error Messages in Log

[<da...@ha...>]

Hi Marc.

Well the error message your'e getting, specifically says "Duplicate
entry 'x--undef---undef-'".
The from AWL table has a primary-key that is made up of the fields src +
sender_domain + sender_name. Lets call this a "triplet".

(i assume you put the "x" into the triplet, to hide the ip?)

Primary keys must be unique. So inserting with a key value with the
triplet of '192.168.0' + '-undef-' + '-undef-' is fine the first time,
but inserting the exact same values will yield a "Duplicate entry" error.

Now, sqlgrey shouldnt never have to insert the same keys twice, but i
can think of a couple of scenarios where they in theory could happen..

First, if i understand you correctly, you ONLY have one database-server?
If this is correct, you do not need to enable db-clustingering. I wrote
the db-clustering code specifically for having multiple server, where
you write to one "master" and read from a lot of slaves. (slaves as in
MySQL replication slaves).
Also i never tested this code on PostGRES and im unsure how that would work.

So with only one mysql-server, i would completly disable the dbcluster
option, just to lower the complexity of debugging.


If you ARE running a mysql setup with a master and a slave, this error
could in theory occur if your MySQL replication has stopped, failed, is
slow or way behind. Because, if the new triplets arent copied to the
slave, it wont be there when you read from your read_host. Thus, sqlgrey
thinks this is a new triplet and gladly tries to insert into the master,
where it actually already existed and boom-goes-the-dynamite.... you get
the Duplicate entry error.

Regards
- Dan



On 2013-12-19 11:45, Grooz, Marc (regio iT) wrote:
> Hi Karl,
>
> I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.
>
> Kind regards
>
> Marc
>
>
> -----Ursprüngliche Nachricht-----
> Von: Karl O. Pinc [mailto:ko...@me...] 
> Gesendet: Mittwoch, 18. Dezember 2013 17:32
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>
> Hi Marc,
>
> Then there's a db connectivity problem of some sort.
> Something that prevents the db from even seeing the connection attempt.  (Unless your db is so lame it's not logging something.)
>
> By the by, I don't believe that these messages are in indication that any mail is lost, etc.
> They are notifications of an unexpected condition.
>
> Perhaps your db has concurrency problems?  (I always use Postgres because it's always been most featurefull and caused the fewest headaches.)
>
> On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
>> Hi Karl,
>>
>> the syslog, db log and db error log are empty of error messages in 
>> that time.
>>
>> Kind regards
>>
>> Marc
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Karl O. Pinc [mailto:ko...@me...]
>> Gesendet: Dienstag, 17. Dezember 2013 16:48
>> An: SQLgrey users mailing-list
>> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
>>
>> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
>>> Hi,
>>>
>>> i configure tree servers with dbcluster support. They only use one 
>>> Database as readhost and dbhost. Sometimes I get these error
>>> messages:
>>>
>>> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
>> from_awl
>>> (sender_name, sender_domain, src, first_seen, last_seen)
>>> VALUES('-undef-','-undef-','x','2013-12-16
>>> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
>>> 'PRIMARY', reconnecting to DB
>>>
>>> any idea?
>> Always the usual to check first.  (And always easy to forget.) What do 
>> the db logs and the os logs say?
>>
>>
>> Karl <ko...@me...>
>> Free Software:  "You don't pay back, you pay forward."
>>                  -- Robert A. Heinlein
>>
>> ----------------------------------------------------------------------
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how 
>> application performance affects their revenue. With AppDynamics, you 
>> get 100% visibility into your Java,.NET, & PHP application. Start your 
>> 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>> ----------------------------------------------------------------------
>> -------- Rapidly troubleshoot problems before they affect your 
>> business. Most IT organizations don't have a clear picture of how 
>> application performance affects their revenue. With AppDynamics, you 
>> get 100% visibility into your Java,.NET, & PHP application. Start your 
>> 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
>> ostg.clktrk
>> _______________________________________________
>> Sqlgrey-users mailing list
>> Sql...@li...
>> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>>
>>
>
>
>
> Karl <ko...@me...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT 
> organizations don't have a clear picture of how application performance 
> affects their revenue. With AppDynamics, you get 100% visibility into your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

Hi Karl,

I always see only one Connection to the database?! So if you're right there must be a query limit per connection or something.

Kind regards

Marc


-----Ursprüngliche Nachricht-----
Von: Karl O. Pinc [mailto:ko...@me...] 
Gesendet: Mittwoch, 18. Dezember 2013 17:32
An: SQLgrey users mailing-list
Betreff: Re: [Sqlgrey-users] #Error Messages in Log

Hi Marc,

Then there's a db connectivity problem of some sort.
Something that prevents the db from even seeing the connection attempt.  (Unless your db is so lame it's not logging something.)

By the by, I don't believe that these messages are in indication that any mail is lost, etc.
They are notifications of an unexpected condition.

Perhaps your db has concurrency problems?  (I always use Postgres because it's always been most featurefull and caused the fewest headaches.)

On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
> Hi Karl,
> 
> the syslog, db log and db error log are empty of error messages in 
> that time.
> 
> Kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: Karl O. Pinc [mailto:ko...@me...]
> Gesendet: Dienstag, 17. Dezember 2013 16:48
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
> 
> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
> > Hi,
> > 
> > i configure tree servers with dbcluster support. They only use one 
> > Database as readhost and dbhost. Sometimes I get these error
> > messages:
> > 
> > sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
> from_awl
> > (sender_name, sender_domain, src, first_seen, last_seen)
> > VALUES('-undef-','-undef-','x','2013-12-16
> > 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
> > 'PRIMARY', reconnecting to DB
> > 
> > any idea?
> 
> Always the usual to check first.  (And always easy to forget.) What do 
> the db logs and the os logs say?
> 
> 
> Karl <ko...@me...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
> 
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how 
> application performance affects their revenue. With AppDynamics, you 
> get 100% visibility into your Java,.NET, & PHP application. Start your 
> 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
> ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> ----------------------------------------------------------------------
> -------- Rapidly troubleshoot problems before they affect your 
> business. Most IT organizations don't have a clear picture of how 
> application performance affects their revenue. With AppDynamics, you 
> get 100% visibility into your Java,.NET, & PHP application. Start your 
> 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
> ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> 




Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Sqlgrey-users mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Karl O. P. <ko...@me...>]

Hi Marc,

Then there's a db connectivity problem of some sort.
Something that prevents the db from even seeing
the connection attempt.  (Unless your db is so
lame it's not logging something.)

By the by, I don't believe that these messages
are in indication that any mail is lost, etc.
They are notifications of an unexpected condition.

Perhaps your db has concurrency problems?  (I always
use Postgres because it's always been most featurefull
and caused the fewest headaches.)

On 12/18/2013 10:25:51 AM, Grooz, Marc (regio iT) wrote:
> Hi Karl,
> 
> the syslog, db log and db error log are empty of error messages in
> that time.
> 
> Kind regards
> 
> Marc
> 
> -----Ursprüngliche Nachricht-----
> Von: Karl O. Pinc [mailto:ko...@me...] 
> Gesendet: Dienstag, 17. Dezember 2013 16:48
> An: SQLgrey users mailing-list
> Betreff: Re: [Sqlgrey-users] #Error Messages in Log
> 
> On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
> > Hi,
> > 
> > i configure tree servers with dbcluster support. They only use one 
> > Database as readhost and dbhost. Sometimes I get these error
> > messages:
> > 
> > sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO
> from_awl 
> > (sender_name, sender_domain, src, first_seen, last_seen)
> > VALUES('-undef-','-undef-','x','2013-12-16
> > 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
> > 'PRIMARY', reconnecting to DB
> > 
> > any idea?
> 
> Always the usual to check first.  (And always easy to forget.) What 
> do
> the db logs and the os logs say?
> 
> 
> Karl <ko...@me...>
> Free Software:  "You don't pay back, you pay forward."
>                  -- Robert A. Heinlein
> 
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most
> IT organizations don't have a clear picture of how application
> performance affects their revenue. With AppDynamics, you get 100%
> visibility into your Java,.NET, & PHP application. Start your 15-day
> FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
> ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most
> IT 
> organizations don't have a clear picture of how application
> performance 
> affects their revenue. With AppDynamics, you get 100% visibility into
> your 
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of
> AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/
> ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users
> 
> 




Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: [Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

Hi Karl,

the syslog, db log and db error log are empty of error messages in that time.

Kind regards

Marc

-----Ursprüngliche Nachricht-----
Von: Karl O. Pinc [mailto:ko...@me...] 
Gesendet: Dienstag, 17. Dezember 2013 16:48
An: SQLgrey users mailing-list
Betreff: Re: [Sqlgrey-users] #Error Messages in Log

On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
> Hi,
> 
> i configure tree servers with dbcluster support. They only use one 
> Database as readhost and dbhost. Sometimes I get these error
> messages:
> 
> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO from_awl 
> (sender_name, sender_domain, src, first_seen, last_seen)
> VALUES('-undef-','-undef-','x','2013-12-16
> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 
> 'PRIMARY', reconnecting to DB
> 
> any idea?

Always the usual to check first.  (And always easy to forget.) What do the db logs and the os logs say?


Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Sqlgrey-users mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

Re: [Sqlgrey-users] #Error Messages in Log

[Karl O. P. <ko...@me...>]

On 12/17/2013 01:53:14 AM, Grooz, Marc (regio iT) wrote:
> Hi,
> 
> i configure tree servers with dbcluster support. They only use one
> Database as readhost and dbhost. Sometimes I get these error 
> messages:
> 
> sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO 
> from_awl
> (sender_name, sender_domain, src, first_seen, last_seen)
> VALUES('-undef-','-undef-','x','2013-12-16
> 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key
> 'PRIMARY', reconnecting to DB
> 
> any idea?

Always the usual to check first.  (And always easy to forget.)
What do the db logs and the os logs say?


Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

[Sqlgrey-users] #Error Messages in Log

[Grooz, M. (r. iT) <Mar...@re...>]

Hi,

i configure tree servers with dbcluster support. They only use one Database as readhost and dbhost. Sometimes I get these error messages:

sqlgrey: dbaccess: warning: couldn't do query:#012INSERT INTO from_awl (sender_name, sender_domain, src, first_seen, last_seen) VALUES('-undef-','-undef-','x','2013-12-16 08:40:16',NOW()):#012Duplicate entry 'x--undef---undef-' for key 'PRIMARY', reconnecting to DB

any idea?

Kind regards

Marc

Re: [Sqlgrey-users] concatenation errors

[Regan Y. <reg...@gm...>]

On 13/11/2013, at 7:26 am, Dan Faerch <da...@ha...> wrote:

> Regan Yelcich wrote:
>> Hi Anyone,
>> I'm not sure what has changed? But my install of sqlgrey is logging these
>> errors....
>> 
>> Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized value
>> $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line
>> 241.
>> 
> 
> So, i assume that i was right in my last reply and DBI was missing? If so,
> my next guess is that this new error is most likely the lack of
> DBD::mysql.
> 
> Regards
> - Dan


Hi Dan,

No, all of the dependencies were installed.

I solved the initial DBIx::DBCluster issue by compiling it from source. Initially I installed using "apt-get install sqlgrey" which I now know isn't/wasn't compiled with cluster support. :|

Just figured out the error (above) was caused the MySQL Master having read_only = 1 after an automated failover. Found that after running the trailing sql command by hand. Slightly ambiguous error from sqlgrey. :)

Last problem to solve... (hopefully).... consistently after X amount of time I'm getting a "database error" email immediately followed by the "recovered DB" email. So it appears to fail, and then immediately retries and makes a successful connection.

Any ideas on this one? I read about changing the database type to ascii so I've done that and re-created it. Still the same error.

Regan

>>>>>>>>>>>

Nov 13 07:42:38 mta-0 sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 387.
Nov 13 07:42:38 mta-0 sqlgrey: dbaccess: error: couldn't get now() from DB: 
Nov 13 07:42:38 mta-0 sqlgrey: dbaccess: Using DBIx:DBCluster

>>>>>>>>>>>

SQLgrey database error
SQLgrey encountered an SQL error and triggered a reconnection to: DBI:mysql:database=sqlgrey;host=10.189.254.22;port=3306

>>>>>>>>>>>

SQLgrey recovered DB
SQLgrey established connection to: DBI:mysql:database=sqlgrey;host=10.189.254.22;port=3306

Re: [Sqlgrey-users] concatenation errors

[Dan F. <da...@ha...>]

Regan Yelcich wrote:
> Hi Anyone,
> I'm not sure what has changed? But my install of sqlgrey is logging these
> errors....
>
> Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized value
> $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line
> 241.
>

So, i assume that i was right in my last reply and DBI was missing? If so,
my next guess is that this new error is most likely the lack of
DBD::mysql.

Regards
- Dan

Re: [Sqlgrey-users] concatenation errors

[Karl O. P. <ko...@me...>]

On 11/11/2013 08:44:44 PM, Regan Yelcich wrote:
> Hi Anyone,
> I'm not sure what has changed? But my install of sqlgrey is logging
> these errors....
> 
> Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized
> value $DBI::errstr in concatenation (.) or string at /usr/sbin/
> sqlgrey
> line 241.
> Nov 12 14:59:10 admin-sql-0 sqlgrey: dbaccess: warning: couldn't do
> query:#012UPDATE config SET value = '1384221550' WHERE parameter =
> 'last_dbclean' AND value = '1383601636':#012, reconnecting to DB
> 
> Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized
> value $DBI::errstr in concatenation (.) or string at /usr/sbin/
> sqlgrey
> line 241.
> Nov 12 14:59:10 admin-sql-0 sqlgrey: dbaccess: warning: couldn't do
> query:#012UPDATE from_awl SET last_seen = NOW(), first_seen =
> first_seen WHERE sender_name = 'XXXXXXXX' AND sender_domain =
> 'gmail.com' AND src = '209.85.192':#012, reconnecting to DB
> 
> Any idea how to fix this one?

No, but if $DBI::error string is uninitialized that suggests
that something (DBI-ish?) is not installed on your system.
Check that you've all the prerequisites installed.



Karl <ko...@me...>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

[Sqlgrey-users] concatenation errors

[Regan Y. <reg...@gm...>]

Hi Anyone,
I'm not sure what has changed? But my install of sqlgrey is logging these errors....

Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 241.
Nov 12 14:59:10 admin-sql-0 sqlgrey: dbaccess: warning: couldn't do query:#012UPDATE config SET value = '1384221550' WHERE parameter = 'last_dbclean' AND value = '1383601636':#012, reconnecting to DB

Nov 12 14:59:10 admin-sql-0 sqlgrey: warning: Use of uninitialized value $DBI::errstr in concatenation (.) or string at /usr/sbin/sqlgrey line 241.
Nov 12 14:59:10 admin-sql-0 sqlgrey: dbaccess: warning: couldn't do query:#012UPDATE from_awl SET last_seen = NOW(), first_seen = first_seen WHERE sender_name = 'XXXXXXXX' AND sender_domain = 'gmail.com' AND src = '209.85.192':#012, reconnecting to DB

Any idea how to fix this one?

Regan

Re: [Sqlgrey-users] Problems with DBIx:DBCluster

[<da...@ha...>]

Hi Regan..

Im unsure, but my guess (off the top of my head) is that DBI isn't
installed?
if you're on a debian system (or ubuntu or other derivative) try
apt-get install libdbi-perl

- Dan

On 2013-11-06 05:38, Regan Yelcich wrote:
> Hi All,
> I'm getting this error when trying to run sqlgrey in cluster mode... any idea how to fix it? Running it on Ubuntu.
>
> Nov  6 15:23:34 mta-0 sqlgrey: dbaccess: Using DBIx:DBCluster
> Nov  6 15:23:34 mta-0 sqlgrey: fatal: Can't locate object method "connect" via package "DBIx::DBCluster" at /usr/sbin/sqlgrey line 843.
>
> Thanks.
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models. Explore
> techniques for threading, error checking, porting, and tuning. Get the most 
> from the latest Intel processors and coprocessors. See abstracts and register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> Sqlgrey-users mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlgrey-users

[Sqlgrey-users] Problems with DBIx:DBCluster

[Regan Y. <reg...@gm...>]

Hi All,
I'm getting this error when trying to run sqlgrey in cluster mode... any idea how to fix it? Running it on Ubuntu.

Nov  6 15:23:34 mta-0 sqlgrey: dbaccess: Using DBIx:DBCluster
Nov  6 15:23:34 mta-0 sqlgrey: fatal: Can't locate object method "connect" via package "DBIx::DBCluster" at /usr/sbin/sqlgrey line 843.

Thanks.

[Sqlgrey-users] Fwd: sqlgrey problems

[Lionel B. <lio...@bo...>]

-------- Message original --------
Sujet: 	sqlgrey problems
Date : 	Wed, 18 Sep 2013 11:58:03 +0000
De : 	Макаренко Андрей Борисович <mak...@al...>
Pour : 	lio...@bo... <lio...@bo...>



Hi!

I found a bad call of the mylog module (). In version 1.8.0 sqlgrey.

See program report:

-----------------------------

Sep 18 14:48:57 relay4 sqlgrey: warning: Use of uninitialized value
$DBI::errstr in concatenation (.) or string at /usr/local/sbin/sqlgrey
line 246.

-----------------------------

 

I looked at a line 246 and saw a call of the subprogramme of mylog ()
with 3 arguments instead of necessary 4.

 

-----------------------------

# joe  /usr/local/sbin/sqlgrey

 

42     my $VERSION = "1.8.0";

 

 

130   sub mylog($$$$) {

                  my ($self, $logtype, $loglevel, $message) = @_;

 

 

246         $self->mylog('dbaccess', 0, "warning: couldn't do query:\n" .

                   "$query:\n" .

                   "$DBI::errstr, reconnecting to DB");

 

 

 

With the best regards Andrey Mak.

 

[Sqlgrey-users] sqlgrey stats report

[<lc...@go...>]

 /usr/local/bin/sqlgrey-stats.pl --today --programname sqlgrey < 
/path/to/maillog

gives:


##################
## Global stats ##
##################

Events        : 0
Passed        : 0
Early         : 0
Delayed       : 0

Probable SPAM : 0
Throttled     : 0

###############################
## Whitelist/AWL performance ##
###############################

Breakdown for 0 accepted messages:

Whitelists  : N/A%      (0)
Domain AWL  : N/A%      (0)
>From AWL    : N/A%      (0)
Delayed     : N/A%      (0)

 ------------
| Domain AWL |
 ------------


 ----------
| From AWL |
 ----------


 ------
| Spam |
 ------


 -----------
| Throttled |
 -----------


I've tried other options but still no stats.

thanks
Len

[Sqlgrey-users] Further DB documentation?

[Holger S. <lis...@li...>]

Hi.

Is there any further documentation for the database tables? For a
private project I will create a handy REST interface that we will use
for web and mobile UIs. For this it would be great to get a good
understanding of the underlying DB ;)

Thanks in advanced.

Cheers,
Holger

Re: [Sqlgrey-users] hughes.net doesn't play nice with Greylisting

[Kyle L. <la...@uc...>]

Doug Mortensen wrote:
> I think the following class C IP block needs added to the global client IP
> whitelist: 64.98.42.0/24

You should go by their SPF record instead:

$ dig hughes.net TXT +short
"v=spf1 ip4:64.98.40.0/22 ip4:64.98.36.17/32 include:hughesspf.smtp.com ?all"

--Kyle

[Sqlgrey-users] hughes.net doesn't play nice with Greylisting

[Douglas M. <do...@im...>]

I think the following class C IP block needs added to the global client IP whitelist: 64.98.42.0/24

This is for Hughes.Net, who is a fairly large ISP in the USA. They use a different SMTP server for every reconnect/retry after having been deferred for greylisting. See an example below from my spam filter's logs this week (I have changed the email addresses in order to protect the privacy of the parties involved):

mail.log.0:Sep 27 12:57:50 mailguard sqlgrey: spam: 64.98.42.78: so...@hu... -> gra...@so... at 2012-09-26 12:48:21
mail.log.0:Sep 27 12:57:50 mailguard sqlgrey: spam: 64.98.42.70: so...@hu... -> gra...@so... at 2012-09-26 12:48:22
mail.log.0:Sep 27 12:57:50 mailguard sqlgrey: spam: 64.98.42.102: so...@hu... -> gra...@so... at 2012-09-26 12:55:33
mail.log.0:Sep 27 13:27:51 mailguard sqlgrey: spam: 64.98.42.138: so...@hu... -> gra...@so... at 2012-09-26 13:05:13
mail.log.0:Sep 27 13:27:51 mailguard sqlgrey: spam: 64.98.42.125: so...@hu... -> gra...@so... at 2012-09-26 13:25:17
mail.log.1.gz:Sep 26 11:21:38 mailguard sqlgrey: spam: 64.98.42.52: so...@hu... -> gra...@so... at 2012-09-25 10:55:38
mail.log.1.gz:Sep 26 11:21:38 mailguard sqlgrey: spam: 64.98.42.19: so...@hu... -> gra...@so... at 2012-09-25 10:55:45
mail.log.1.gz:Sep 26 11:21:38 mailguard sqlgrey: spam: 64.98.42.86: so...@hu... -> gra...@so... at 2012-09-25 11:05:17
mail.log.1.gz:Sep 26 11:21:38 mailguard sqlgrey: spam: 64.98.42.25: so...@hu... -> gra...@so... at 2012-09-25 11:15:18
mail.log.1.gz:Sep 26 11:51:41 mailguard sqlgrey: spam: 64.98.42.56: so...@hu... -> gra...@so... at 2012-09-25 11:35:17
mail.log.1.gz:Sep 26 12:21:43 mailguard sqlgrey: spam: 64.98.42.95: so...@hu... -> gra...@so... at 2012-09-25 12:15:18
mail.log.1.gz:Sep 26 13:25:18 mailguard postfix/smtpd[11067]: NOQUEUE: reject: RCPT from smtprelay0125.b.hostedemail.com[64.98.42.125]: 554 5.7.1 Service unavailable; Client host [64.98.42.125] blocked using bl.spamcop.net; Blocked - see http://www.spamcop.net/bl.shtml?64.98.42.125; from=<so...@hu...> to=<gra...@so...> proto=ESMTP helo=<smtprelay.b.hostedemail.com>
mail.log.1.gz:Sep 26 13:51:59 mailguard sqlgrey: spam: 64.98.42.117: so...@hu... -> gra...@so... at 2012-09-25 13:25:16
mail.log.1.gz:Sep 26 14:52:02 mailguard sqlgrey: spam: 64.98.42.64: so...@hu... -> gra...@so... at 2012-09-25 14:35:17
mail.log.1.gz:Sep 26 15:52:04 mailguard sqlgrey: spam: 64.98.42.135: so...@hu... -> gra...@so... at 2012-09-25 15:45:16
mail.log.1.gz:Sep 26 17:22:13 mailguard sqlgrey: spam: 64.98.42.8: so...@hu... -> gra...@so... at 2012-09-25 16:55:16
mail.log.1.gz:Sep 26 18:22:34 mailguard sqlgrey: spam: 64.98.42.77: so...@hu... -> gra...@so... at 2012-09-25 18:05:18
mail.log.1.gz:Sep 26 19:22:39 mailguard sqlgrey: spam: 64.98.42.82: so...@hu... -> gra...@so... at 2012-09-25 19:15:19
mail.log.1.gz:Sep 26 20:52:52 mailguard sqlgrey: spam: 64.98.42.227: so...@hu... -> gra...@so... at 2012-09-25 20:25:17
mail.log.1.gz:Sep 26 21:52:58 mailguard sqlgrey: spam: 64.98.42.223: so...@hu... -> gra...@so... at 2012-09-25 21:35:16
mail.log.1.gz:Sep 26 22:53:14 mailguard sqlgrey: spam: 64.98.42.51: so...@hu... -> gra...@so... at 2012-09-25 22:45:19
mail.log.1.gz:Sep 27 00:23:30 mailguard sqlgrey: spam: 64.98.42.40: so...@hu... -> gra...@so... at 2012-09-25 23:55:17
mail.log.1.gz:Sep 27 01:24:27 mailguard sqlgrey: spam: 64.98.42.144: so...@hu... -> gra...@so... at 2012-09-26 01:05:16
mail.log.1.gz:Sep 27 02:25:07 mailguard sqlgrey: spam: 64.98.42.129: so...@hu... -> gra...@so... at 2012-09-26 02:15:15

I have added the following to my clients_ip_whitelist.local:

64.98.42        # Hughes.Net uses a pool of IPs for SMTP & retries from a different IP every time. Doesn't work for greylisting.

I suggest everyone else do the same until it is added to the globally updated list.

Thanks,
-
Doug Mortensen
Network Consultant
Impala Networks Inc
CCNA, MCSA, Security+, A+
Linux+, Network+, Server+
A.A.S. Information Technology
.
www.impalanetworks.com<http://www.impalanetworks.com/>
P: (505) 327-7300
F: (505) 327-7545