dspam-devel Mailing List for DSPAM (Page 2)
Brought to you by:
paulcockings,
sbajic
You can subscribe to this list here.
2009 |
Jan
|
Feb
(8) |
Mar
(13) |
Apr
(21) |
May
(70) |
Jun
(92) |
Jul
(125) |
Aug
(149) |
Sep
(52) |
Oct
(86) |
Nov
(116) |
Dec
(163) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2010 |
Jan
(113) |
Feb
(28) |
Mar
(24) |
Apr
(107) |
May
(59) |
Jun
(9) |
Jul
(38) |
Aug
(503) |
Sep
(16) |
Oct
(12) |
Nov
(19) |
Dec
(17) |
2011 |
Jan
(2) |
Feb
(20) |
Mar
(2) |
Apr
(3) |
May
(36) |
Jun
(40) |
Jul
(2) |
Aug
(76) |
Sep
(10) |
Oct
(10) |
Nov
(14) |
Dec
(8) |
2012 |
Jan
(1) |
Feb
(9) |
Mar
(4) |
Apr
(9) |
May
(8) |
Jun
(15) |
Jul
|
Aug
(18) |
Sep
(6) |
Oct
(6) |
Nov
(5) |
Dec
(6) |
2013 |
Jan
(16) |
Feb
(13) |
Mar
(5) |
Apr
(1) |
May
|
Jun
(17) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(3) |
Dec
|
2014 |
Jan
(4) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(5) |
Aug
(1) |
Sep
|
Oct
|
Nov
(7) |
Dec
|
2016 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2019 |
Jan
(5) |
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
|
2021 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
From: kael <ka...@cr...> - 2014-01-02 11:24:50
|
Hi there, I've seen in the few last emails on this list that there is no active development on dspam. That shouldn't be an issue if no new feature is planned. But as Thomas Preud'homme (debian maintainer) explained, the problem is that bug fixes are not included upstream. Do you know what could be done to change that state ? How can we help you ? (we=people using dspam and ready to help improving it) Maybe a migration to github (or something like that) would ease the process of reviewing patches ? -- Mickaël. |
From: Thomas Preud'h. <ro...@ce...> - 2013-11-29 11:41:18
|
Le mercredi 27 novembre 2013, 21:28:31 Tom Hendrikx a écrit : > > Hi Thomas, Hi Tom, thanks for your quick answer. > > You are right in stating that there is no active development on dspam. > This is due to a small team of volunteers that don't have the manpower > or even knowledge (I don't know C and dislike perl) to actively > maintain the source. > Attempts in the past on getting more help on board hasn't really > resulted in anything, but the current sleeping state of the project > won't help in solving that situation imho. I understand. I was not doing any judgement. > > I took a quick look at the bug reports you submitted, and I'd be happy > to apply the patches to the source tree, but I'm unable to actually > test them. The hash/css backend is known to be problematic when used > in a larger scale environment, which is why almost nobody uses it in > production. (Considering that POV, it would be unwise to ship the hash > as the default backend like Debian does, but that's a different > discussion.) Well, that could be changed. Out of the other backends, which one is the most tested? As to the patches, I see several problems with the current situation. 1) The first problem is that patches only in the Debian package only benefits to Debian user while I think all of them are *not* Debian specific. 2) I did my best when writing these patches but although I know very wellh C, I am not familiar with dspam code so I might have made some mistakes. Thus, what I seek in merging them in dspam proper is, in addition to the first reason given above, mostly about getting review of the code. I'm willing to improve the patch myself if necessary but I need guidance. On the other hand, I can take the time to walk any reviewer through my patches, explaining the motivation if necessary (it's normally already explained in the associated bug report) and the way the patch works. > > There is still a nice userbase, as you can see in the dspam-user > mailing list, including questions about new installs now and then. For > most people, dspam is a stable tool that works as advertised, albeit > with some 'stay away from this part!' and 'don't turn this knob > unless...' cautions on the box. Even stable, a software always needs now and then a small fix for compiling on a new platform or work with a new version of gcc. And if you look at the patches in Debian [0], at least one of them is a fix for a possible loss of email with dspam (no matter what driver is used). So although I don't expect new feature, I think it is important to have at least someone respond to bug report and react to the most important of them. [0] http://patch-tracker.debian.org/package/dspam/3.10.2+dfsg-12 > > The question is whether Debian (and derative) users will be bothered > by dspam dropping out of the repositories. That would be a question > for dspam-user ml. Personal opinion: I use the packages on ubuntu, and > am glad they're available. Not really. I don't doubt some people use dspam and I maintain some packages with fewer users. But I don't want to give a false impression of security. Debian users can assume that a package in a stable release of Debian will be fixed if any security issue is discovered. If the issue is complicated enough, I would need some help from someone more familiar with the code base to answer some questions. Again, I am not asking for some new feature but I want someone responsible in case I bring a problem up. Else, it means I need to fix any possible bug myself and I certainly cannot do it for all bugs. > > For now, there is no new release planned anytime soon. Maybe that > should change but (I guess) everyone's priorities are somewhere else. > I'll try to look into your patches and commit them. It's alright. Even in Debian very few people expressed the need for new features and when that happened, a patch was proposed in the bug tracker (AFAIK). But it would be nice if patches for bugfix would be considered and applied if correct. If not, a comment to help the author of the patch improve it would already be a significant improvement over the current situation. > > I do like that fact that you care whether we're still alive, and I'd > say: yes, but only just :/ Good. So let me state it one last time. I am willing to keep dspam in Debian in the next release if there is someone I can talk to in case of a problem I cannot solve myself. Someone that knows the code better than I do and that could (i) review patches I provide and (ii) help me fix a bug if it is too complex for me. Note that this shouldn't require much time as so far I managed to provide a patch for all bugs submitted and only an occasional review is necessary. So far I have a bug report every 2 months I would say (I didn't check). > > Kind regards, > Tom Best regards, Thomas |
From: Tom H. <to...@wh...> - 2013-11-27 20:28:43
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 26-11-13 15:34, Thomas Preud'homme wrote: > Hi there, Hi Thomas, > > TL;DR: maintainer of dspam's Debian package speaking. I would like > to know what is the current status of the development of dspam. > > As for any software, bugs are regularly found in dspam. Everytime a > Debian user has reported such a bug, I did my best to triage it, > debug it if it's a real bug and if possible provide a patch in > dspam's bug tracker. However, I didn't see any reaction to my bug > report there and there was no new commit in dspam since august > 2012. There is also not much traffic on this mailing list. You are right in stating that there is no active development on dspam. This is due to a small team of volunteers that don't have the manpower or even knowledge (I don't know C and dislike perl) to actively maintain the source. Attempts in the past on getting more help on board hasn't really resulted in anything, but the current sleeping state of the project won't help in solving that situation imho. I took a quick look at the bug reports you submitted, and I'd be happy to apply the patches to the source tree, but I'm unable to actually test them. The hash/css backend is known to be problematic when used in a larger scale environment, which is why almost nobody uses it in production. (Considering that POV, it would be unwise to ship the hash as the default backend like Debian does, but that's a different discussion.) > > Therefore I am wondering what is the future of dspam. You might > already know it, a stable release of Debian is supported for about > 3 years. I commited myself to support dspam (in version 3.10.1) in > the current Debian release starting from october 2013 but given the > current momentum, I am wondering if I will do the same for the next > release. There is still a nice userbase, as you can see in the dspam-user mailing list, including questions about new installs now and then. For most people, dspam is a stable tool that works as advertised, albeit with some 'stay away from this part!' and 'don't turn this knob unless...' cautions on the box. The question is whether Debian (and derative) users will be bothered by dspam dropping out of the repositories. That would be a question for dspam-user ml. Personal opinion: I use the packages on ubuntu, and am glad they're available. > > So I'm waiting to hear from you about the plan regarding dspam. Is > there still some people working on it? Will there be a new release > soon? What about the patches reported with a patch? For now, there is no new release planned anytime soon. Maybe that should change but (I guess) everyone's priorities are somewhere else. I'll try to look into your patches and commit them. I do like that fact that you care whether we're still alive, and I'd say: yes, but only just :/ Kind regards, Tom -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJSllXvAAoJEJPfMZ19VO/1XakP/19brQErLSW715TeNrxBWu27 YAR6h3Bq+dETk8S8NZqobAmF1Q8N1I9BTdGHcEA5zBQnIajnUl+tsgkgsFRlUOpR jU1HVoWTUtIQ1xnpoL7Q45OvJV7lMdIwjb3MGYDcUwyvLSxESbLz04JTsFTvxqHd tqxwjuxGDiRIvJ7gbh21AYBM4UwU+tZoiUCEypUftCenemIGUzgjb8Q6l8cWCmAj 7fLoN2dgjCFQE5Huagj6Lwh8L6DU4rYdNIsbdXSAwM6TXclBb2kGzUQopEqnEUwA z7Jn9yS3nJyl+f/ezVMURci/voNEe2RWtmn6xT+qi2rE7FcGnWozgH96JCP729Rw zonSzBTq6mPG8zulxXm4Oj9qa90gA47FunyGPbusqhbVv8Ga8JTGOKBW6Jo79FVH MpVbq9EejYghSd0AgJKAmSH8DQO9foDBUBQI/exJZkK6LzcIrl0EdPJ1CUTwgsXF hhTBOX4ev4eElET+Wgawrim5MnjtImGlboQjFfSq/YOEQM3+fSYjdF3Ph+V+R8LV ZFYLxLqVlZ+1tfz/1XmsYEveufqa8hoIL1Yj61b/0uON5ZdcPN9qSIUeY0wZDOYU fhc4MeHIw9J3A2B2mXSD6Yyp5MlT4ibfFc2+By5bMGtTA28vjoB9EwrJcht9fDKt q+CDBtMyj9qx7GJvHa4l =oqjy -----END PGP SIGNATURE----- |
From: Thomas Preud'h. <ro...@ce...> - 2013-11-26 14:50:02
|
Hi there, TL;DR: maintainer of dspam's Debian package speaking. I would like to know what is the current status of the development of dspam. As for any software, bugs are regularly found in dspam. Everytime a Debian user has reported such a bug, I did my best to triage it, debug it if it's a real bug and if possible provide a patch in dspam's bug tracker. However, I didn't see any reaction to my bug report there and there was no new commit in dspam since august 2012. There is also not much traffic on this mailing list. Therefore I am wondering what is the future of dspam. You might already know it, a stable release of Debian is supported for about 3 years. I commited myself to support dspam (in version 3.10.1) in the current Debian release starting from october 2013 but given the current momentum, I am wondering if I will do the same for the next release. So I'm waiting to hear from you about the plan regarding dspam. Is there still some people working on it? Will there be a new release soon? What about the patches reported with a patch? Best regards, Thomas Preud'homme |
From: bofh <bo...@im...> - 2013-06-28 10:20:25
|
Hello everyone, After doing some testing in my Production system :-( i found something intresting. I Use now OSB again, TOE, and the other standards together with Mysql, imple,mented by amavisd - Zimbra setup So now if i use latest git Version i get Results like: X-DSPAM-Result: Spam X-DSPAM-Processed: Fri Jun 28 04:14:52 2013 X-DSPAM-Confidence: 0.9076 X-DSPAM-Probability: 1.0000 X-DSPAM-Signature: 51ccf19c82621724820715 Which seems odd, i send the same test message from gmail over and over again. reclassify it retrain it - training give PASS, still next time same thing again X-DSPAM-Result: Says spam, probability always 1.0000 Then i testet 3.9.0 - because iam using this on zimbra 3.9.0 is the default installation (i jsut symlinked the git version so i can switch quickly) The difference was massive out of the box without retrain the last message i got X-DSPAM-Result: Innocent X-DSPAM-Processed: Fri Jun 28 04:30:03 2013 X-DSPAM-Confidence: 1.0000 X-DSPAM-Probability: 0.0023 X-DSPAM-Signature: 51ccf52b148228935421196 Same Message same Sender just about 30 seconds later after switched dspam Version. only difference was dspam 3.9.0 instead of git. Well i dont say of course this is a better result, i just say its looks more like what ife expected after retrain it 20 times as innocent. Of course i checked if iam using the right database (dspam stats and so show me the right stats, because i switched to mysql theres no hashfiel anywhere so i think both version should really work with the same database) also as a side note, there was almost no difference with feature tb - off and tb=9 in the git version btw why is it marked as spam anyway with a confidence of 0.9 it should be my guess its because of probability is always 1.00000 , maybe something off there So after some Digging in the History i see something about changed calculation in OSB and markov weighting - well exactly waht iam using and yes there is a changed calculation method - i just think it cant be right - even it comes from steeeeveeee - still hate to say but could it be that this patch is a little buggy ? |
From: David R. <dr...@gm...> - 2013-06-13 20:40:12
|
On Thu, Jun 13, 2013 at 1:23 PM, kt...@ri... <kt...@ri...> wrote: > On Thu, Jun 13, 2013 at 12:58:18PM -0700, David Rees wrote: >> On Thu, Jun 13, 2013 at 12:13 PM, kt...@ri... <kt...@ri...> wrote: >> > We have it working here with dspam-3.10.2 and postgresql-9.2.4. >> >> Interesting - what OS are you running? I wonder if it may be because I >> don't have dspam linked against the same version of PostgreSQL. I have >> two systems, one a CentOS 6 system where dspam is linked against the >> OS provided pg libs (8.4) and another Fedora 17 system where dspam is >> linked against a self-compiled pg 9.0... >> >> Maybe I'll try recompiling dspam against pg 9.2 and see what happens... > > We are running on a RHEL 6.3 and using a locally built dspam to allow > access to some internal errors via STDERR that the current source does > not include. But I do not recall having any problems training with the > pre-built packages. Weird. I just recompiled dspam against pg 9.2 on both affected systems and now training appears to work again. I can't believe that worked! Moral of the story - not a bad idea to link dspam against the same version of PostgreSQL that you are running! -Dave |
From: <kt...@ri...> - 2013-06-13 20:23:42
|
On Thu, Jun 13, 2013 at 12:58:18PM -0700, David Rees wrote: > On Thu, Jun 13, 2013 at 12:13 PM, kt...@ri... <kt...@ri...> wrote: > > We have it working here with dspam-3.10.2 and postgresql-9.2.4. > > Interesting - what OS are you running? I wonder if it may be because I > don't have dspam linked against the same version of PostgreSQL. I have > two systems, one a CentOS 6 system where dspam is linked against the > OS provided pg libs (8.4) and another Fedora 17 system where dspam is > linked against a self-compiled pg 9.0... > > Maybe I'll try recompiling dspam against pg 9.2 and see what happens... > > -Dave > We are running on a RHEL 6.3 and using a locally built dspam to allow access to some internal errors via STDERR that the current source does not include. But I do not recall having any problems training with the pre-built packages. Cheers, Ken |
From: David R. <dr...@gm...> - 2013-06-13 19:58:44
|
On Thu, Jun 13, 2013 at 12:13 PM, kt...@ri... <kt...@ri...> wrote: > We have it working here with dspam-3.10.2 and postgresql-9.2.4. Interesting - what OS are you running? I wonder if it may be because I don't have dspam linked against the same version of PostgreSQL. I have two systems, one a CentOS 6 system where dspam is linked against the OS provided pg libs (8.4) and another Fedora 17 system where dspam is linked against a self-compiled pg 9.0... Maybe I'll try recompiling dspam against pg 9.2 and see what happens... -Dave |
From: <kt...@ri...> - 2013-06-13 19:42:32
|
On Thu, Jun 13, 2013 at 11:28:57AM -0700, David Rees wrote: > On Wed, Jun 12, 2013 at 6:06 AM, Chris Moules <chr...@gm...> wrote: > > NOTE: the patch that I sent was only for the single (repeated) error > > that I was receiving. This was while hitting the DB via dspam_train. > > > > I have been evaluating all the storage backends and found Hash to be the > > fastest, but seems buggy with TOE as trainingMode (it crashes, looks > > like when it tries to 'extend' the storage). > > Is training actually working for you on PostgreSQL? It does not appear > to be working for me... > > See: http://sourceforge.net/mailarchive/message.php?msg_id=30708939 > > Haven't had time to dig into the issue further yet. > > As a testament to dspam, on established accounts it still works very > well despite the lack of training, though it consistently misses some > newsletters because of it. > > -Dave > Hi Dave, We have it working here with dspam-3.10.2 and postgresql-9.2.4. Cheers, Ken |
From: David R. <dr...@gm...> - 2013-06-13 19:37:50
|
On Thu, Jun 13, 2013 at 12:13 PM, Tom Hendrikx <to...@wh...> wrote: > On 13-06-13 20:28, David Rees wrote: >> Is training actually working for you on PostgreSQL? It does not appear >> to be working for me... >> >> See: http://sourceforge.net/mailarchive/message.php?msg_id=30708939 >> >> Haven't had time to dig into the issue further yet. >> >> As a testament to dspam, on established accounts it still works very >> well despite the lack of training, though it consistently misses some >> newsletters because of it. > > Yes I saw those messages, but unfortunately did not have anything to add > to it. Training works here, I'm calling dspamc (i.e. client-server mode) > from dovecot-antispam but otherwise I see no huge differences with your > setup. Postgresql version is 9.1.9. Yeah, I'm calling dspam directly. > I don't have any useful C knowledge to dig into the actual code and find > out where the magic breaks for you... WHat does raise questions is that > the last we saw similar trouble, STevan actually removed the escape > string syntax, as you can see in the commit I referred to. Yeah, intersting. I also find that for PostgreSQL 9.1+, you also need to set standard_conforming_strings = off - the default changed in 9.1+. What non-standard settings are you using? > Also the everlasting question comes bck: if you trained other accounts > successfully with the postgres backend, what did you change to make it > break? Good question! It was either a dspam upgrade or PostgreSQL upgrade for me - unfortunately I'm not sure which. It certainly could be related to the PostgreSQL version and settings... -Dave |
From: Tom H. <to...@wh...> - 2013-06-13 19:13:39
|
On 13-06-13 20:28, David Rees wrote: > On Wed, Jun 12, 2013 at 6:06 AM, Chris Moules <chr...@gm...> wrote: >> NOTE: the patch that I sent was only for the single (repeated) error >> that I was receiving. This was while hitting the DB via dspam_train. >> >> I have been evaluating all the storage backends and found Hash to be the >> fastest, but seems buggy with TOE as trainingMode (it crashes, looks >> like when it tries to 'extend' the storage). > > Is training actually working for you on PostgreSQL? It does not appear > to be working for me... > > See: http://sourceforge.net/mailarchive/message.php?msg_id=30708939 > > Haven't had time to dig into the issue further yet. > > As a testament to dspam, on established accounts it still works very > well despite the lack of training, though it consistently misses some > newsletters because of it. > Yes I saw those messages, but unfortunately did not have anything to add to it. Training works here, I'm calling dspamc (i.e. client-server mode) from dovecot-antispam but otherwise I see no huge differences with your setup. Postgresql version is 9.1.9. I don't have any useful C knowledge to dig into the actual code and find out where the magic breaks for you... WHat does raise questions is that the last we saw similar trouble, STevan actually removed the escape string syntax, as you can see in the commit I referred to. Also the everlasting question comes bck: if you trained other accounts successfully with the postgres backend, what did you change to make it break? -- Tom |
From: David R. <dr...@gm...> - 2013-06-13 18:29:23
|
On Wed, Jun 12, 2013 at 6:06 AM, Chris Moules <chr...@gm...> wrote: > NOTE: the patch that I sent was only for the single (repeated) error > that I was receiving. This was while hitting the DB via dspam_train. > > I have been evaluating all the storage backends and found Hash to be the > fastest, but seems buggy with TOE as trainingMode (it crashes, looks > like when it tries to 'extend' the storage). Is training actually working for you on PostgreSQL? It does not appear to be working for me... See: http://sourceforge.net/mailarchive/message.php?msg_id=30708939 Haven't had time to dig into the issue further yet. As a testament to dspam, on established accounts it still works very well despite the lack of training, though it consistently misses some newsletters because of it. -Dave |
From: David R. <dr...@gm...> - 2013-06-13 18:26:48
|
On Wed, Jun 12, 2013 at 3:53 AM, Tom Hendrikx <to...@wh...> wrote: > This is already fixed some time ago, I think it was this commit [1]. > What version of dspam are you running, and which version of postgresql > server? I currently see this on dspam 3.10.2 and PostgreSQL 9.2.4 - both built from source. I turn off escape_string_warning on databases dedicated to dspam to avoid those warnings from clogging up the logs. -Dave |
From: Chris M. <chr...@gm...> - 2013-06-12 13:06:12
|
On 12/06/13 14:51, Chris Moules wrote: > > Could it be that postgres is built with "escape_string_warning = on" ? Just downloaded the source of postgres. I found this in the HISTORY file when grep'ing for escape_string_warning Migration to Version 8.2 ... * Set escape_string_warning to on by default (Bruce) This issues a warning if backslash escapes are used in non-escape (non-E'') strings. So, yes, this is on by default! The Debian patch side-steps the query from *not working* but there is a warning. It might be worth simply updating the queries or having them in a 'version' block. NOTE: the patch that I sent was only for the single (repeated) error that I was receiving. This was while hitting the DB via dspam_train. I have been evaluating all the storage backends and found Hash to be the fastest, but seems buggy with TOE as trainingMode (it crashes, looks like when it tries to 'extend' the storage). Regards, Chris |
From: Chris M. <chr...@gm...> - 2013-06-12 12:52:05
|
On 12/06/13 14:18, Thomas Preud'homme wrote: > Le mercredi 12 juin 2013 14:10:25, Chris Moules a écrit : >> Hello Tom, >> >> System is a Debian 7 test system dspam: 3.10.2 - backport from >> testing postgresql: 9.1 >> >> The line that I patched is still in git HEAD as-is. I can look >> at re-building my full-debug version and posting full output. > > Strange, there is normally a patch (see attached file) to handle > this in Debian. > >> >> Regards, >> >> Chris > > Best regards, > > Thomas of the Debian Dspam team > Hello Thomas, just found that. I had built with DEB_BUILD_OPTIONS="verbose_debug". Could it be that postgres is built with "escape_string_warning = on" ? A grep through /etc/postgresql: postgresql.conf:#escape_string_warning = on The Debian patch sets standard_conforming_strings to off. http://www.postgresql.org/docs/9.1/static/runtime-config-compatible.html <quote> escape_string_warning (boolean) When on, a warning is issued if a backslash (\) appears in an ordinary string literal ('...' syntax) and standard_conforming_strings is off. The default is on. Applications that wish to use backslash as escape should be modified to use escape string syntax (E'...'), because the default behavior of ordinary strings is now to treat backslash as an ordinary character, per SQL standard. This variable can be enabled to help locate code that needs to be changed. </quote> Chris |
From: Thomas Preud'h. <ro...@ce...> - 2013-06-12 12:37:50
|
Le mercredi 12 juin 2013 14:10:25, Chris Moules a écrit : > Hello Tom, > > System is a Debian 7 test system > dspam: 3.10.2 - backport from testing > postgresql: 9.1 > > The line that I patched is still in git HEAD as-is. I can look at > re-building my full-debug version and posting full output. Strange, there is normally a patch (see attached file) to handle this in Debian. > > Regards, > > Chris Best regards, Thomas of the Debian Dspam team |
From: Chris M. <chr...@gm...> - 2013-06-12 12:10:33
|
On 12/06/13 12:53, Tom Hendrikx wrote: > On 06/12/2013 11:56 AM, Chris Moules wrote: >> During some recent testing I was getting: 'WARNING: nonstandard >> use of \\ in a string literal' 'HINT: Use the escape string >> syntax for backslashes, e.g., E'\\'.' >> >> I believe the query in question was not updated since PgSQL 7 >> times. >> >> Patch attached, not heavily tested as I do not use PgSQL in >> production. Maybe a dspam & PgQSL expert can check this. If >> PgSQL is constantly issuing warnings, I can imagine that it slows >> it down a fraction. >> > > Hi Chris, > > This is already fixed some time ago, I think it was this commit > [1]. What version of dspam are you running, and which version of > postgresql server? > > [1] > http://dspam.git.sourceforge.net/git/gitweb.cgi?p=dspam/dspam;a=commitdiff;h=9bc231ae79f84a973ced531356833142c4ae2766 > > > -- Regards, Tom > Hello Tom, System is a Debian 7 test system dspam: 3.10.2 - backport from testing postgresql: 9.1 The line that I patched is still in git HEAD as-is. I can look at re-building my full-debug version and posting full output. Regards, Chris |
From: Tom H. <to...@wh...> - 2013-06-12 10:53:19
|
On 06/12/2013 11:56 AM, Chris Moules wrote: > During some recent testing I was getting: > 'WARNING: nonstandard use of \\ in a string literal' > 'HINT: Use the escape string syntax for backslashes, e.g., E'\\'.' > > I believe the query in question was not updated since PgSQL 7 times. > > Patch attached, not heavily tested as I do not use PgSQL in production. > Maybe a dspam & PgQSL expert can check this. If PgSQL is constantly > issuing warnings, I can imagine that it slows it down a fraction. > Hi Chris, This is already fixed some time ago, I think it was this commit [1]. What version of dspam are you running, and which version of postgresql server? [1] http://dspam.git.sourceforge.net/git/gitweb.cgi?p=dspam/dspam;a=commitdiff;h=9bc231ae79f84a973ced531356833142c4ae2766 -- Regards, Tom |
From: Chris M. <chr...@gm...> - 2013-06-12 10:15:08
|
During some recent testing I was getting: 'WARNING: nonstandard use of \\ in a string literal' 'HINT: Use the escape string syntax for backslashes, e.g., E'\\'.' I believe the query in question was not updated since PgSQL 7 times. Patch attached, not heavily tested as I do not use PgSQL in production. Maybe a dspam & PgQSL expert can check this. If PgSQL is constantly issuing warnings, I can imagine that it slows it down a fraction. Regards Chris |
From: SourceForge.net <no...@so...> - 2013-06-07 17:38:28
|
Bug Tracker item #2943438, was opened at 2010-01-31 15:48 Message generated for change (Comment added) made by bofh999 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2943438&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: v3.9.0 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Enrico Scholz (ensc) Assigned to: Nobody/Anonymous (nobody) Summary: cssclean corrupts hashfile Initial Comment: cssclean.c makes | hash_rec_max = strtol(READ_ATTRIB("HashRecMax"), NULL, 0); | | if (_hash_drv_open(filename, &old, 0, max_seek, | max_extents, extent_size, pctincrease, flags)) | ... | if (_hash_drv_open(newfile, &new, hash_rec_max, max_seek, | max_extents, extent_size, pctincrease, flags)) { | ... | /* preserve counters */ | memcpy(new.header, old.header, sizeof(*new.header)); This causes errors (e.g. a floating point exception due to division by zero) or data corruption when the old file has a hash-rec size (old.header.hash_rec_max) which is different from the configured one. Such a situation happens e.g. when configuration has been changed or 'csscompress' has been run over the file. Then, the hash-driver will work with 'hash_rec_max' some time but when closing the file, the hash_rec_max value of the old file will be written and subsequent operate on wrong hash divider and calculate wrong offset of next extent. ---------------------------------------------------------------------- Comment By: Imposit.com Webmaster (bofh999) Date: 2013-06-07 10:38 Message: Many thanks Since i switched to sbph i had to use the hashdriver however i got major issues at first because the cssclean didnt worked in the current version of zimbra after adding the git version to zimbra i started to use cssclean and of course csscompress - argh bad mistake shot my database now 4 times to hell - well many thanks youve discovered that one so it means it should be fine if we not use csscompress? whats the downsides? how big can it grow? any safe way todo the compress at somepoint? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=2943438&group_id=250683 |
From: SourceForge.net <no...@so...> - 2013-06-05 22:18:30
|
Bug Tracker item #3489939, was opened at 2012-02-21 03:41 Message generated for change (Comment added) made by You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3489939&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon >Group: v3.10.x Status: Open >Resolution: Works For Me Priority: 5 Private: No Submitted By: Norbert () Assigned to: Nobody/Anonymous (nobody) Summary: BUG in DSPAM? Initial Comment: Hi there, today I was debugging some problems with DSPAM and I found this: BUG in DSPAM. Please report this bug: --> Skipping merged group spam: user [dspam-user] is already in merged group I'm using DSPAM in daemon mode with PostgreSQL server - latest GIT version. Here is my configuration string: ./configure --prefix=/usr --sysconfdir=/etc --with-dspam-home=/var/dspam --with-logdir=/var/log --with-storage-driver=pgsql_drv --with-pgsql-includes=/usr/include/postgresql --enable-preferences-extension --enable-large-scale --enable-daemon --enable-virtual-users My group file has only one line: spam:merged:* Here is a full debug from an example connection: 13440: [02/21/2012 11:17:40] DSPAM Instance Startup 13440: [02/21/2012 11:17:40] input args: dspam --deliver=innocent,spam 13440: [02/21/2012 11:17:40] pass-thru args: 13440: [02/21/2012 11:17:40] processing user [dspam-user] 13440: [02/21/2012 11:17:40] uid = 0, euid = 0, gid = 0, egid = 8 13440: [02/21/2012 11:17:40] loading preferences for user [dspam-user] 13440: [02/21/2012 11:17:40] _pgsql_drv_getpwnam: successful returning struct for name: [dspam-user] 13440: [02/21/2012 11:17:40] Loading preferences for uid 1655 13440: [02/21/2012 11:17:40] Loading preferences for uid 0 13440: [02/21/2012 11:17:40] Loading preferences for uid 0 13440: [02/21/2012 11:17:40] default preferences empty. reverting to dspam.conf preferences. 13440: [02/21/2012 11:17:40] Loading preferences from dspam.conf 13440: [02/21/2012 11:17:40] using /var/dspam/opt-in/[dspam-user].dspam as path 13440: [02/21/2012 11:17:40] using /var/dspam/opt-out/[dspam-user].nodspam as path 13440: [02/21/2012 11:17:40] BUG in DSPAM. Please report this bug: 13440: [02/21/2012 11:17:40] --> Skipping merged group spam: user [dspam-user] is already in merged group 13440: [02/21/2012 11:17:40] sedation level set to: 5 13440: [02/21/2012 11:17:40] _pgsql_drv_getpwnam: successful returning struct for name: [dspam-user] 13440: [02/21/2012 11:17:40] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] Loading 3 BNR patterns 13440: [02/21/2012 11:17:42] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] Whitelist threshold: 10 ... 13440: [02/21/2012 11:17:42] Graham-Bayesian Probability: 0.000000 Samples: 15 13440: [02/21/2012 11:17:42] Burton-Bayesian Probability: 0.000000 Samples: 27 13440: [02/21/2012 11:17:42] no factors specified; using default 13440: [02/21/2012 11:17:42] Result Confidence: 0.99 13440: [02/21/2012 11:17:42] auto-whitelisting this message 13440: [02/21/2012 11:17:42] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] Control: [10 10] [10 11] Delta: [0 1] 13440: [02/21/2012 11:17:42] total processing time: 2.46797s 13440: [02/21/2012 11:17:42] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] saving signature as 4f436f46134401406812574 13440: [02/21/2012 11:17:42] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] libdspam returned probability of 0.000000 13440: [02/21/2012 11:17:42] message result: NOT SPAM 13440: [02/21/2012 11:17:42] _pgsql_drv_getpwnam: returning cached name '[dspam-user]' 13440: [02/21/2012 11:17:42] delivering message 13440: [02/21/2012 11:17:42] Establishing connection to /var/imap/socket/lmtp:0 13440: [02/21/2012 11:17:42] Connection established 13440: [02/21/2012 11:17:42] DSPAM Instance Shutdown. Exit Code: 0 From time to time I am experiencing lots of "Broken pipe"s error messages in my sendmail daemon (sendmail <-> DSPAM). Then messages are not being sent - sendmail holds them in a queue. When I exclude DSPAM from my configuration, messages are sent directly to LMTP server and queue is emtied. Just after that I rollback the configuration to sendmail -> DSPAM -> LMTP server (Cyrus) and everything goes back to normal. I assume this "BUG" is not a cause of this problem. I will try to debug it soon. Does this "BUG" can cause real problems? Norbert ---------------------------------------------------------------------- >Comment By: Norbert () Date: 2013-06-05 15:18 Message: Hi again Stevan, I had to examine in depth the problem with "BUG in DSPAM". It seems that in my configuration (take a look above again) it does matter a lot. I receive this error on every message that was addressed to more than one receipient on my system. From my short examination it seems that the agent context flag DAF_MERGED should not be used as agent context wide flag. As I understood from the code it supports just the logic implemented in ctx_init procedure and nothing more. But the agent context wide implementation is causing the problems as described above. When I add the code to reset the flag at the beggining of the ctx_init procedure system is working as expected. Please advise if I am on right track here... Code (dspam.c - ctx_init): ---- int is_group_member_inoculation = 0; int is_group_member_classification = 0; int is_group_member_global = 0; int is_group_member_shared = 0; int is_group_member_merged = 0; char *group; char buffer[10240]; // NDA FIX - clear DAF_MERGED from agent context flags ATX->flags &= ~DAF_MERGED; while (fgets (buffer, sizeof (buffer), file) != NULL) ---- Thanks for all the support you provided so far for the community of DSPAM! ---------------------------------------------------------------------- Comment By: Norbert () Date: 2012-02-22 00:21 Message: Thanks I though so looking at the code... [dspam-user] is just a common reference to real user - I simply replace the real one with this. :) ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-02-21 04:06 Message: > Does this "BUG" can cause real problems? Not really. It is more or less informational. The code is not doing any harm. Just printing that this (in theory) should not happen and therefore it must be a bug in DSPAM. I will need to look into that later. btw: Have you really named your DSPAM User "[dspam-user]"? Why? Why the brackets and especially why that dash? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3489939&group_id=250683 |
From: SourceForge.net <no...@so...> - 2013-04-06 18:42:58
|
Bug Tracker item #3610192, was opened at 2013-04-06 11:42 Message generated for change (Tracker Item Submitted) made by robotux You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3610192&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: GIT Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Preud'homme (robotux) Assigned to: Nobody/Anonymous (nobody) Summary: Add IPv6 support Initial Comment: Please provide support for IPv6. Dspam currently assume IPv4 in several places but the change shouldn't be too big. The IPv6 API is able to handle IPv4 as well so switching should not introduce much code, it's more about rewriting. Some of the places I've noticed relying on IPv4 are: daemon_listen() tracksource() dspam_getsource() is_blacklisted() Some of the changes needed are: char ip[32] -> char [INET6_ADDRSTRLEN] inet_addr -> inet_pton(AF_INET6, …) sizeof(struct sockaddr *) -> sizeof(var) // it's also good practice inet_ntoa_r -> inet_ntop I had started a patch but it's unfinished work so I prefer not to induce you in error. Best regards. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3610192&group_id=250683 |
From: SourceForge.net <no...@so...> - 2013-03-25 12:41:11
|
Bug Tracker item #3609004, was opened at 2013-03-25 05:41 Message generated for change (Tracker Item Submitted) made by You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3609004&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: v3.10.x Status: Open Resolution: None Priority: 5 Private: No Submitted By: tom () Assigned to: Nobody/Anonymous (nobody) Summary: Postgresql 9.2 vs INSERT INTO dspam_token_data duplicate key Initial Comment: Hi, I'm experiencing a bug which might be related to https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3413998&group_id=250683 but on a different table. I use dspam 3.10.2-4 postgresql 9.2.3-1 and see a lot of these in the logs: postgres[15536]: ERROR: duplicate key value violates unique constraint "dspam_token_data_uid_token_key" postgres[15536]: DETAIL: Key (uid, token)=(1, -9183792722717634573) already exists. postgres[15536]: STATEMENT: PREPARE dspam_update_plan (bigint) AS UPDATE dspam_token_data SET last_hit=CURRENT_DATE,innocent_hits=innocent_hits+1 WHERE uid=1 AND token=$1;PREPARE dspam_insert_plan (bigint,int,int) AS INSERT INTO dspam_token_data (uid,token,spam_hits,innocent_hits,last_hit) VALUES (1,$1,$2,$3,CURRENT_DATE); postgres[15536]: ERROR: duplicate key value violates unique constraint "dspam_token_data_uid_token_key" postgres[15536]: DETAIL: Key (uid, token)=(1, -1460994571532515437) already exists. postgres[15536]: STATEMENT: PREPARE dspam_update_plan (bigint) AS UPDATE dspam_token_data SET last_hit=CURRENT_DATE,innocent_hits=innocent_hits+1 WHERE uid=1 AND token=$1;PREPARE dspam_insert_plan (bigint,int,int) AS INSERT INTO dspam_token_data (uid,token,spam_hits,innocent_hits,last_hit) VALUES (1,$1,$2,$3,CURRENT_DATE); postgres[15536]: ERROR: duplicate key value violates unique constraint "dspam_token_data_uid_token_key" postgres[15536]: DETAIL: Key (uid, token)=(1, 9222095682490567990) already exists. postgres[15536]: STATEMENT: PREPARE dspam_update_plan (bigint) AS UPDATE dspam_token_data SET last_hit=CURRENT_DATE,innocent_hits=innocent_hits+1 WHERE uid=1 AND token=$1;PREPARE dspam_insert_plan (bigint,int,int) AS INSERT INTO dspam_token_data (uid,token,spam_hits,innocent_hits,last_hit) VALUES (1,$1,$2,$3,CURRENT_DATE); Setting "standard_conforming_strings = Off" in postgresql.conf helps to workaround the issue, but breaks other things running against this database. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3609004&group_id=250683 |
From: SourceForge.net <no...@so...> - 2013-03-21 16:00:06
|
Bug Tracker item #3536088, was opened at 2012-06-18 11:27 Message generated for change (Comment added) made by drpsycho You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: None Status: Open Resolution: None Priority: 9 Private: No Submitted By: polaris1 (polaris1) Assigned to: Nobody/Anonymous (nobody) Summary: dot stuffing bug Initial Comment: I tried this in both 3.9.0 and in 3.10.1 with similar results. I run dspam in client/daemon mode ( daemon is 3.10.1, clients I tried both 3.9.0 and 3.10.1 ) DSPAM is invoked from C with these arguments: dspam --client --user myuser --deliver=innocent,spam --stdout while the message is being fed in through stdin. I have a copy of the message and the relevant part is here: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you quoted printable only allows 76 characters per line and as it happens, that line ends with a dot on the 77th character thus putting it on a new line ( what are the odds, right ? ) situation can be reproduced like this: cat messagefile | dspam --client --user myuser --deliver=innocent,spam --stdout For that message, the output will stop at 'next week to you='. If I modify the message to stuff the dot manually (..) it will output: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you Obviously something is wrong. I see that dspam has dot stuffing mechanisms in client.c but it's obviously not working. Is it because of the quoted printable ? ---------------------------------------------------------------------- Comment By: Edwin Eefting (drpsycho) Date: 2013-03-21 09:00 Message: After hours of digging though the code i found the bug and came up with a workaround: The problem is basically that the deamon removes the dotstuffing, but fogets to add it in the message that is send back. This cause the client to see a single dot and stop reading the message, resulting in a truncated message. However, if the server would re-add the dotstuffing, the client wouldt remove it, resulting in extra dots send to stdout. I couldn't find an easy way of fixing in the right way(tm), since i dont know enough of the dspam architecture, and i since we want to use this in production right away the risk of breaking stuff was too big. (it might even involve a rewrite of some code) So i came with this workaround: https://github.com/psy0rz/stuff/blob/master/patches/dspam-dotstuffing-workaround.patch This fixes the client, but fixing the deamon the right way involves some more steps. I think this should be easy to do for the core developers since the source of the problem is known now. (i've added some FIXME's and comments in the appropriate places) Edwin Eefting ---------------------------------------------------------------------- Comment By: Edwin Eefting (drpsycho) Date: 2013-03-21 05:26 Message: We also ran against this problem. I definitely think this is a bug: According to the smtp rfc, the dot stuffing/destuffing is part of the smtp-conversation (http://tools.ietf.org/html/rfc5321#section-4.5.2) Dspam usually happens AFTER the conversation and should expect whole messages without worrying about any smtp protocol handling. (such as stuffing/destuffing) There is no need for dspam to know about the special meaning of a single dot on an empty line. (in smtp this in the end-of-message indicator) When calling dspam without --client everything is fine and somehow magically works, but when using the --client option the bug is triggered somehow. This inconsistent behaviour only contributes to the fact that something is not right. Tested this with dspam version 3.10.2 Edwin Eefting Edwin Edwin Eefting ---------------------------------------------------------------------- Comment By: Paul van Tilburg (mozillion) Date: 2013-02-25 07:32 Message: I wonder what --client really should have to do with dot stuffing? I mean, --client is just about how it is handle in the backend (either run a sole instance or use the client/daemon model). In my case I use a pipe to forward the (not dot-stuffed message) in combination with --client for scalability reasons, but then suddenly, dot stuffing is not applied and the message gets truncated. Without --client it all works fine. I find this weird and, I hope we can agree on this, at least undocumented behaviour. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 17:18 Message: When running with --stdout, it's obvious that there should not be any dot stuffing happening, so maybe this behavior is ok after all. The question is, why is DSPAM concerned about dots as a message end marker, when the daemon replies. In daemon.c we have: /* Send a terminating '.' if --stdout in 'dspam' mode */ if (ATX->sockfd_output) { if (send_socket(TTX, ".")<=0) goto CLOSE; and in client.c there is while(line != NULL && strcmp(line, ".")) { Why can't that code be removed entirely from daemon.c and the line in client.c turned into: while(line != NULL) { It would take care of the problem it seems. What am I missing ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:54 Message: So yeah, I think we can safely call this a bug now. 1. dspam client properly does dot stuffing 2. dspam daemon properly does dot de-stuffing when analyzing the message 3. dspam daemon does not re-stuff the dot when sending the reply back to the client in client.c we have this line: while(line != NULL && strcmp(line, ".")) { which automatically cuts the message off at the first lone dot, unprocessed by #3 above ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:40 Message: Yeah, it's definitely within the daemon code where there is no dot stuffing code. Using TCPDUMP here's the output of the outgoing packets towards the daemon: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you . and Incoming from Daemon to Client: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you . It fails to stuff it properly on the way back... ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:30 Message: Furthermore, I added this code in client.c in order to make sure the dot stuffing code is working: r = send(TTX.sockfd, buf+t, buflen - t, 0); strncpy(sbuf, buf+t, r); sbuf[r]='\0'; printf ("SENT: %s\n\n\n", sbuf); And it prints properly the following: SENT: "_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"; type="multipart/alternative" MIME-Version: 1.0 --_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: multipart/alternative; boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_" --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you So obviously the client function detects the lone dot, stuffs it and sends it over properly. There has to be a bug in the daemon code ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:49 Message: There is no MTA involved here. dspam in --client mode, according to my interpretation of client.c should perform dot stuffing. If I read that incorrectly, can you tell me what all that code I pasted in client.c is supposed to do ? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 12:43 Message: Yes. DSPAM binary is responsible doing the dot stuffing WHEN sending to a delivery agent OR sending to the DSPAM daemon. But in your case you feed the DSPAM binary with wrong data. So DSPAM stops processing the message after you just send ONE dot. It is the same as if you would open a connection to port 25 on your system and post something like this: < 220 mail.example.com ESMTP Postfix (2.9.3)\r > EHLO mail.example.com\r < 250-mail.example.com\r < 250-PIPELINING\r < 250-SIZE 52428800\r < 250-ETRN\r < 250-STARTTLS\r < 250-AUTH PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-AUTH=PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-ENHANCEDSTATUSCODES\r < 250-8BITMIME\r < 250 DSN\r > MAIL FROM:<ro...@ex...>\r < 250 2.1.0 Ok\r > RCPT TO:<st...@ba...>\r < 250 2.1.5 Ok\r > DATA\r < 354 End data with <CR><LF>.<CR><LF>\r > Return-Path: <ro...@ex...>\r > Delivered-To: st...@ba...\r > To: <st...@ba...>\r > From: <ro...@ex...>\r > MIME-Version: 1.0\r > Content-Type: multipart/related;\r > boundary="_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_";\r > type="multipart/alternative"\r > Subject: Testing\r > \r > --_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"\r > \r > --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > Content-Type: text/plain; charset="us-ascii"\r > Content-Transfer-Encoding: quoted-printable\r > \r > Dear Daniela,\r > \r > According to Michela, I will send the loading schedule for next week to you=\r > .\r > \r < 250 2.0.0 Ok: queued as 7FDC0A3C170\r > Hello World, how are you\r < 502 5.5.2 Error: command not recognized\r > RSET\r < 250 2.0.0 Ok\r > QUIT\r < 221 2.0.0 Bye\r Assuming you run Postfix, would you say that this is a Postfix bug? I personally would say that this is NOT a bug. Now on the other hand if you have a MTA that is getting the above mail and then sends that message to DSPAM over SMTP then the MTA is responsible to do the dot stuffing, while DSPAM is responsible for doing the dot destuffing. When the message is inside DSPAM (the client) and the client is sending the data to the DSPAM daemon (because of the use of --client) then the DSPAM client is doing dot stuffing while the DSPAM daemon is doing the dot destuffing. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:06 Message: Hmm, I saw this in client.c. I thought DSPAM client would take care of that: * DESCRIPTION * connect to a dspam daemon socket and attempt to process a message * this function is called by the dspam agent when --client is specified /* * fill buf with partial msg, replacing \n with \r\n * and do dot stuffing, if needed. */ /* take care of dot stuffing \n */ if (message->data[i + 1] && message->data[i + 1] == '.') { buf[buflen] = '\n'; buflen++; buf[buflen] = '.'; buflen++; buf[buflen] = '.'; buflen++; i += 2; continue; } I haven't really debugged the code, but it seems there is code there that is supposed to do dot stuffing... ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 11:54 Message: I am not really sure if this is a bug or not. Actually the application sending data to DSPAM is responsible for doing the dot stuffing. In your case you use cat to pipe the message to DSPAM. So 'cat' should do the dot stuffing. Correct me if you see that differently. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683 |
From: SourceForge.net <no...@so...> - 2013-03-21 12:26:56
|
Bug Tracker item #3536088, was opened at 2012-06-18 11:27 Message generated for change (Comment added) made by drpsycho You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: daemon Group: None Status: Open Resolution: None Priority: 9 Private: No Submitted By: polaris1 (polaris1) Assigned to: Nobody/Anonymous (nobody) Summary: dot stuffing bug Initial Comment: I tried this in both 3.9.0 and in 3.10.1 with similar results. I run dspam in client/daemon mode ( daemon is 3.10.1, clients I tried both 3.9.0 and 3.10.1 ) DSPAM is invoked from C with these arguments: dspam --client --user myuser --deliver=innocent,spam --stdout while the message is being fed in through stdin. I have a copy of the message and the relevant part is here: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you quoted printable only allows 76 characters per line and as it happens, that line ends with a dot on the 77th character thus putting it on a new line ( what are the odds, right ? ) situation can be reproduced like this: cat messagefile | dspam --client --user myuser --deliver=innocent,spam --stdout For that message, the output will stop at 'next week to you='. If I modify the message to stuff the dot manually (..) it will output: --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you Obviously something is wrong. I see that dspam has dot stuffing mechanisms in client.c but it's obviously not working. Is it because of the quoted printable ? ---------------------------------------------------------------------- Comment By: Edwin Eefting (drpsycho) Date: 2013-03-21 05:26 Message: We also ran against this problem. I definitely think this is a bug: According to the smtp rfc, the dot stuffing/destuffing is part of the smtp-conversation (http://tools.ietf.org/html/rfc5321#section-4.5.2) Dspam usually happens AFTER the conversation and should expect whole messages without worrying about any smtp protocol handling. (such as stuffing/destuffing) There is no need for dspam to know about the special meaning of a single dot on an empty line. (in smtp this in the end-of-message indicator) When calling dspam without --client everything is fine and somehow magically works, but when using the --client option the bug is triggered somehow. This inconsistent behaviour only contributes to the fact that something is not right. Tested this with dspam version 3.10.2 Edwin Eefting Edwin Edwin Eefting ---------------------------------------------------------------------- Comment By: Paul van Tilburg (mozillion) Date: 2013-02-25 07:32 Message: I wonder what --client really should have to do with dot stuffing? I mean, --client is just about how it is handle in the backend (either run a sole instance or use the client/daemon model). In my case I use a pipe to forward the (not dot-stuffed message) in combination with --client for scalability reasons, but then suddenly, dot stuffing is not applied and the message gets truncated. Without --client it all works fine. I find this weird and, I hope we can agree on this, at least undocumented behaviour. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 17:18 Message: When running with --stdout, it's obvious that there should not be any dot stuffing happening, so maybe this behavior is ok after all. The question is, why is DSPAM concerned about dots as a message end marker, when the daemon replies. In daemon.c we have: /* Send a terminating '.' if --stdout in 'dspam' mode */ if (ATX->sockfd_output) { if (send_socket(TTX, ".")<=0) goto CLOSE; and in client.c there is while(line != NULL && strcmp(line, ".")) { Why can't that code be removed entirely from daemon.c and the line in client.c turned into: while(line != NULL) { It would take care of the problem it seems. What am I missing ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:54 Message: So yeah, I think we can safely call this a bug now. 1. dspam client properly does dot stuffing 2. dspam daemon properly does dot de-stuffing when analyzing the message 3. dspam daemon does not re-stuff the dot when sending the reply back to the client in client.c we have this line: while(line != NULL && strcmp(line, ".")) { which automatically cuts the message off at the first lone dot, unprocessed by #3 above ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:40 Message: Yeah, it's definitely within the daemon code where there is no dot stuffing code. Using TCPDUMP here's the output of the outgoing packets towards the daemon: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you . and Incoming from Daemon to Client: Dear Daniela, According to Michela, I will send the loading schedule for next week to you= . Hello World, how are you . It fails to stuff it properly on the way back... ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 14:30 Message: Furthermore, I added this code in client.c in order to make sure the dot stuffing code is working: r = send(TTX.sockfd, buf+t, buflen - t, 0); strncpy(sbuf, buf+t, r); sbuf[r]='\0'; printf ("SENT: %s\n\n\n", sbuf); And it prints properly the following: SENT: "_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"; type="multipart/alternative" MIME-Version: 1.0 --_005_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: multipart/alternative; boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_" --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Dear Daniela, According to Michela, I will send the loading schedule for next week to you= .. Hello World, how are you So obviously the client function detects the lone dot, stuffs it and sends it over properly. There has to be a bug in the daemon code ? ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:49 Message: There is no MTA involved here. dspam in --client mode, according to my interpretation of client.c should perform dot stuffing. If I read that incorrectly, can you tell me what all that code I pasted in client.c is supposed to do ? ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 12:43 Message: Yes. DSPAM binary is responsible doing the dot stuffing WHEN sending to a delivery agent OR sending to the DSPAM daemon. But in your case you feed the DSPAM binary with wrong data. So DSPAM stops processing the message after you just send ONE dot. It is the same as if you would open a connection to port 25 on your system and post something like this: < 220 mail.example.com ESMTP Postfix (2.9.3)\r > EHLO mail.example.com\r < 250-mail.example.com\r < 250-PIPELINING\r < 250-SIZE 52428800\r < 250-ETRN\r < 250-STARTTLS\r < 250-AUTH PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-AUTH=PLAIN LOGIN DIGEST-MD5 CRAM-MD5\r < 250-ENHANCEDSTATUSCODES\r < 250-8BITMIME\r < 250 DSN\r > MAIL FROM:<ro...@ex...>\r < 250 2.1.0 Ok\r > RCPT TO:<st...@ba...>\r < 250 2.1.5 Ok\r > DATA\r < 354 End data with <CR><LF>.<CR><LF>\r > Return-Path: <ro...@ex...>\r > Delivered-To: st...@ba...\r > To: <st...@ba...>\r > From: <ro...@ex...>\r > MIME-Version: 1.0\r > Content-Type: multipart/related;\r > boundary="_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_";\r > type="multipart/alternative"\r > Subject: Testing\r > \r > --_004_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > boundary="_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_"\r > \r > --_000_9D6F18858B1E854EBC490A2C24C011D629FA52B099SERVER02Royal_\r > Content-Type: text/plain; charset="us-ascii"\r > Content-Transfer-Encoding: quoted-printable\r > \r > Dear Daniela,\r > \r > According to Michela, I will send the loading schedule for next week to you=\r > .\r > \r < 250 2.0.0 Ok: queued as 7FDC0A3C170\r > Hello World, how are you\r < 502 5.5.2 Error: command not recognized\r > RSET\r < 250 2.0.0 Ok\r > QUIT\r < 221 2.0.0 Bye\r Assuming you run Postfix, would you say that this is a Postfix bug? I personally would say that this is NOT a bug. Now on the other hand if you have a MTA that is getting the above mail and then sends that message to DSPAM over SMTP then the MTA is responsible to do the dot stuffing, while DSPAM is responsible for doing the dot destuffing. When the message is inside DSPAM (the client) and the client is sending the data to the DSPAM daemon (because of the use of --client) then the DSPAM client is doing dot stuffing while the DSPAM daemon is doing the dot destuffing. ---------------------------------------------------------------------- Comment By: polaris1 (polaris1) Date: 2012-06-18 12:06 Message: Hmm, I saw this in client.c. I thought DSPAM client would take care of that: * DESCRIPTION * connect to a dspam daemon socket and attempt to process a message * this function is called by the dspam agent when --client is specified /* * fill buf with partial msg, replacing \n with \r\n * and do dot stuffing, if needed. */ /* take care of dot stuffing \n */ if (message->data[i + 1] && message->data[i + 1] == '.') { buf[buflen] = '\n'; buflen++; buf[buflen] = '.'; buflen++; buf[buflen] = '.'; buflen++; i += 2; continue; } I haven't really debugged the code, but it seems there is code there that is supposed to do dot stuffing... ---------------------------------------------------------------------- Comment By: Stevan Bajic (sbajic) Date: 2012-06-18 11:54 Message: I am not really sure if this is a bug or not. Actually the application sending data to DSPAM is responsible for doing the dot stuffing. In your case you use cat to pipe the message to DSPAM. So 'cat' should do the dot stuffing. Correct me if you see that differently. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=1126467&aid=3536088&group_id=250683 |