pyzor-users — general discussion of Pyzor and Pyzor-related topics

Re: pyzor

[Frank T. <ft...@ne...>]

Jimmy Hayes, on 2004-12-28, wrote:

> Hello I just installed pyzor and when I run spamassassin -D < test.txt I
> get the below output for pyzor.
>
> executable for pyzor was found at /usr/bin/pyzor
> Pyzor is available: /usr/bin/pyzor
> entering helper-app run mode
> Pyzor: got response: Traceback (most recent call last):
> leaving helper-app run mode
> Pyzor: couldn't grok response "Traceback (most recent call last):"

> Is the last line normal, or am I missing something?
> Also spamassasin is configured to send all spam to a spam mailbox In my
> server, will pyzor do the same thing?

It's possible that pyzor choked on the test mail.  Pyzor doesn't by itself 
filter anything; it's simply used by spamassasin to determine where to 
filter.

-- 
Frank

pyzor

[Jimmy H. <jh...@ve...>]

Hello I just installed pyzor and when I run spamassassin -D < test.txt I
get the below output for pyzor.

executable for pyzor was found at /usr/bin/pyzor
Pyzor is available: /usr/bin/pyzor
entering helper-app run mode
Pyzor: got response: Traceback (most recent call last):
leaving helper-app run mode
Pyzor: couldn't grok response "Traceback (most recent call last):"


Is the last line normal, or am I missing something?
Also spamassasin is configured to send all spam to a spam mailbox In my
server, will pyzor do the same thing?




thanks

Re: Accidental listings?

[Bill W. <wo...@ne...>]

Santiago Vila <sa...@un...> wrote:

> None of the above, pyzor works on the *body* of the message, like razor.
> More precisely, on fuzzy checksums of the body.

Frank Tobin <ft...@ne...> wrote:

> There isn't really much to say except that someone else reported a
> message with an identical digest as spam.

OK, I think I get the picture now ;-). Thanks for the feedback!

-- 
Bill Wohler <wo...@ne...>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD
Maintainer of comp.mail.mh FAQ and MH-E. Vote Libertarian!
If you're passed on the right, you're in the wrong lane.

Re: Accidental listings?

[Santiago V. <sa...@un...>]

On Mon, 13 Dec 2004, Bill Wohler wrote:

> A friend of mine reported that my mail triggered a PYZOR_CHECK check.
> I read the docs on your site and was surprised since that message had
> only been sent to him. How could that message have triggered the pyzor
> check?
>
> Also, the docs on your site weren't specific about what went into your
> maps, or how to get out of them. Does pyzor map the sender's email
> address, the host, or the domain?

None of the above, pyzor works on the *body* of the message, like razor.
More precisely, on fuzzy checksums of the body.

Accidental listings?

[Bill W. <wo...@ne...>]

A friend of mine reported that my mail triggered a PYZOR_CHECK check.
I read the docs on your site and was surprised since that message had
only been sent to him. How could that message have triggered the pyzor
check?

Also, the docs on your site weren't specific about what went into your
maps, or how to get out of them. Does pyzor map the sender's email
address, the host, or the domain? In this case, how can I get
*@*.newt.com out of the map? Or is an entire block of IP addresses
mapped? In other words, did my neighbor cause trouble for me? If so, can
any of you provide any information can I give my ISP to track the
villain?

If this message also flunks the pyzor test, can any of you provide more
specific information why?

-- 
Bill Wohler <wo...@ne...>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD
Maintainer of comp.mail.mh FAQ and MH-E. Vote Libertarian!
If you're passed on the right, you're in the wrong lane.

Re: The other side of spam

[Robert P. <RPr...@As...>]

     Before I get into details, I'll state my hypothesis:

        Brighthope has a problem with Simplicato, not with
        anti-spam software.

     Have you discovered Senderbase?  It can be a very helpful resource.

        http://www.senderbase.org/search?searchString=207.99.47.53

shows this IP address on no blacklists (currently).  As has already been 
pointed out, DCC, Razor and Pyzor operate by filtering based on a 
checksum or hash of the message.  I'm not as familiar with DCC, but in 
the cases of Razor and Pyzor, messages are nominated for spam status by 
individual end users.  This doesn't seem likely in your case.

http://groups.google.com/groups?group=news.admin.net-abuse.sightings

, another very useful resource, shows no occurrences of the string 
brighthope

http://groups.google.com/groups?q=brighthope&meta=group%3Dnews.admin.net-abuse.sightings

and the hits for simplicato don't seem relevant (3 are about messages 
received at (rather the transmitted from) simplicato and the remaining 
one was transmitted from a different address at simplicato - at
http://forum.spamcop.net/forums/lofiversion/index.php/t2220.html
you can a wlandman (from simplicato?) resolving a problem at this 
address).

     So, having checked with the usual suspects, I would start looking 
elsewhere.  Some internal problem at Simplicato or a communication 
between Simplicato's machines and Brighthope's seems like a logical next 
candidate.  A virus infection in a Simplicato or Brighthope machine 
might be a possible cause.  There are probably many other possible 
causes, but these are what pop immediately to my mind.

                                                          best of luck,

                                                             Robt. P.

> I am a volunteer network admin at Bright Hope, International, a
> non-profit org whose mail domain (mail.brighthope.org) was recently
> blacklisted as spamming.  We do not send spam, but do communicate with
> our constituents via email newsletters or project-specific email
> broadcasts.  In either case, the volume of email we send is usually to
> less than 100 recipients.  However, mail.brighthope.org is hosted by
> Simplicato.com, a web- and email-hosting service that also sends many
> thousand other, unrelated messages thru the same server that we are
> assigned.
> 
> The Monday before Thanksgiving, the president of Bright Hope sent a
> message to about 85 constituents.  Shortly thereafter, he received
> notification thru various alternate channels (phone calls, his personal
> email on AOL) that constituents were unable to reply to or contact Bright
> Hope via email.  All received an automated reply indicating their email
> was blocked by SpamCop. SpamCop's blacklisting continued for 48 hours,
> but some email sent never reached brighthope.org mail accounts.
> 
> While working thru this, we discovered that Pyzor, Razor and other
> databases were also used to blacklist spam.  Apparently, many of these
> alternate blacklisting database systems, however, maintain the
> blacklisted domain names such that blocking continues until someone
> attempts to appeal or counteract or whitelist previously blacklisted
> names.
> 
> Being unfamiliar with how Pyzor functions, I am inquiring whether
> mail.brighthope.org (207.99.47.53) may be in your blacklist and if so,
> how we may appeal to take it out.
> 
> Thank you for your consideration.
> 
> Bob Biegon
> 
> 
> 
> 
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://productguide.itmanagersjournal.com/
> _______________________________________________ pyzor-users mailing list
> pyz...@li...
> https://lists.sourceforge.net/lists/listinfo/pyzor-users

Re: The other side of spam

[Santiago V. <sa...@un...>]

On Thu, 2 Dec 2004, Bob Biegon wrote:

> Being unfamiliar with how Pyzor functions, I am inquiring whether
> mail.brighthope.org (207.99.47.53) may be in your blacklist and if so, how
> we may appeal to take it out.

Pyzor is not a blacklist and does not detect spam on a per-IP basis,
but on a per-message basis.

Your best bet is to make sure you are not sending stuff to people who
have not asked to receive it. If you are not using a closed-loop
opt-in procedure for your lists, you must. Otherwise person A could
subscribe person B to your list against the will of person B.

The other side of spam

[Bob B. <bob...@br...>]

I am a volunteer network admin at Bright Hope, International, a non-profit
org whose mail domain (mail.brighthope.org) was recently blacklisted as
spamming.  We do not send spam, but do communicate with our constituents via
email newsletters or project-specific email broadcasts.  In either case, the
volume of email we send is usually to less than 100 recipients.  However,
mail.brighthope.org is hosted by Simplicato.com, a web- and email-hosting
service that also sends many thousand other, unrelated messages thru the
same server that we are assigned.

The Monday before Thanksgiving, the president of Bright Hope sent a message
to about 85 constituents.  Shortly thereafter, he received notification thru
various alternate channels (phone calls, his personal email on AOL) that
constituents were unable to reply to or contact Bright Hope via email.  All
received an automated reply indicating their email was blocked by SpamCop.
SpamCop's blacklisting continued for 48 hours, but some email sent never
reached brighthope.org mail accounts.

While working thru this, we discovered that Pyzor, Razor and other databases
were also used to blacklist spam.  Apparently, many of these alternate
blacklisting database systems, however, maintain the blacklisted domain
names such that blocking continues until someone attempts to appeal or
counteract or whitelist previously blacklisted names.

Being unfamiliar with how Pyzor functions, I am inquiring whether
mail.brighthope.org (207.99.47.53) may be in your blacklist and if so, how
we may appeal to take it out.

Thank you for your consideration.

Bob Biegon

Re: I'm listed in pyzor.sf.net?

[Kelson <ke...@sp...>]

Chris wrote:
> I don't run a mail server.

Frank Tobin wrote:
> It's possible that your logcheck output is not unique

To clarify what Frank said: Pyzor does not list email addresses or 
servers.  *You* aren't listed... a hash signature that matches the 
signature of your log message is.

-- 
Kelson Vibber
SpeedGate Communications <www.speed.net>

Re: I'm listed in pyzor.sf.net?

[Frank T. <ft...@ne...>]

Chris, on 2004-11-26, wrote:

> I don't run a mail server.  Started yesterday using fethmail to try and 
> cut some of the lagtime caused when mail is processed through kmail and 
> spamd. I hourly run a script called 'logcheck' on my syslog and have the 
> output mailed to me.  Since using fetchmail everything seems to be ran 
> through spamassassin.  My latest logcheck msg shows this:

It's possible that your logcheck output is not unique, and someone simply 
reported that digest. I recommend whitelisting reports if you can, as many 
times people's digests will end up the same.

-- 
Frank

I'm listed in pyzor.sf.net?

[Chris <cpo...@ea...>]

I don't run a mail server.  Started yesterday using fethmail to try and c=
ut=20
some of the lagtime caused when mail is processed through kmail and spamd=
=2E =20
I hourly run a script called 'logcheck' on my syslog and have the output=20
mailed to me.  Since using fetchmail everything seems to be ran through=20
spamassassin.  My latest logcheck msg shows this:

 tests=3DAWL,BAYES_50,NO_DNS_FOR_FROM,
        PYZOR_CHECK autolearn=3Ddisabled version=3D3.0.1
X-Spam-Pyzor: Reported 0 times.
X-Spam-Report:=20
        *  0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
        *      [score: 0.4958]
        *  3.5 PYZOR_CHECK Listed in Pyzor (http://pyzor.sf.net/)
        *  1.6 NO_DNS_FOR_FROM DNS: Envelope sender has no MX or A DNS=20
records
        *  1.0 AWL AWL: From: address is in the auto white-list

--=20
Chris
Registered Linux User 283774 http://counter.li.org
4:08pm up 22 days, 20:36, 2 users, load average: 0.18, 0.12, 0.16
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I'm having a RELIGIOUS EXPERIENCE ... and I don't take any DRUGS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Re: Status of project

[<lar...@fl...>]

Me too. I guess I actively submit at least 50 mails a day while I got at
least twice as many filtered out
/L


----- Original Message ----- 
From: "John Schneider" <joh...@da...>
To: "'Duane Voth'" <du...@io...>; "'Kelson'" <ke...@sp...>
Cc: <pyz...@li...>
Sent: Saturday, November 13, 2004 10:08 PM
Subject: RE: Status of project


> >> I think Mark is right.
>
> I agree.
>
> >> Furthermore, I don't think many
> >> users actively submit spam so they don't see the (still?)
> >> frequent mime attachment errors issued by the python (not
> >> pyzor) mail parsing code.
>
> I actively submit spam each and every day. With my company, it never
amazes
> me the new spam my users seem to attract. It's like flies on ****.   ;-)
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: InterSystems CACHE
> FREE OODBMS DOWNLOAD - A multidimensional database that combines
> robust object and relational technologies, making it a perfect match
> for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8
> _______________________________________________
> pyzor-users mailing list
> pyz...@li...
> https://lists.sourceforge.net/lists/listinfo/pyzor-users
>
>

RE: Status of project

[John S. <joh...@da...>]

>> I think Mark is right.  

I agree.

>> Furthermore, I don't think many
>> users actively submit spam so they don't see the (still?) 
>> frequent mime attachment errors issued by the python (not 
>> pyzor) mail parsing code.

I actively submit spam each and every day. With my company, it never amazes
me the new spam my users seem to attract. It's like flies on ****.   ;-)

Re: Status of project

[Duane V. <du...@io...>]

On Thu, Nov 11, 2004 at 11:36:28AM -0800, Kelson wrote:
> Mark Nienberg wrote:
> > I think the reason there is very little activity on the
> > list is that the code is mature and works very well for 
> > most users.  can't think of anything that would be needed
> > to improve it.  
> 
> Better error handling, perhaps?
> 
> There are still certain input errors (bad values for 
> Content-Transfer-Encoding, for instance) that cause Pyzor to crash and 
> spit out 25 lines of Python exception info instead of simply remark 
> "invalid  header" and continue on its way.  This is especially 
> frustrating when reporting an mbox file.

I think Mark is right.  Furthermore, I don't think many
users actively submit spam so they don't see the (still?)
frequent mime attachment errors issued by the python
(not pyzor) mail parsing code.  My memory of all this has
definitely faded since my mailbox has been delightfully
clean for quite a while now.  :)

Duane

Re: Pyzor server not responding (fwd)

[Francisco <li...@na...>]

Resend. Sent from wrong ID before.

> If this is the same server, why not simply replace
> 217.160.253.84:24441 with 82.165.246.93:24441 in
> http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x
> 
> Using 82.165.246.93:24441 appears to work.
> 
> Or set it back to 66.250.40.33:24441 until the issue is resolved.

Looking at the archives it seems that one neede to open the outgoing 24441 to 
66 and the incoming to 82.

It would have also helped if the docs even mentioned that it was UDP. When I 
first tried I thought it was TCP.

Moreover, even after I fixed the firewall sometimes I see timeouts on the 82 
server.

Re: Pyzor server not responding

[Francisco <fra...@na...>]

> If this is the same server, why not simply replace
> 217.160.253.84:24441 with 82.165.246.93:24441 in
> http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x
>
> Using 82.165.246.93:24441 appears to work.
>
> Or set it back to 66.250.40.33:24441 until the issue is resolved.

Looking at the archives it seems that one neede to open the outgoing 24441 
to 66 and the incoming to 82.

It would have also helped if the docs even mentioned that it was UDP. When 
I first tried I thought it was TCP.

Moreover, even after I fixed the firewall sometimes I see timeouts on the 
82 server.

RE: Pyzor server not responding

[<li...@na...>]

> This definitely needs to be fixed. I ran 'pyzor discover' the other
> day and Pyzor stopped working. I thought it was a problem with
> SpamAssassin and amavisd-new (there was a bug in 3.0.0).

Given the little response I see on the list I kind of wonder if the 
project is still active. Asked a few questions only got answer to one 
of the questions. There has been no activity in the program for several 
years and the answer I got was that it's "stable" so nothing has
needed to be done.

If I knew Python I would try and help, but to learn an entire language 
just for a single program doesn't make sense to me.

The concept is great, but I am already considering to move on.
There were also several suggestions in the list of things that could be 
improved to the program and there was no answer.

Re: Pyzor server not responding

[Gary V. <li...@jo...>]

> echo "217.160.253.84:24441" > /root/.pyzor/servers
> echo "82.165.246.93:24441" >> /root/.pyzor/servers

> pyzor ping
> 217.160.253.84:24441    TimeoutError:
> 82.165.246.93:24441     (200, 'OK')

> pyzor ping
> 217.160.253.84:24441    (200, 'OK')
> 82.165.246.93:24441     (200, 'OK')


> This definitely needs to be fixed. I ran 'pyzor discover' the other
> day and Pyzor stopped working. I thought it was a problem with
> SpamAssassin and amavisd-new (there was a bug in 3.0.0). You
> may never get a response from 217.160.253.84 unless you first
> initiate a session with 82.165.246.93. Most properly set up packet
> filters will not allow a session from a host unless it is first
> ESTABLISHED. No one who uses Pyzor should have to spend the time
> like I have trying to figure this out.

If this is the same server, why not simply replace
217.160.253.84:24441 with 82.165.246.93:24441 in
http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x

Using 82.165.246.93:24441 appears to work.

Or set it back to 66.250.40.33:24441 until the issue is resolved.

RE: Pyzor server not responding

[Gary V. <li...@jo...>]

>> pyzor --homedir /etc/mail/spamassassin/ ping
>>  82.165.246.93:24441     (200, "OK")
>>  217.160.253.84:24441    (200, "OK")

>> Tobias Stefan Richter, on 2004-11-10, wrote:

>> Is there a good reason the new server (82.165.246.93) answering requests
>> to 217.160.253.84 isn"t officially announced (via "pyzor discover" or at
>> least mailing list)?

> Currently the server is on a multihomed host; I"ll see what I can do about
> having it bind to the advertised address.
> --
> Frank


echo "217.160.253.84:24441" > /root/.pyzor/servers
echo "82.165.246.93:24441" >> /root/.pyzor/servers

pyzor ping
217.160.253.84:24441    TimeoutError:
82.165.246.93:24441     (200, 'OK')

pyzor ping
217.160.253.84:24441    (200, 'OK')
82.165.246.93:24441     (200, 'OK')


This definitely needs to be fixed. I ran 'pyzor discover' the other
day and Pyzor stopped working. I thought it was a problem with
SpamAssassin and amavisd-new (there was a bug in 3.0.0). You
may never get a response from 217.160.253.84 unless you first
initiate a session with 82.165.246.93. Most properly set up packet
filters will not allow a session from a host unless it is first
ESTABLISHED. No one who uses Pyzor should have to spend the time
like I have trying to figure this out.

Re: Status of project

[Francisco <li...@na...>]

On Thu, 11 Nov 2004, Kelson wrote:

> Mark Nienberg wrote:
>> triggers the pyzor score.  I can't think of anything that would be needed 
>> to improve it. 
>
> Better error handling, perhaps?


Agree. Maybe the errors make sense for someone 
familiar with the language that the program was developed (think it was 
python), but for others probably makes little sense.

I also mentioned a few other things that could be improved on, such as 
making it possibly to sync servers.

Re: Status of project

[Kelson <ke...@sp...>]

Mark Nienberg wrote:
> I think the reason
> there is very little activity on the list is that the code is mature and works very well for 
> most users.  A fair percentage of the spam we receive triggers the pyzor score.  I 
> can't think of anything that would be needed to improve it.  

Better error handling, perhaps?

There are still certain input errors (bad values for 
Content-Transfer-Encoding, for instance) that cause Pyzor to crash and 
spit out 25 lines of Python exception info instead of simply remark 
"invalid  header" and continue on its way.  This is especially 
frustrating when reporting an mbox file.

-- 
Kelson Vibber
SpeedGate Communications <www.speed.net>

Pyzor crashing

[Francisco <li...@na...>]

I am getting a strange error when running pyzor.

procmail: [64576] Thu Nov 11 12:34:55 2004
procmail: Locking "/home/mailuser/.pyzor/pyzor-check.lock"
procmail: Executing "/usr/local/bin/pyzor,check"
Traceback (most recent call last):
   File "/usr/local/bin/pyzor", line 4, in ?
     pyzor.client.run()
   File "/usr/local/lib/python2.3/site-packages/pyzor/client.py", line 934,
           in run ExecCall().run()
   File "/usr/local/lib/python2.3/site-packages/pyzor/client.py", line 188,
           in run  if not apply(dispatch, (self, args)):
   File "/usr/local/lib/python2.3/site-packages/pyzor/client.py", line 264,
           in check  response = runner.run(server, (digest, server))
   File "/usr/local/lib/python2.3/site-packages/pyzor/client.py", line 725,
           in run  response = apply(self.routine, varargs, kwargs)
   File "/usr/local/lib/python2.3/site-packages/pyzor/client.py", line 57,
           in check  msg = CheckRequest(digest)
   File "/usr/local/lib/python2.3/site-packages/pyzor/__init__.py", line
           381, in __init__ typecheck(digest, str)
   File "/usr/local/lib/python2.3/site-packages/pyzor/__init__.py", line
           494, in typecheck  raise TypeError
TypeErrorprocmail: Non-zero exitcode (1) from "/us

my procmail is:
LOGFILE=rc.log
VERBOSE=yes
# LOGABSTRACT=all

# Spam check with Pyzor
:0 Wc : /home/mailuser/.pyzor/pyzor-check.lock
| /usr/local/bin/pyzor check
:0 a
/home/mailuser/mail/spam



any ideas?
This setup was working ealier. I was trying on additional procmail 
recipies which were failing to do what I wanted, then decided to go back 
to the original I had (the one posted above) and started to get the pyzor 
error.

Multiple servers - my experiment

[Francisco <li...@na...>]

Since I did not get an answer to whether listing multiple servers would 
work I just tried and added the -d flag. It seems that both the check and 
the report commands report to all IPs listed in the "servers" file.

The one thing I did not see was IPs which I 
think would be helpfull.

Re: Status of project

[Francisco <li...@na...>]

On Wed, 10 Nov 2004, Mark Nienberg wrote:

> On 10 Nov 2004 at 18:47, Francisco wrote:
>> What is the status of the project?
>
> .... I think the reason
> there is very little activity on the list is that the code is mature and works very well for
> most users.  A fair percentage of the spam we receive triggers the pyzor score.  I
> can't think of anything that would be needed to improve it.

Documentation can be improved.
For instance there are features that may even exist, that one doesn know 
about.

For example I asked on the list, without answer, what happens when 
there are multiple servers listed in the "servers" file?
Ideally the client would support multiple servers so once cour report spam 
to both the public and a private server. At the same time one would like 
to have the same arrangement at checking time. This way one could check 
the local server first and if no match is found then check the public 
server.

Next would be to be able to sync between servers. Perhaps an export/import 
function for the server. An even simpler to allow servers to import the 
log file from another server. Then post weekly the public server log.

I would be willing to help with documentation, hosting a mirror of the log 
or even becoming the main site for the log. I am sure others would not 
mind to mirror.

The ability to make the system more distributed will greatly help the 
system since it depends on having a good feed of spam.

AW: Pyzor server not responding

[<Fra...@co...>]

> > proxy:~ # pyzor ping
> > 217.160.253.84:24441    TimeoutError:
> 
> sch...@ww...:~$ pyzor ping
> 217.160.253.84:24441    (200, 'OK')
> 

Without comment.......

more /etc/mail/spamassassin/servers
82.165.246.93:24441
217.160.253.84:24441

pyzor --homedir /etc/mail/spamassassin/ ping
82.165.246.93:24441     (200, 'OK')
217.160.253.84:24441    (200, 'OK')

vi /etc/mail/spamassassin/servers
more /etc/mail/spamassassin/servers
217.160.253.84:24441

pyzor --homedir /etc/mail/spamassassin/ ping
217.160.253.84:24441    TimeoutError:

vi /etc/mail/spamassassin/servers
more /etc/mail/spamassassin/servers
82.165.246.93:24441
217.160.253.84:24441

pyzor --homedir /etc/mail/spamassassin/ ping
82.165.246.93:24441     (200, 'OK')
217.160.253.84:24441    (200, 'OK')