[Sqlgrey-users] impressive results from using memcached

[Michael S. <Mic...@lr...>]

Hi *,

I've again played around with my special version of sqlgrey and made a
prototype implementation for using memcached in the last days. This is the
statistc I got from running this code over the last 37 hours with my
spamtrap system:

    type       requests            hit             miss
=========================================================
  domain_awl      17368        177   1 %      17191  98 %
     dsn_awl       1052        342  32 %        710  67 %
    from_awl      17188        163   0 %      17025  99 %
    rcpt_awl      17368          0   0 %      17368 100 %
    throttle     166355     139594  83 %      26761  16 %
     unknown     168870     150450  89 %      18420  10 %

Some remarks:

- my spamtrap system cannot be compared to a normal production system.
  This is a virtual email server which accepts only emails for some
  domains, which are no longer in use for more than 10 years. Therefore
  there should be no ligit emails coming in, only spam mails and DSNs for
  spam mails which used our old domains as originator. The rcpt_awl
  is empty and all the entries in domain_awl and from_awl are from spam
  mails, which were able to get through greylisting.

- the number of processed emails are the emails which are whitelisted plus
  the 168870 emails which are checked for ip state unknown.

- unknown is a new state. It means there is no entry in my 4 AWLs
  domain_awl, dsn_awl, from_awl, rcpt_awl with the ip address of this
  triplet. Therefore sqlgrey can skip the checks for these AWLs and
  immediately check for a reconnect.

The statistic means, for 83 % of the triplets no check of the AWLs was
made and for 83 % there were only 1 SQL SELECT and 2 memcached checks
needed and then immediately reject.

I suppose the situation could be similar with a production system and
therefore the usage of memcached especially for the 2 states unknown and
throttle would give Sqlgrey a real performance boost.

Michael Storz
PS: Lionel, can you now imagine why I found the mydie error? :-)
--
======================================================
Leibniz-Rechenzentrum      |    <mailto:St...@lr...>
Boltzmannstr. 1            |    Fax: +49 89 35831-9700
85748 Garching / Germany   |    Tel: +49 89 35831-8840
======================================================