Re: [Sqlgrey-users] idea for improving Discrimination feature

[Riaan K. <ria...@gm...>]

On 04/02/07, Riaan Kok <ria...@gm...> wrote:
> On 02/02/07, Dan Faerch <da...@ha...> wrote:
> > Riaan Kok wrote:
> > > a case that I saw in the logs intrigued me, so I did a quick lookup of
> > > the
> > > keys() function and read a bit about hashes..  Perl's hash structure
> > > is not
> > > ordered in any way.  So, iterating through a hash returns the elements in
> > > undefined order..
> >
>
> It's just that, if you're curious about statistics, the random order
> of the hash list of rules means that the only way of knowing what
> percentage of connections get nailed by a rule is to have only one
> rule.
>
> Another advantage of knowing the order in which rules will execute is
> that, in production, you can place cheap and broad rules first, and
> more expensive rules last (such as that badass rule for catching
> dynamic IP client hostnames in dyn_fqdn.regexp).  If your traffic is
> sufficient, your CPU might just appreciate it..
>
> > > So, how about rather storing the list of rules in an array, which does
> > > away
> > > with the need for storing the $rulenr, and each array item like $rule
> > > containing:
> > > $rule->{attrib}
> > > $rule->{oper}
> > > $rule->{regexp}
> > Hmm well.. Its seems like a lot of work for a very small result. I dont
> > think ill be coding this anytime soon ;).. But if youre a perl coder,
> > patches are welcome.
>


Okay, I finally got some time to do my proposed change.  It wasn't as
bad as I thought it might be.  The attached patch is against Sqlgrey
1.7.5.  I had to do some tricks to get a clean patch, but the code in
my sqlgrey has been running fine now for a day (in the order of 100k
connections/day).

Some day when I get around to playing with my rules and stats I'll
post if there's anything interesting.

regards,
Riaan