Re: [Assp-test] Assp dying

[GrayHat <gr...@gm...>]

>> Strange, considering that if you're using A/MX, PTR, DNSBL,URIBL
>> and SenderBase checks the lookups should be much more than those,
>> it would be a good idea using the "querylog" option in "rndc" to
>> enable query logging in BIND and then keeping an eye (e.g. using
>> tail) on the live logs;

> That is exactly what I was doing.  I have query logging in in
> named.conf set to rotate out at 10M and keep 5 copies.

Hm... try using "trace++" to increase the trace level in BIND, you'll
end
up with bigger log files but you'll also be able to see if something
isn't
working as expected

>> see, judging from your figures I suspect that a lot of your DNS
>> queries are timing out; as for memory/CPU, BIND is quite a PIG,

> I have named/BIND on the same machine as the MTA, and there
> are minimal timeouts, usually just when a DNSBL goes down and
> out of service.

I meant timeouts in BIND, not between BIND and ASSP; what I suspect
is that your named may be sending out queries but then answers time
out and get discarded so ASSP doesn't record any timeout but BIND
does; that's why I'm suggesting to increase logging in BIND; also keep
in mind that some/many DNSBLs/URIBLs have query rate limiters in
place so, if you generate "high traffic" you'd better obtain a paid
account
with those lists to avoid the capping or to be able to use a local copy
of
the lists (so speeding up lookup and decreasing bandwidth usage)

> If I was timing out, I would get too much spam.  My DNSBL's and
> WL's seem to be working pretty good for me.

Yet, given your figures the query rate seems quite unbalanced and
that may be worth investigating


>> you need a resolver I'd suggest you looking at
http://www.unbound.net/
>> which is far MORE efficient (and fast) than the BIND dinosaur <g>

> Thanks, I will take a look.  I do run rbldnsd for some internal
> DNSBL's of my own that I maintain, just for myself, because it is
> easier for me to manage those in a lttle web interface I have built.
> That is a fast DNS resolver, but also not a full blown DNS resolver,
> and more specifically made for BL's.

Yes, rbldnsd is a good choice, it allows you to keep local copies of
DNS blacklists maximizing lookup speeds; on the other hand "unbound"
is a full blown regular DNS resolver, it doesn't work as an
"authoritative"
but is quite efficient and solid as a resolver; so the idea may be
setting
up unbound on the same box on which are you running ASSP (assuming
you can't set it up elsewhere) and then configuring it to work as a full
recursive resolver for ASSP and to forward queries toward blacklists
which you're hosting on your local rbldnsd to those servers, that will
allow you to maximize speed and reduce traffic and response times

> But hey, if it is working for SORBS and Spamhaus, then it should be
> good enough for my meager needs.  I have found named/BIND to be a
> little resource heavy after a point, but for the most part, it has

again, BIND is a full recursive/authoritative DNS, rbldnsd is a
different
critter it was born EXACTLY to serve blacklist zone data and NOT to
act as a full resolver, it's perfect for SORBS, spamhaus and others
since they serve blacklist data, but it isn't suitable to work as a DNS
resolver and you'll need BOTH

> that ASSP needs, like guaranteed data integrity, etc.

Data integrity and isolation come first when you have to access the
data from multiple, independent threads otherwise you may end up
with a totally screwed bunch of data :)

>> and btw MySQL is just one of the possible choices, but a good
>> one I've to say, others may be PicoSQL
http://www.picosoft.it/picosql/
>> or FireBird http://www.firebirdsql.org/ but then you'll probably have
to

> Cool, I will look at both of those when I have a chance to eval some
> other database options.

Not saying they're the "best of the best" just that both may be good
alternatives; the Pico may be suitable for smaller setups due to its
reduced requirements, firebird may be ok for bigger setups due to
all its features, speed and solidity (and btw it's FULLY opensource
and Oracle didn't buy it ... yet <g>)

> "going away" because the query was taking too long. I was doing
> a union'd select with a date range limiter across 2+ million records,
> and not a lot of placed to index on for best performance.

well... try giving FireBird a spin then decide by yourself... but I'll
stop
this here since we already are OT; getting back to ASSP I think you
may need to revise your setup/infrastucture a little to ensure your
ASSP(s) will be able to sustain the load, not that it can't do that as
it does, but why stretching it too much when you may have it humming
along and avoiding some possible pitfalls/bottlenecks ?