#109 Transparent proxying support?
Hi, how about adding support for transparent proxying?
e.g. perl equiv from a perl pop3proxy I wrote
#$daddr:$dport = where the proxy will actually make
connections to
#$FORCEDEFSVR is whether to force usage of the
configured default server
#$USE_SO_ORIGINAL_DST is whether to do Linux-style
transparent proxying or FreeBSD style
# $SO_ORIGINAL_DST = 80 if linux headers not available
#$client holds the inbound socket object
# $BOUNDPORT and $BOUNDADDRESS = actual addr and
port proxy is listening on
#should probably do more checks to stop loops
#e.g. enforce bound addr:port != default addr:port
if ($FORCEDEFSVR) {
$daddr=$DEFADDRESS;
$dport=$DEFPORT;
} else {
if ($USE_SO_ORIGINAL_DST) {
my $p= getsockopt
$client, $SOL_IP, $SO_ORIGINAL_DST;
$daddr=inet_ntoa(substr
($p,4,4));
$dport=unpack('n',substr
($p,2,2));
} else {
#FreeBSD style
$daddr=$client->sockhost;
$dport=$client->sockport;
}
#dolog('DEBUG', "dest:
$daddr:$dport");
if (($dport==$BOUNDPORT) and
($daddr eq $BOUNDADDRESS)) {
# If seem to be connecting to
self connect to default server instead
$daddr=$DEFADDRESS;
$dport=$DEFPORT;
}
}
--
I don't know python, but I might see if I can figure out
spambayes+ python. Feature may be harder to add than
I think ;).
Logged In: YES
user_id=552329
My Perl is extremely rusty. Exactly what do you mean by
transparent proxying? i.e. what changes do you want from
the current proxying?
Logged In: YES
user_id=1060689
Transparent proxying will require configuration of Linux
iptables or FreeBSD IPFW to redirect desired TCP connections
to the proxy.
The proxy will then figure out the actual intended destination
addr and port as per the code above, make the connections
and do stuff - spamfiltering, protocol enforcement.
NOTE: if it is possible to tell spambayes to connect to a
particular popserver by using the following USER command
then the need for a transparent proxy may be less, and one
could also more easily chain proxies.
e.g.
USER user@name:popserver1
If the feature is active that should tell spambayes to connect
to popserver1 and send
USER user@name
The delimiter ":" should be configurable. Not sure what would
be best - maybe looking at the wrong RFC.
Logged In: YES
user_id=552329
I'm not familiar with transparent proxying, so forgive me if
these are obvious questions:
1. Can this be done with Windows as well as Linux? There's
a minimum of platform specific code in sb_server at the
moment, and it's probably good to stay that way.
2. Does this offer any significant benefit over simply
reconfiguring the client (either manually as at the moment,
or automatically)? Python likes explicit over implicit, and
this doesn't seem very explicit.
SpamBayes chose to not use the USER method of electing which
server to connect to, and has one proxy port for each server
instead. (Although the pspam code that's in the source dist
does use this method). This was decided a long time back,
and is highly unlikely to change now. You'd have to put
forth some pretty compelling arguments to do so.
Logged In: YES
user_id=85414
[Tony]
> SpamBayes chose to not use the USER method of electing
> which server to connect to
The main reason for this is that the client can send requests
before the USER request. For instance, it can issue a CAPA
to ask the server which POP3 extensions it supports. If you
don't know which real server to connect to until the USER
request, you can't serve the CAPA request (or any others
that are made before USER).
Logged In: YES
user_id=1060689
OK, the support for CAPA, APOP etc is a valid reason to not
support the proxy chaining with USER.
Back to transparent proxying. With transparent proxying the
mail clients can retain their mail settings as it is.
What typically happens is the network is set up so that pop3
requests pass through a server (e.g. firewall) and are
intercepted by the firewall rules and forwarded to a proxy
listening on a local address:port - e.g. 127.0.0.3:10110 or
something.
The proxy figures out the original intended destination, makes
the connections and proxies between the client and the
destination. The client thinks it is talking directly to the
destination (connection end point = dest IP:port), and in
most setups the destination thinks it's talking to the proxy.
If the proxy is down, the firewall rules could be changed to
pass through the connections.
Figuring out the original intended destination can be platform
dependent. Apparently earlier versions of Linux were like
FreeBSD. Newer versions of Linux require you to use the
socket/IP options thing to get it.
I'm not sure about Windows.
Other issues: is it important for spambayes to distinguish one
user-popserver pair from another? Transparent proxying could
either complicate that, or the proxy admin will have to train
and configure spambayes based on all messages passing
through the proxy for all users and servers.
It is in theory possible to set things up so that say a hundred
different users use the same transparent proxy, but still get
to administer and train their own "proxy". This could be done
by creating spambayes accounts based on a successful pop3
login (perhaps only if automatic creation is enabled). If there's
no spambayes account (either manually/automatically
created), then the proxy goes to a "pass through" mode - no
filtering or parsing. Alternatively an admin could administer on
behalf of all users. Spambayes for the masses.
That said, there may be performance and memory issues. Well
there'd be another use for those multiGHz 64 bit CPUs eh?
BTW spambayes takes 40MB of mem in peak usage on my
Win2K machine. Is that shared amongst multiple connections
or is that per connection? Fortunately I'm not using WinXP -
based on some personal/anecdotal evidence it seems to have
a memory handling algorithm that fails more disgracefully than
W2K.
Logged In: YES
user_id=552329
I suspect that this is something that will only get added if
lots of people ask for it or if someone supplies a patch.
In any case, this is certainly a feature request and not a
bug, so I'm changing the type.
demo patch for transparent proxying
Logged In: YES
user_id=1060689
OK I just downloaded the spambayes 1.0.4 source and
modified it to support transparent proxying for my FreeBSD
machine. I've attached a "demo" patch. Please note that I
don't know python and this is my first (or 2nd?) go at
writing python, so the patch is probably far from "best
practices".
For FreeBSD you can use getsockname() for getting the
original destination address:port for connections that
are "fwd"ed by ipfw. That's what I have done.
For Linux you can't do that, you need to getsockopt
clientsock, SOL_IP, SO_ORIGINAL_DST. I have not added
support for this in the patch. Some real python programmer
should do it ;).
My demo patch forces the destination port to be 110. This
is because I'm using tcp port 7000 for test purposes.
In the demo config below if the pop3 client tries to
connect to serverA:7000 they will get transparently
connected to spambayes on tcp/5000 which then connects to
serverA:110
If the client tries to connect to serverA:110 that firewall
rule won't match, so they will connect straight to
serverA:110 (unless there are other firewall rules).
Commandline:
sb_transserver.py -T
bayescustomize.ini
[pop3proxy]
listen_ports: 5000
remote_servers: 127.0.0.1
allow_remote_connections:192.168.1.*
Freebsd config:
Add something like following line to /etc/rc.firewall
${fwcmd} add fwd 127.0.0.1,5000 tcp from 192.168.1.0/24 to
any 7000 keep-state
Issues:
Privacy and security. One should probably turn off caching
and web configuration support. I'm not sure how to do that.
I have not really tested multiple concurrent users, so
performance, reliability and stability is not proven, nor
is data integrity confirmed.
A different command line opt other than -T may be desirable.
Notes:
I used a pretrained hammie.db and it seems to work.
With this, the clients can be on windows and the spambayes
server on FreeBSD, and all one needs is to configure the
client mail filters to use the "X-Spambayes-Classification"
headers.
This should also work on other *BSD. But I have not tested
it.