[bwm-tools-tech] Where do bwmd processed packets get sent next?

[Andrew K. <and...@ad...>]

Hello,

I have bwm_tools-0.3.0 running on CentOS 5 with kernel 2.6.18-53.1.14.el5 SMP 
i686/i386 (32-bit). There were some minor bumps, but I got it working in an 
evening.

This is a test installation on a host with a single interface.

On the filter table, we can jump packets from the end of the INPUT chain to the 
bwmd chain where they get queued out to bwmd in userspace. All packets incoming 
to the host are properly rate limited.

However, we cannot do the same with the OUTPUT chain of the filter table. If we 
jump those packets over to bwmd, bwmd starts spewing messages related to 
truncation. There is an initial burst of data let through, then the stream  
sputters and dies. It looks to me like it creates a loop for the data which 
causes trouble. It's only processing one 256Kbit/sec flow via a fast ethernet 
interface on a Celeron 2.53GHz (Pentium D generation) with 512MB RAM. The 
machine is only running bwm_tools and is dedicated to this testing process. Our 
production machine will be much beefier and will have multiple gigabit 
interfaces run in bridge mode (packets hitting the FORWARD chains mainly).

So, my question is, where does bwmd reinsert the packets that it processes?

As I understand it, here is the path a packet and its reply takes without bwmd. 
We're starting with the packet coming in the interface, going to the 
application, and then back out the interface (flowing top to bottom, input 
towards output). I've excluded NAT because we're not using it right now.

wire in
eth0
raw PREROUTING
conntrack
mangle PREROUTING
routing
mangle INPUT (here we mark inbound packets for bwmd)
filter INPUT (at the end of this chain, we jump to bwmd)
application (browser, ftp client, web server, whatever)
---
application
routing
raw OUTPUT
conntrack
mangle OUTPUT (here we mark outbound packets for bwmd)
routing (second pass in case mangle changed something)
filter OUTPUT (logically, this is what should jump to bwmd, but doesn't work)
mangle POSTROUTING
eth0
wire out

I looked through the code and it was obvious to me that it just waits for 
packets, checks link budgets, trims flows, but then I lose track of where that 
data exits bwmd and heads back into the networking stack of the OS. I'm sure I 
just missed it while skimming the code.

Can anyone lend some insight on this? To me, it is most important that it work 
well with the FORWARD chains, which seems to be the original design intent, but 
it would be nice if it could be used for limiting flows originating/terminating 
on the host itself, as well.

Thanks in advance. If I've boneheaded something, please be gentle. If this 
suits our basic needs, we may very well fund future development to smooth some 
rough edges and extend the features (particularly, support for more threads). 
Heck, I might even write a little code and/or documentation.

Sincerely,
Andrew Kinney
CTO, Advantagecom Networks
http://www.simplywebhosting.com