[Divert] Re: Can Divert Sockets Solve My Problem

[Ilia B. <ib...@nc...>]

VLAN might be one option for you I think (IEEE 802.1q),
although it does not do exactly what you describe, it
is intended for bandwidth sharing on a single physical
link between (possibly large) number of logical subnets.

it has been under development for Linux, I believe it
works to a degree - not totally sure how far they are
as I have not been keeping up.

Your problem sounds like something that can be used for
DDoS prevention - am I far off?

The short answer to your question is - its possible but may
not be advisable in production environment. Look further 
for answers to specific questions and you'll understand.

Gilbert C Healton wrote:
> 
> I've been approaced by someone I know to find a solution to an
> interesting problem.  Divert sockets look like a possible solution
> (any other known solutions would be appreciated as well).
> 
> I've read all the documentation on the web site so I hope I haven't
> missed anything.
> 
> PROBLEM
> A Linux box serving as a gateway to a number of computers
> has a normal connection to the outside world that we whish to control
> but expensive links to the computers on the subnet we are performing
> router services for.
> 
> IMPORTANT POINTS
> 
>   o There can be over 1000 computers on the controlled subnet.
> 
>   o Each subnet IP address has a priority in the maximum amount of
>     bandwidth it can use in a short period of time (do not have
>     to be super close here, like the linux traffic shaper is, but
>     the idea is the same). I would be happy to let them burst over
>     at high speed until a quota is exceeded, then stop serving them
>     till enough time goes by to give them another burst. A resolution
>     of one second should be suffcient though I would prever 0.25
>     second.
> 
>   o The restrictions must be by IP address. Each IP address on
>     the controlled network is independent of each others configuration.
> 
>   o I must also validate users on the controlled network before letting
>     them on the expensive link. A web login should be suffcient
>     (fairly trivial, but be aware of it).
> 
>   o The number of users on the subnet can approach 1000. Using
>     the traditional Linux traffic shaper would require TWO shaper
>     devices for each IP address (one for each direction). Along with
>     vastly exceeding the network device limit of 100 in
>     kernel/net/core/dev.c, I shutter to thing what 500+ so network
>     devices would do to router software (zebra), etc., trying to
>     cope with all those entries.
> 
> As I understand Divert Sockets:
> 
>   o I could capture incoming and outgoing operations on the main link.
>     Controlling the speed of the traffic on it would effectively
>     reduce the speed of transmissions on the expensive link.
> 
> QUESTIONS:
> 
>   1. The sample code only diverted port 4321... is there a way to divert
>      all ports for the common link using a single process?
> 


That was the divert port the code diverted packets *to*. You can 
divert any type of traffic to any number of divert ports 
(including just one). It all depends on the specification you give 
to the flow in the firewall.

You can for instance divert all TCP packets into a single divert port
and all UDP packets into another (something you might have to do:
see below on flow control)


>   2. If not, can I fake it somehow by starting diversions on ports
>      as required BEFORE I need the information?  I think I could live
>      with a separate process for each diverted port.
>

I think the above answers that, but you can also do what you are
describing.
 
>   3. While I haven't been involved in IP on low levels, I have been
>      extensively involved in propritary protocols for special hardware.
>      I've been looking through various TCP/IP documents. Some of which are at
>      http://www.exit109.com/~ghealton/.computer_inet.html#@Technical.TCP.IP
> 
>      Any other really good documents?
>

I go for RFCs myself. That and Stevens's TCP/IP and UNIX Network Prgrmg
books.

 
>   4. If I decide to slow down outgoing packets over the main link, is
>      there a way I can tell the sender to slow down as not to fill
>      the router's internal buffers during large file transfers?
>      Or will this not be an issue under TCP/IP?
>

You are talking about flow control and it depends whether the
traffic is connection-oriented (TCP) or not. With TCP there
are a number of mechanisms you can use to control the sending
rate (ECN [Early Congestion Notification] is one of the newer 
ones, but you can also use MSS and window adjustment within TCP flow,
by overwriting parts of the TCP header in response packets)
With datagram-based flows the applications typically excercise
their own flow control and it could be harder.
 
> NOTES:
> 
> If I use Divert Sockets, I'll likely contribute additional documentation
> to the project, if not additional code.
>

Always appreciated...
 
> My "customer" would like something running next month, even if less
> than complete. Speed control is the largest issue.
> 

OK, so maybe I did not totally answer your first question, so here
it is: divert sockets are a user-space mechanism with all the
trappings, including kernel-space to user-space and back copying
of packets upon interception and reinsertion, which can get 
expensive on a high-throughput system.

While I have no doubt you can get a prototype going in no-time if
your outgoing link speed is under 100Mbps with a decent PIII in
the GHz range and lots of memory, your ultimate solution would
lie in writing a kernel module to do all of this work. 

I am starting work on the divert sockets for 2.4. This new kernel
has a number of mechanism that can be helpful to you. One is
netfilter - an API framework *within* the kernel to grab and
insert packets in a number of points along the protocol stack
(not necessarily IP only). So your final solution would be a 
module that would use netfilter. Hope I answered your question.
Let me know if you have any more. Your problem sounds like an
interesting one...

-ilia

PS: http://netfilter.samba.org/ for 2.3 and 2.4 kernels netfilter
-------------------------------------------------------------------------
Divert Sockets for Linux List
mailto:di...@li...
http://www.anr.mcnc.org/~divert
-------------------------------------------------------------------------