linux-l7sw-devel — Development Stuff

Re: [Linux-l7sw-devel] [PATCH] Linux 2.6.25 Support

[Alexandre L. <ale...@sm...>]

Le Sunday 07 September 2008 21:23:33, vous avez écrit :
> hi guys,
>
> Well, Ok.... I really need to find time :((((   have lot of pending
> patch for keepalived too...
>
> can you repost please
>
> cheers,
> Alexandre
>
> On Sep 7, 2008, at 7:14 PM, Willy Tarreau wrote:
> > On Sun, Sep 07, 2008 at 07:09:07PM +0200, Alexandre LISSY wrote:
> >> Hi,
> >>
> >> I had to test TCP Splicing using a 2.6.25 debian kernel, so I had to
> >> update the latest 0.1.2 patch available to compile against this new
> >> kernel.
> >>
> >> It works, with few connections. I ran into problems with many
> >> simultaneous connections, but I had the same issue with a 2.6.18
> >> (debian) and the original patch.
> >>
> >> Anyway, as I got no answer from Alexandre Cassen, I post it here,
> >> maybe
> >> it might help somebody.
> >>
> >> [Attachment stripped: Original attachment type: "text/x-diff",
> >> name: "layer7switch-2.6.25.patch"]
> >
> > Alexandre,
> >
> > there was a problem with your patch, it's empty. See above, it is
> > indicated
> > "attachment stripped". Could you please repost ?
> >
> > Thanks,
> > Willy
> >
> >
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> > Build the coolest Linux based applications with Moblin SDK & win
> > great prizes
> > Grand prize is a trip for two to an Open Source event anywhere in
> > the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Linux-l7sw-devel mailing list
> > Lin...@li...
> > https://lists.sourceforge.net/lists/listinfo/linux-l7sw-devel

My fault, I didn't checked the patch before sending it, and I've been fooled 
by the IMAP server :)

Here is the good file.

Re: [Linux-l7sw-devel] [PATCH] Linux 2.6.25 Support

[Alexandre L. <ale...@sm...>]

Le Sunday 07 September 2008 21:23:33, vous avez écrit :
> hi guys,
>
> Well, Ok.... I really need to find time :((((   have lot of pending
> patch for keepalived too...
>
> can you repost please
>
> cheers,
> Alexandre
>
> On Sep 7, 2008, at 7:14 PM, Willy Tarreau wrote:
> > On Sun, Sep 07, 2008 at 07:09:07PM +0200, Alexandre LISSY wrote:
> >> Hi,
> >>
> >> I had to test TCP Splicing using a 2.6.25 debian kernel, so I had to
> >> update the latest 0.1.2 patch available to compile against this new
> >> kernel.
> >>
> >> It works, with few connections. I ran into problems with many
> >> simultaneous connections, but I had the same issue with a 2.6.18
> >> (debian) and the original patch.
> >>
> >> Anyway, as I got no answer from Alexandre Cassen, I post it here,
> >> maybe
> >> it might help somebody.
> >>
> >> [Attachment stripped: Original attachment type: "text/x-diff",
> >> name: "layer7switch-2.6.25.patch"]
> >
> > Alexandre,
> >
> > there was a problem with your patch, it's empty. See above, it is
> > indicated
> > "attachment stripped". Could you please repost ?
> >
> > Thanks,
> > Willy
> >
> >
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge
> > Build the coolest Linux based applications with Moblin SDK & win
> > great prizes
> > Grand prize is a trip for two to an Open Source event anywhere in
> > the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > _______________________________________________
> > Linux-l7sw-devel mailing list
> > Lin...@li...
> > https://lists.sourceforge.net/lists/listinfo/linux-l7sw-devel

My fault, I didn't checked the patch before sending it, and I've been fooled 
by the IMAP server :)

Here is the good file.

Re: [Linux-l7sw-devel] [PATCH] Linux 2.6.25 Support

[Alexandre C. <ac...@gm...>]

hi guys,

Well, Ok.... I really need to find time :((((   have lot of pending  
patch for keepalived too...

can you repost please

cheers,
Alexandre



On Sep 7, 2008, at 7:14 PM, Willy Tarreau wrote:

> On Sun, Sep 07, 2008 at 07:09:07PM +0200, Alexandre LISSY wrote:
>> Hi,
>>
>> I had to test TCP Splicing using a 2.6.25 debian kernel, so I had to
>> update the latest 0.1.2 patch available to compile against this new
>> kernel.
>>
>> It works, with few connections. I ran into problems with many
>> simultaneous connections, but I had the same issue with a 2.6.18
>> (debian) and the original patch.
>>
>> Anyway, as I got no answer from Alexandre Cassen, I post it here,  
>> maybe
>> it might help somebody.
>
>> [Attachment stripped: Original attachment type: "text/x-diff",  
>> name: "layer7switch-2.6.25.patch"]
>
> Alexandre,
>
> there was a problem with your patch, it's empty. See above, it is  
> indicated
> "attachment stripped". Could you please repost ?
>
> Thanks,
> Willy
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Linux-l7sw-devel mailing list
> Lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-l7sw-devel

Re: [Linux-l7sw-devel] [PATCH] Linux 2.6.25 Support

[Willy T. <w...@1w...>]

On Sun, Sep 07, 2008 at 07:09:07PM +0200, Alexandre LISSY wrote:
> Hi,
> 
> I had to test TCP Splicing using a 2.6.25 debian kernel, so I had to 
> update the latest 0.1.2 patch available to compile against this new 
> kernel.
> 
> It works, with few connections. I ran into problems with many 
> simultaneous connections, but I had the same issue with a 2.6.18 
> (debian) and the original patch.
> 
> Anyway, as I got no answer from Alexandre Cassen, I post it here, maybe 
> it might help somebody.

> [Attachment stripped: Original attachment type: "text/x-diff", name: "layer7switch-2.6.25.patch"]

Alexandre,

there was a problem with your patch, it's empty. See above, it is indicated
"attachment stripped". Could you please repost ?

Thanks,
Willy

[Linux-l7sw-devel] [PATCH] Linux 2.6.25 Support

[Alexandre L. <ale...@sm...>]

Hi,

I had to test TCP Splicing using a 2.6.25 debian kernel, so I had to 
update the latest 0.1.2 patch available to compile against this new 
kernel.

It works, with few connections. I ran into problems with many 
simultaneous connections, but I had the same issue with a 2.6.18 
(debian) and the original patch.

Anyway, as I got no answer from Alexandre Cassen, I post it here, maybe 
it might help somebody.

[Linux-l7sw-devel] make error

[<j-...@16...>]

bGludXgtbDdzdy1kZXZlbKOsxPq6w6OhDQoNCmNlbnRvcy00LjQNCkkgY2FuJ3QgbWFrZSBpdDoo
DQoNCltyb290QHRlc3Qgc3dpdGNoZF0jIG1ha2UNCm1ha2VbMV06IEVudGVyaW5nIGRpcmVjdG9y
eSBgL2hvbWUvcGtnL2xheWVyN3N3aXRjaC0wLjEuMi91c2VybGFuZC9saWInDQptYWtlWzFdOiBO
b3RoaW5nIHRvIGJlIGRvbmUgZm9yIGBhbGwnLg0KbWFrZVsxXTogTGVhdmluZyBkaXJlY3Rvcnkg
YC9ob21lL3BrZy9sYXllcjdzd2l0Y2gtMC4xLjIvdXNlcmxhbmQvbGliJw0KbWFrZVsxXTogRW50
ZXJpbmcgZGlyZWN0b3J5IGAvaG9tZS9wa2cvbGF5ZXI3c3dpdGNoLTAuMS4yL3VzZXJsYW5kL3N3
aXRjaGQvc3JjJw0KZ2NjIC1nIC1PMiAtSWluY2x1ZGUgLUkuLi8uLi9saWIgLUkuLi8uLi8uLi9r
ZXJuZWwvYnVpbGQgLUkuLi8uLi9saWJ0Y3BzcGxpY2UgLVdhbGwgLVd1bnVzZWQgLVdzdHJpY3Qt
cHJvdG90eXBlcyAgLWMgbWFpbi5jDQpJbiBmaWxlIGluY2x1ZGVkIGZyb20gL3Vzci9pbmNsdWRl
L2xpbnV4L3RjcC5oOjIxLA0KICAgICAgICAgICAgICAgICBmcm9tIG1haW4uYzoyMjoNCi91c3Iv
aW5jbHVkZS9hc20vYnl0ZW9yZGVyLmg6NjoyOiB3YXJuaW5nOiAjd2FybmluZyB1c2luZyBwcml2
YXRlIGtlcm5lbCBoZWFkZXI7IGluY2x1ZGUgPGVuZGlhbi5oPiBpbnN0ZWFkIQ0KSW4gZmlsZSBp
bmNsdWRlZCBmcm9tIC91c3IvaW5jbHVkZS9hcnBhL2luZXQuaDoyMywNCiAgICAgICAgICAgICAg
ICAgZnJvbSAuLi8uLi9saWIvdXRpbHMuaDoyNiwNCiAgICAgICAgICAgICAgICAgZnJvbSBtYWlu
LmM6Mjg6DQovdXNyL2luY2x1ZGUvbmV0aW5ldC9pbi5oOjM1NDogZXJyb3I6IHN5bnRheCBlcnJv
ciBiZWZvcmUgJygnIHRva2VuDQovdXNyL2luY2x1ZGUvbmV0aW5ldC9pbi5oOjM1NDogZXJyb3I6
IHN5bnRheCBlcnJvciBiZWZvcmUgIl9fdTMyIg0KL3Vzci9pbmNsdWRlL25ldGluZXQvaW4uaDoz
NTU6IGVycm9yOiBzeW50YXggZXJyb3IgYmVmb3JlICcoJyB0b2tlbg0KL3Vzci9pbmNsdWRlL25l
dGluZXQvaW4uaDozNTU6IGVycm9yOiBzeW50YXggZXJyb3IgYmVmb3JlICJfX3UxNiINCi91c3Iv
aW5jbHVkZS9uZXRpbmV0L2luLmg6MzU3OiBlcnJvcjogc3ludGF4IGVycm9yIGJlZm9yZSAnKCcg
dG9rZW4NCi91c3IvaW5jbHVkZS9uZXRpbmV0L2luLmg6MzU3OiBlcnJvcjogc3ludGF4IGVycm9y
IGJlZm9yZSAiX191MzIiDQovdXNyL2luY2x1ZGUvbmV0aW5ldC9pbi5oOjM1OTogZXJyb3I6IHN5
bnRheCBlcnJvciBiZWZvcmUgJygnIHRva2VuDQovdXNyL2luY2x1ZGUvbmV0aW5ldC9pbi5oOjM1
OTogZXJyb3I6IHN5bnRheCBlcnJvciBiZWZvcmUgIl9fdTE2Ig0KbWFrZVsxXTogKioqIFttYWlu
Lm9dIEVycm9yIDENCm1ha2VbMV06IExlYXZpbmcgZGlyZWN0b3J5IGAvaG9tZS9wa2cvbGF5ZXI3
c3dpdGNoLTAuMS4yL3VzZXJsYW5kL3N3aXRjaGQvc3JjJw0KbWFrZTogKioqIFthbGxdIEVycm9y
IDEJDQoNCg0KoaGhoaGhoaGhoaGhoaGhodbCDQrA8aOhDQogCQkJCQ0KDQqhoaGhoaGhoaGhoaGh
oaGhvarOxLawDQqhoaGhoaGhoaGhoaGhoaGhai13ZEAxNjMuY29tDQqhoaGhoaGhoaGhoaGhoaGh
oaGhoTIwMDctMDQtMDINCg==

[Linux-l7sw-devel] HTTP parsing via regex (libpcre3) in switch

[Momtchil M. <mom...@li...>]

    Hello,

    I've added a (somewhat primitive) support for scheduling via regex 
matching on the HTTP session, allowing for session support by matching 
the server cookies (or scheduling according to the URL). I've used 
libpcre3, which supports both NFA et DFA regex matching.

    I'm currently looking to add hierarchical scheduling (so that you 
can do round-robin over the regex scheduling) and WSCALE support to 
tcpsplice.

    If you decide to test it, I'll be happy to get some feedback on it.

http://momtchil.momtchev.com/layer7switch-0.1.2-httpparser.tar.gz

-- 
Momtchil MOMTCHEV
Ingénieur Expert / Cellule de TM2L (http://www.08000linux.com/)
LINAGORA SA - http://www.linagora.com
Tél.: +33 (0)1 58 18 68 28 - Fax : +33 (0)1 58 18 68 29
Portable : +33 (0)6 11 64 09 37

[Linux-l7sw-devel] [ANNOUNCE] L7SW 0.1.2

[Alexandre C. <ac...@fr...>]

Hello,

Just to announce a new release available on www.linux-l7sw.org

ChangeLog looks like :

2007-01-15 Alexandre Cassen <acassen [at] linux-l7sw.org>
        * layer7switch-0.1.2 released.
        * tcp_splice: fixed a locking issue while setting socket
          to ECONNRESET.
        * tcp_splice: Willy Tarreau <w <at> 1wt [dot] eu> backported
          tcp_splice kernel module to kernel v2.4.
        * tcp_splice: Willy Tarreau <w <at> 1wt [dot] eu> backported
          tcp_splice kernel module to lower v2.6 release than 2.6.19.
          Created defines to use proper checksum constant while
          forwarding a skb.
        * libtcpsplice: exported tcp_splice_init() and
          modprobe_tcp_splice().
        * haproxy: Willy Tarreau <w <at> 1wt [dot] eu> extended HAProxy
          code to support kernel tcp splicing. This daemon is already
          supporting smat content switching engine, futur work of L7SW
          will use haproxy as userspace daemon. The current switchd
          still in the tarball but will no longer be extended, just
          a simple working proof of concept daemon.
        * switchd: Quickly fixed an issue while handling asynchronous
          wait for completion (wait for socket to be reset).

best regs,
Alex

Re: [Linux-l7sw-devel] [ANNOUNCE] Haproxy 1.3.5 + tcp splicing

[Willy T. <w...@1w...>]

Hi Alex,

On Mon, Jan 08, 2007 at 09:15:17AM +0100, Alexandre Cassen wrote:
> > I migrated the tests on my
> > notebook because the kernel repeatedly freezes on my main PC after
> > a random number of sessions (a few thousands), with 2.6.19.1,
> > 2.6.16.37 and 2.4.33+patches. Since it's a dual athlon, I suspect
> > an SMP race condition somewhere. Everytime, all processes stopped
> > scheduling (vmstat stopped, the keyboard and mouse did not respond),
> > but Alt-SysRQ could still dump crap on the frame buffer, sync,
> > unmount and reboot. It has never happened even once on the notebook.
> 
> outch... a lock issue IMHO, have you got calltrace, I will try on a SMP
> box during the week.

No, no trace because I was under X the first time, and in the frame
buffer after that, but in all cases, the trace became unreadable as
soon as the screen scrolled. I'll have to boot with the console on
the serial port to give you more info. Also, my notebook's NIC is a
tg3 while my desktop's is an e1000. I don't think it matters, but
it might be possible that their buffering methods trigger different
access patterns too.

> > I have also noticed that when sessions are terminated, they seem to
> > be reused without passing through the process if a SYN comes in
> > during the TIME_WAIT state. I could even maintain a constant load
> > of about 45 sessions/s each transfering 1 MB while the proxy had
> > been stopped for one minute.
> 
> hmm... a mistake in the FSM, will investigate.

I'm used to check this on firewalls where it represents a security
breach :-)

You have to consider that for an TCP session, if you see a SYN with
a lower SEQ than the start of your window, it is a retransmit, and
you just send an ACK. But if you see a SYN with a higher SEQ than
the end of the window, either your session is still alive, and you
send an RST, or your session is terminated (CLOSE or TIME_WAIT),
then you consider it as a new one and everything must be reinitialized.

In our case, it might be slightly more difficult because we also rely
on local socket delivery. To be honnest, I don't know what this will
imply.

> > I have written a small introduction doc explaining how to build
> > L7SW and haproxy and how to get them to work. There is also a
> > sample configuration. That way, you will not have to waste your
> > time maintaining your switchd (except for fun and experimentation).
> > BTW, I don't know why, but it often crashes with the following
> > message :
> > 
> >   switchd: scheduler.c:572: thread_fetch: Assertion `0' failed.
> >   Aborted
> 
> ? outch... anyway switchd is just my debugging testbed, your haproxy is
> BEST !

It's not best, it's just that it has been working for years before being
adapted to L7SW, and that was already a hard job. Starting from scratch
with L7SW in mind is an optimistic but necessary experience at the
beginning :-)

(...)
> Nice, I will merge all your work during this week.

OK fine !

Cheers,
Willy

Re: [Linux-l7sw-devel] [ANNOUNCE] Haproxy 1.3.5 + tcp splicing

[Alexandre C. <ac...@fr...>]

Hi Willy,

> as promised, I've adapted Haproxy to support your TCP splicing code.
> It was really easy, I'm relying on the libtcpsplice. I think we will
> have to slightly change the initialization method, because I would
> like to be able to init the socket without doing the modprobe, so
> I would need to access tcp_splice_init(), which currently is static.
> As you see, that's no big deal.

agreed will extend lib.

> I've noticed that the process must run as root all the time because
> a check on CAP_NET_ADMIN is performed at the top of the function
> do_tcp_splice_set_ctl(). I think that we should relax the test so
> that once the process has grabbed its IP_RAW socket and done the
> initialization, it should be able to splice sockets without being
> root anymore.

hmm... agreed too.

> The performance gain is very interesting. I have installed haproxy
> on my notebook, tux on my PC and my injector on the PC too. So the
> HTTP requests travel from PC to NB to PC. Without TCP splicing, I
> reach about 370 Mbps in both directions with 65% CPU used on the
> notebook. With TCP splicing, the bit rate is exactly the same, but
> the load goes down by a half. Since in both cases, the poor dirty
> NIC is saturated first, it is normal that I get same data rates.
> But seeing the CPU usage being cut in half is impressive. And the
> CPU usage for haproxy goes down from 65% to about 5% in such a case.

:) excellent :)

> Everything's not pink or blue though.

yes there still some pending points...

> I migrated the tests on my
> notebook because the kernel repeatedly freezes on my main PC after
> a random number of sessions (a few thousands), with 2.6.19.1,
> 2.6.16.37 and 2.4.33+patches. Since it's a dual athlon, I suspect
> an SMP race condition somewhere. Everytime, all processes stopped
> scheduling (vmstat stopped, the keyboard and mouse did not respond),
> but Alt-SysRQ could still dump crap on the frame buffer, sync,
> unmount and reboot. It has never happened even once on the notebook.

outch... a lock issue IMHO, have you got calltrace, I will try on a SMP
box during the week.

> I have also noticed that when sessions are terminated, they seem to
> be reused without passing through the process if a SYN comes in
> during the TIME_WAIT state. I could even maintain a constant load
> of about 45 sessions/s each transfering 1 MB while the proxy had
> been stopped for one minute.

hmm... a mistake in the FSM, will investigate.

> If I transfer small objects (say, empty or 1 kB pages), the ports
> quickly wrap around (of course) and it seems like sessions cannot
> establish anymore. I'll have to check this further, because it
> looked exactly like a firewall I have benchmarked, and which
> randomized SEQ numbers. I had to turn timestamps on to achieve
> "decent" speeds of 50000 hits/s. Right now they are disabled, so
> it might be a valid reason, but it's too late to retry right now.

IMHO, the same issue as previous point.

> I have written a small introduction doc explaining how to build
> L7SW and haproxy and how to get them to work. There is also a
> sample configuration. That way, you will not have to waste your
> time maintaining your switchd (except for fun and experimentation).
> BTW, I don't know why, but it often crashes with the following
> message :
> 
>   switchd: scheduler.c:572: thread_fetch: Assertion `0' failed.
>   Aborted

? outch... anyway switchd is just my debugging testbed, your haproxy is
BEST !

> Oh, last but not least, before I forget. Running tcpdump on eth0
> with a client connecting to the proxy from 127.0.0.1 made me see
> the packets from/to 127.0.0.1 on eth0. I found it a bit strange,
> but I think it's just the interface pointer associated to the
> skb which causes this, and no packet really goes out of the NIC,
> but it might be worth finding why it does this.

will investigate.

> Useful resources :
> 
>    Haproxy 1.3.5 :
>    http://haproxy.1wt.eu/download/1.3/bin/
>    http://haproxy.1wt.eu/download/1.3/src/
>    http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.5.tar.gz
> 
>    Install doc :
>    http://haproxy.1wt.eu/download/1.3/doc/
>    http://haproxy.1wt.eu/download/1.3/doc/tcp-splicing.txt
> 
>    Sample configuration :
>    http://haproxy.1wt.eu/download/1.3/test/
>    http://haproxy.1wt.eu/download/1.3/test/tcp-splicing-sample.cfg
> 
> I will send an announce on freshmeat to try to get more testers.

Nice, I will merge all your work during this week.

Thanks lot for your feedback !


best regs,
Alex

[Linux-l7sw-devel] [ANNOUNCE] Haproxy 1.3.5 + tcp splicing

[Willy T. <w...@1w...>]

Hi Alex,

as promised, I've adapted Haproxy to support your TCP splicing code.
It was really easy, I'm relying on the libtcpsplice. I think we will
have to slightly change the initialization method, because I would
like to be able to init the socket without doing the modprobe, so
I would need to access tcp_splice_init(), which currently is static.
As you see, that's no big deal.

I've noticed that the process must run as root all the time because
a check on CAP_NET_ADMIN is performed at the top of the function
do_tcp_splice_set_ctl(). I think that we should relax the test so
that once the process has grabbed its IP_RAW socket and done the
initialization, it should be able to splice sockets without being
root anymore.

The performance gain is very interesting. I have installed haproxy
on my notebook, tux on my PC and my injector on the PC too. So the
HTTP requests travel from PC to NB to PC. Without TCP splicing, I
reach about 370 Mbps in both directions with 65% CPU used on the
notebook. With TCP splicing, the bit rate is exactly the same, but
the load goes down by a half. Since in both cases, the poor dirty
NIC is saturated first, it is normal that I get same data rates.
But seeing the CPU usage being cut in half is impressive. And the
CPU usage for haproxy goes down from 65% to about 5% in such a case.

Everything's not pink or blue though. I migrated the tests on my
notebook because the kernel repeatedly freezes on my main PC after
a random number of sessions (a few thousands), with 2.6.19.1,
2.6.16.37 and 2.4.33+patches. Since it's a dual athlon, I suspect
an SMP race condition somewhere. Everytime, all processes stopped
scheduling (vmstat stopped, the keyboard and mouse did not respond),
but Alt-SysRQ could still dump crap on the frame buffer, sync,
unmount and reboot. It has never happened even once on the notebook.

I have also noticed that when sessions are terminated, they seem to
be reused without passing through the process if a SYN comes in
during the TIME_WAIT state. I could even maintain a constant load
of about 45 sessions/s each transfering 1 MB while the proxy had
been stopped for one minute.

If I transfer small objects (say, empty or 1 kB pages), the ports
quickly wrap around (of course) and it seems like sessions cannot
establish anymore. I'll have to check this further, because it
looked exactly like a firewall I have benchmarked, and which
randomized SEQ numbers. I had to turn timestamps on to achieve
"decent" speeds of 50000 hits/s. Right now they are disabled, so
it might be a valid reason, but it's too late to retry right now.

I have written a small introduction doc explaining how to build
L7SW and haproxy and how to get them to work. There is also a
sample configuration. That way, you will not have to waste your
time maintaining your switchd (except for fun and experimentation).
BTW, I don't know why, but it often crashes with the following
message :

  switchd: scheduler.c:572: thread_fetch: Assertion `0' failed.
  Aborted

Oh, last but not least, before I forget. Running tcpdump on eth0
with a client connecting to the proxy from 127.0.0.1 made me see
the packets from/to 127.0.0.1 on eth0. I found it a bit strange,
but I think it's just the interface pointer associated to the
skb which causes this, and no packet really goes out of the NIC,
but it might be worth finding why it does this.


Useful resources :

   Haproxy 1.3.5 :
   http://haproxy.1wt.eu/download/1.3/bin/
   http://haproxy.1wt.eu/download/1.3/src/
   http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.5.tar.gz

   Install doc :
   http://haproxy.1wt.eu/download/1.3/doc/
   http://haproxy.1wt.eu/download/1.3/doc/tcp-splicing.txt

   Sample configuration :
   http://haproxy.1wt.eu/download/1.3/test/
   http://haproxy.1wt.eu/download/1.3/test/tcp-splicing-sample.cfg

I will send an announce on freshmeat to try to get more testers.

Have fun
Willy

Re: [Linux-l7sw-devel] [ANNOUNCE] L7SW backport to 2.4.33 and 2.6.16

[Alexandre C. <ac...@fr...>]

Hi Willy,
> Hi Alex !
>
> I finally found the time to backport tcp_splice-0.1.1 to 2.6.16 for
> people who want continuous kernel support, and to 2.4.33 for other
> who need to run 2.4.
>
> As you told me, the backport was really easy. I tried to touch as
> few code as possible. It would make sense to merge the 2.6.16 patch
> in mainstream as it's straightforward and still compatible with 2.6.19.
> I would then rediff the 2.4.33 patch against this one.
>
> I post them to the list so that interested people can find them in the
> archive. BTW, I've also dumped them here :
>
>    http://haproxy.1wt.eu/download/patches/
>   

Woww excellent, will merge your work during the week-end.

best regs,
Alex

[Linux-l7sw-devel] [ANNOUNCE] L7SW backport to 2.4.33 and 2.6.16

[Willy T. <w...@1w...>]

Hi Alex !

I finally found the time to backport tcp_splice-0.1.1 to 2.6.16 for
people who want continuous kernel support, and to 2.4.33 for other
who need to run 2.4.

As you told me, the backport was really easy. I tried to touch as
few code as possible. It would make sense to merge the 2.6.16 patch
in mainstream as it's straightforward and still compatible with 2.6.19.
I would then rediff the 2.4.33 patch against this one.

I post them to the list so that interested people can find them in the
archive. BTW, I've also dumped them here :

   http://haproxy.1wt.eu/download/patches/

Best regards,
Willy

[Linux-l7sw-devel] [ANNOUNCE] L7SW release 0.1.1

[Alexandre C. <ac...@fr...>]

Hello,

Have just release the new code that fixe some issues and extend code for
better testing. Need to take time to create a TODO list to define kind
of roadmap for project. Currently we need to extend kernel splicing
engine to support wscale and make more work on MSS to deal with
fragmentation for huge POST, current MSS code assumption is: most of
large data stream is coming from server to client. if peer are
supporting PMTU then can use PMTU to tweak socket mss, but we need to
preserve code as possible against fragmentation handling since this will
introduce latency.... There is lot of work in userspace switchd daemon
to become a real content aware guy.

ChangeLog for current release is :

2006-12-28 Alexandre Cassen <ac...@li...>
        * layer7switch-0.1.1 released.
        * tcp_splice: test if sock is already released while resetting
          it. This can happen if userspace daemon is playing with
          fully asynchronous and non blocking stream handling.
        * switchd: extended TCP listener to support fully asynchronous
          and non blocking stream handling. Accept and connect
          operations are asynchronous.
        * switchd: rewrote splice wait_for_completion to be 
          asynchronous.
        * switchd: extend configuration parser and daemon engine to
          support multiple virtual_server definitions. Each
          virtual_server can refer to a list of real_server.
        * switchd: added support to Round-Robin and Least-Connection
          loadbalancing schedulers.
        * extended configuration file and readme accordingly.


cya,
Alexandre

Re: [Linux-l7sw-devel] [ANNOUNCE] New Layer7 switching project

[home_king <hom...@16...>]

Hi Alexandre,
 > You need to bridge the traffic in order to perform accurate layer7
 > switching decision especially for HTTP/1.1 where multiple requests are
 > sent into the same persitent connection. So L7SW must be a proxy but a
 > high speed proxy. I mean, with the current design, socket API is used in
 > order to have a compatibility layer with current proxy software and
 > proxy design, but as soon as the socket are spliced the socket context
 > for each connection is released. tcp_splice module create a layer4
 > forwarding rules, so packets are switched from on side to the other just
 > with the same performance as layer4 switching system (IPVS). We just
 > have context switching overhead induced by client GET request relaying
 > from client to server. Parallel balancing can be handled by an upper
 > layer4 balancing system (IPVS) to a pool of L7SW.

What about the TCP sequence masquerade for incoming & outcoming packet of
the fake pipe after scheduled (Given at this time, the real connections
to bridge the client and real server has been released), on which I think
the TCP splicing will base. The masquerade may consume a bit CPU time.

I think one NIC with TCP/IP checksum capability may be a good auxiliary
thing to implement splicing-style L7 switch, right? :-)

On the other hand, to do Parallel balancing, the sole toplevel ipvs L4
director that you said, brings in another new performance bottleneck,
I think.

Cheers,

Jinhua

Re: [Linux-l7sw-devel] New Layer7 switching project

[Willy T. <w...@1w...>]

On Thu, Dec 14, 2006 at 10:32:32PM +0100, Alexandre Cassen wrote:
> Hi Willy,
> 
> >Alex, if I may suggest, you should not lose too much of your time on exotic
> >features for a proof of concept. Your time seems more valuable to me on 
> >hard
> >to solve problems such as MSS and window scaling where fewer people can 
> >work
> >than regex. Better add a wishlist to the project with difficulty levels ;-)
> 
> yes you're right :) just want to extend userland soon since I spent
> already most of time in kernelspace.

I fairly can understand, sometimes I need to leave kernel for haproxy and
vice-versa :-)

> I already 'solved' the MSS
> problem with the current code setting server user_mss to client
> mss_clamp negociated => this is partial since in the other direction
> fragmentation can appear, need to deal with this, but most of
> datastream is from server to client so this assumption 'work' in most
> of HTTP workflow but to be rigurous must add fragmentation support :)

Hmmm be very careful with those assumptions. On a large scale infrastructure,
we encountered two problems with MSS :

  - ADSL users with broken PPPoE who cannot send large packets between their
    PC/router and their modem, and whose PC/router does not fragment nor
    clamp the MSS. They can do everything _except_ a few requests involving
    large POST requests. We had to perform this on netfilter just as a
    workaround for them since they were really many.

  - users on Token ring or FDDI announcing MSS values of 3960 and 4460, making
    it impossible to use Solaris 10's workaround consisting in announcing the
    same MSS as the client - 8 for the problem above.

  - fragments not being correctly supported by several network equipments such
    as a well known hardware-based load balancers and a well known firewall
    editor.

This leaves you with very few reliable options :

  - announcing 536 bytes to both client and server. This is dirty, but RFC791
    requires that any IP stack supports at least an MTU of 576 bytes, which
    means 536 bytes MSS in the default case (20 IP + 20 TCP), which is also
    the default MSS value. Reserving some space under this value for IP options
    (at most 60 bytes) and TCP options, it _must_ work everywhere.

  - slice and concatenate segments. This is a difficult work, which either
    requires to memorize skbs (client MSS < server MSS) or to create more
    skbs (client MSS > server MSS), but would be optimally efficient.


> I need now to extend to support wscale and sack. Will focus on those
> points during next week end.

SACK will not be easy at all I guess... An easy workaround still consists in
de-activating it. :-/

Cheers,
Willy

Re: [Linux-l7sw-devel] New Layer7 switching project

[Alexandre C. <ac...@gm...>]

Hi Willy,

> Alex, if I may suggest, you should not lose too much of your time on exotic
> features for a proof of concept. Your time seems more valuable to me on hard
> to solve problems such as MSS and window scaling where fewer people can work
> than regex. Better add a wishlist to the project with difficulty levels ;-)

yes you're right :) just want to extend userland soon since I spent
already most of time in kernelspace. I already 'solved' the MSS
problem with the current code setting server user_mss to client
mss_clamp negociated => this is partial since in the other direction
fragmentation can appear, need to deal with this, but most of
datastream is from server to client so this assumption 'work' in most
of HTTP workflow but to be rigurous must add fragmentation support :)

I need now to extend to support wscale and sack. Will focus on those
points during next week end.

cya,
Alexandre

Re: [Linux-l7sw-devel] [ANNOUNCE] New Layer7 switching project

[Alexandre C. <Ale...@fr...>]

Hi Jinhua,
> How do you think about the performance of the new L7 switching system?
> Traditionally, the load balancer establishs a "pipe" between the client
> and the selected real server. In other word, the load balancer acts as
> a proxy. In this way, one TCP connection corresponds to two TCP 
> connections.
> Why not try other new manners to ensure the low cost of L7 switch?
>
> Besides, how do you think about the parallel workmode among the multiple
> load balancers? That is, active & ative. I think it's meaningful to 
> improve
> the Scalability of L7 switch.
You need to bridge the traffic in order to perform accurate layer7 
switching decision especially for HTTP/1.1 where multiple requests are 
sent into the same persitent connection. So L7SW must be a proxy but a 
high speed proxy. I mean, with the current design, socket API is used in 
order to have a compatibility layer with current proxy software and 
proxy design, but as soon as the socket are spliced the socket context 
for each connection is released. tcp_splice module create a layer4 
forwarding rules, so packets are switched from on side to the other just 
with the same performance as layer4 switching system (IPVS). We just 
have context switching overhead induced by client GET request relaying 
from client to server. Parallel balancing can be handled by an upper 
layer4 balancing system (IPVS) to a pool of L7SW.

cya,
Alexandre

Re: [Linux-l7sw-devel] New Layer7 switching project

[Willy T. <w...@1w...>]

Hi Alex, Hi Jacob,

On Thu, Dec 14, 2006 at 01:03:30PM +0100, Alexandre Cassen wrote:
> Hi Jacob :)
> 
> > What about the haproxy-project as found on http://haproxy.1wt.eu/
> > That's a great level-7 switching project. If you are concerned about
> > memory usage or context switching, try it first. It runs in a single
> > threads and uses 'select' as its sole dispatcher.
> 
> sure I am aware of Willy work ;) and we off-line spoke about this. The
> framework I will try to propose with L7SW project is for high perf relay
> using kernel machinery and haproxy can be extended to use this
> machinery.

I can quite confirm this. Right now I'm very busy working on implementing
content-switching (URL switching, virtual hosts support, etc...), and of
course with my job, but I firmly intend to try to port haproxy to support
Alex's work. It will also be a very interesting solution to measure the
gains it provides (and I expect a lot on large files).

> > But you might have good reasons not to rely purely on level-7 proxies. 
> > However, since existing cookies can be used to remember to which real
> > server a request should be forwarded, session-data can be stored
> > locally. This is a great advantage over level-4 loadbalancing.
> 
> exactly my focus :) during next week-end I will try to find time to work
> on loadbalancing support and regexp support (will dig in pcre engine for
> highspeed parsing).

Alex, if I may suggest, you should not lose too much of your time on exotic
features for a proof of concept. Your time seems more valuable to me on hard
to solve problems such as MSS and window scaling where fewer people can work
than regex. Better add a wishlist to the project with difficulty levels ;-)

Cheers,
Willy

Re: [Linux-l7sw-devel] New Layer7 switching project

[Alexandre C. <ac...@fr...>]

Hi Jacob :)

> What about the haproxy-project as found on http://haproxy.1wt.eu/
> That's a great level-7 switching project. If you are concerned about
> memory usage or context switching, try it first. It runs in a single
> threads and uses 'select' as its sole dispatcher.

sure I am aware of Willy work ;) and we off-line spoke about this. The
framework I will try to propose with L7SW project is for high perf relay
using kernel machinery and haproxy can be extended to use this
machinery.

> But you might have good reasons not to rely purely on level-7 proxies. 
> However, since existing cookies can be used to remember to which real
> server a request should be forwarded, session-data can be stored
> locally. This is a great advantage over level-4 loadbalancing.

exactly my focus :) during next week-end I will try to find time to work
on loadbalancing support and regexp support (will dig in pcre engine for
highspeed parsing).

cya,
Alexandre