Re: [Linux-l7sw-devel] [ANNOUNCE] Haproxy 1.3.5 + tcp splicing

[Willy T. <w...@1w...>]

Hi Alex,

On Mon, Jan 08, 2007 at 09:15:17AM +0100, Alexandre Cassen wrote:
> > I migrated the tests on my
> > notebook because the kernel repeatedly freezes on my main PC after
> > a random number of sessions (a few thousands), with 2.6.19.1,
> > 2.6.16.37 and 2.4.33+patches. Since it's a dual athlon, I suspect
> > an SMP race condition somewhere. Everytime, all processes stopped
> > scheduling (vmstat stopped, the keyboard and mouse did not respond),
> > but Alt-SysRQ could still dump crap on the frame buffer, sync,
> > unmount and reboot. It has never happened even once on the notebook.
> 
> outch... a lock issue IMHO, have you got calltrace, I will try on a SMP
> box during the week.

No, no trace because I was under X the first time, and in the frame
buffer after that, but in all cases, the trace became unreadable as
soon as the screen scrolled. I'll have to boot with the console on
the serial port to give you more info. Also, my notebook's NIC is a
tg3 while my desktop's is an e1000. I don't think it matters, but
it might be possible that their buffering methods trigger different
access patterns too.

> > I have also noticed that when sessions are terminated, they seem to
> > be reused without passing through the process if a SYN comes in
> > during the TIME_WAIT state. I could even maintain a constant load
> > of about 45 sessions/s each transfering 1 MB while the proxy had
> > been stopped for one minute.
> 
> hmm... a mistake in the FSM, will investigate.

I'm used to check this on firewalls where it represents a security
breach :-)

You have to consider that for an TCP session, if you see a SYN with
a lower SEQ than the start of your window, it is a retransmit, and
you just send an ACK. But if you see a SYN with a higher SEQ than
the end of the window, either your session is still alive, and you
send an RST, or your session is terminated (CLOSE or TIME_WAIT),
then you consider it as a new one and everything must be reinitialized.

In our case, it might be slightly more difficult because we also rely
on local socket delivery. To be honnest, I don't know what this will
imply.

> > I have written a small introduction doc explaining how to build
> > L7SW and haproxy and how to get them to work. There is also a
> > sample configuration. That way, you will not have to waste your
> > time maintaining your switchd (except for fun and experimentation).
> > BTW, I don't know why, but it often crashes with the following
> > message :
> > 
> >   switchd: scheduler.c:572: thread_fetch: Assertion `0' failed.
> >   Aborted
> 
> ? outch... anyway switchd is just my debugging testbed, your haproxy is
> BEST !

It's not best, it's just that it has been working for years before being
adapted to L7SW, and that was already a hard job. Starting from scratch
with L7SW in mind is an optimistic but necessary experience at the
beginning :-)

(...)
> Nice, I will merge all your work during this week.

OK fine !

Cheers,
Willy