Re: FRESH_CONNECT=1 and TIMER_AFTER_URL_SLEEP =5000

[BuraphaLinux S. <bur...@gm...>]

Hello,

    I will get the wireshark trace and post a URL for you to download
it.  I can do it 3 times, once with 5000, once with 10000, and once
with 15000.  I will change TIMER_TCP_CONN_SETUP to 10.   It will take
a while to get the data; I am starting now.  I will save all log files
from the run on the client and server so you can look at them too.

JGH


On 8/5/07, Robert Iakobashvili <cor...@gm...> wrote:
> On 8/5/07, BuraphaLinux Server <bur...@gm...> wrote:
> >
> > Hello,
> >
> >     In testing I did not see any difference.  In the logs I get blocks
> > like this when curl-loader is cycling too fast (sorry gmail wraps
> > badly):
>
>
> The difference is when FRESH_CONNECT=1 and sleep time positive
> curl-loader is supposed to close its connection to server, go to sleep
> for the sleep timeout, and re-establish the connection after sleeping.
>
> If you will increase the sleeping time, you can hopefully observe the
> behavior in sniffer.
>
>
> : eff-url:
> > ftp://anonymous:joe%040@10.16.68.213/apache/httpd/httpd-2.0.59.tar.bz2,
> > url:
> > 121 20 (10.16.68.197)  <= Recv header: eff-url:
> > ftp://anonymous:joe%040@10.16.68.213/apache/httpd/httpd-2.0.59.tar.bz2,
> > url:
> > 121 20 (10.16.68.197) :<= WARNING: parsing error: wrong response code
> > (FTP?) 0 .
> > 121 20 (10.16.68.197)  !! ERROR: This doesn't seem like a nice
> > ftp-server response
> > : eff-url:
> > ftp://anonymous:joe%040@10.16.68.213/apache/httpd/httpd-2.0.59.tar.bz2,
> > url:
> > 121 20 (10.16.68.197)  :== Info: Expire cleared
> > : eff-url:
> > ftp://anonymous:joe%040@10.16.68.213/apache/httpd/httpd-2.0.59.tar.bz2,
> > url:
> > 121 20 (10.16.68.197)  :== Info: Closing connection #117
> >
> > I worry that 'parsing error' means it doesn't do the delay, but I
> > don't really know.
>
>
> it's not the issue.
>
> The server side of this will say this in the vsftpd log file:
> >
> > Sun Aug  5 18:17:58 2007 [pid 10839] CONNECT: Client "10.16.68.197",
> > "Connection refused: too many sessions."
> >
> > Using ss -tan (or netstat) I can verify there are too many connections
> > on the server.
> > The vsftpd server has been set for 100 clients with these entries in
> > vsftpd.conf:
> >
> > #
> > # Maximum number of simultaneous clients
> > #
> > max_clients=100
> > #
> > # Maximum number of connections from 1 ip
> > #
> > max_per_ip=100
> >
> > (and I painfully tested by hand that this limit works as expected)
> >
> > My configuration is ftp.small for curl-loader and is this:
> >
> > ########### GENERAL SECTION ################################
> >
> > BATCH_NAME= ftpsmall
> > CLIENTS_NUM_MAX=50 # Same as CLIENTS_NUM
> > CLIENTS_NUM_START=5
> > CLIENTS_RAMPUP_INC=5
> > INTERFACE=eth0
> > NETMASK=32
> > IP_ADDR_MIN=10.16.68.197
> > IP_ADDR_MAX=10.16.68.197
> > CYCLES_NUM=-1
> > URLS_NUM=1
> >
> > ########### URL SECTION ####################################
> >
> > URL=ftp://anonymous:joe%040@10.16.68.213/apache/httpd/httpd-2.0.59.tar.bz2
> > FRESH_CONNECT=1
> > URL_SHORT_NAME="small"
> > TIMER_URL_COMPLETION = 0 # In msec. When positive, Now it is enforced
> > by cancelling url fetch on timeout
> > TIMER_AFTER_URL_SLEEP =5000
> > TIMER_TCP_CONN_SETUP=50
>
>
> Looks good, but I would  make TIMER_TCP_CONN_SETUP not larger than 10
> seconds.
>
> and was invoked with this script (as root user):
> >
> > #! /sbin/bash
> > rm -fr ftpsmall.*
> > ulimit -d unlimited
> > ulimit -f unlimited
> > ulimit -m unlimited
> > ulimit -n 19999
> > ulimit -t unlimited
> > ulimit -u unlimited
> > ulimit -v unlimited
> > ulimit -x unlimited
> > echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
> > echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
> > echo 1 > /proc/sys/net/ipv4/tcp_ecn
> > echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
> > curl-loader -f ftp.small -v -u -w
> >
> > With the limit set to 50 on the client and the delays,
>
>
>
> OK.
>
> Do you see connections closed by client (curl-loader)
> in  network  (sniffer wireshark/ethereal capture with a single curl-loader
> client could be of
> assistance) and re-established after 5 seconds,
> or the connections are remaining stalled?
>
> We are calling libcurl function, which is supposed to release the
> connection,
> but if this is buggy, thus, we can dig into the issue.
>
> Another option is that the release by FTP could take some time.
> Could you try a larger timeout, like 10000, 15 000?
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> A web testing and traffic generation tool.
>