Re: slow transfer speed and ftp issues

[Robert I. <cor...@gm...>]

Hi Eric,

On Tue, Jul 20, 2010 at 6:27 AM, ERIC FORMO <eri...@ho...> wrote:

>  The PROBLEM REPORTING FORM
>
>
> CONFIGURATION-FILE (The most common source of problems):
> Place the file inline here:
> ########### GENERAL SECTION ################################
> BATCH_NAME= ftp
> CLIENTS_NUM_MAX=100
> INTERFACE   = eth1
> NETMASK=255.255.0.0
> #IP_SHARED_NUM = 1
> IP_ADDR_MIN= 10.0.0.1
> IP_ADDR_MAX= 10.0.100.253  #Actually - this is for self-control
> CYCLES_NUM= -1
> URLS_NUM = 1
>
> ########### URL SECTION ####################################
>
> #URL=ftp://tester:tester@10.0.10.52/100k-random.html
> URL=ftp://tester:tester@10.0.0.101/100k-random.html
> #URL=ftp://tester:tester@10.0.10.52/curl-loader-0.52.tar.gz
> FRESH_CONNECT=1                                  # At least my proftpd has
> problems with connection re-use
> TIMER_URL_COMPLETION = 0 # In msec. When positive, Now it is enforced by
> cancelling url fetch on timeout
> TIMER_AFTER_URL_SLEEP =3000
> FTP_ACTIVE=1
>
> DESCRIPTION:
> 1. I can do regular ftp and wget using that ftp command, it just won't work
> using this config file.  This config file was working when i used a 24 bit
> netmask and 10.0.10.52 as the server.  Now I want to test more hosts and
> would like to use 16 bit netmask.  My ifconfig on client and apache is the
> following:
> Client - eth1      Link encap:Ethernet  HWaddr xxx
>           inet addr:10.10.10.75  Bcast:10.10.255.255  Mask:255.255.0.0
> Server - eth1      Link encap:Ethernet  HWaddr xxx
>           inet addr:10.10.10.52  Bcast:10.10.255.255  Mask:255.255.0.0
>

Thank you for the PRF!


Could you explain a bit more, what happens, what types of errors you are
getting
in log file, when starting with -v option?
Are you traffic routes normally to server?

Anyhow, when using more clients, please, consider using the tags below:

CLIENTS_NUM_START - number of clients to start the load with. If the tag is
not
present or zero CLIENTS_RAMPUP_INC value will be taken instead. If neither
CLIENTS_NUM_START nor CLIENTS_RAMPUP_INC are present or have any positive
value,
the loader will take CLIENTS_NUM_MAX as the number of clients to start the
load.

CLIENTS_RAMPUP_INC - number of clients to be added to the load in the auto
mode
every second till CLIENTS_NUM_MAX number will be reached. For machines with
a
single CPU we would recommend not to keep the number above 50-100, whereas
for
dual-CPU machines you can keep it as large as 200-300.


>
> 2. When i use the linux ftp client to transfer the 100k file once i get
> 484.8 Mib/sec.  When i use curl-loader to ftp the same file with 1 FTP
> client i get 0.02971 Mib/sec.  What is the cause?  Does curl-loader not
> calculate the Ti the same way the native ftp client does?
> The slow down is also seen in getting html files from my apache server. I
> should be able to get close to line rate grabbing one 100K file with 10
> Concurrent clients, but curl-loader can only get up to 276.8 Mib/sec.
>


curl-loader is using libcurl for HTTP and FTP protocol stacks.

I've seen there a bug fixed, when using FTP, which may be your case.

Namely:

 Sat, Jul 17, 2010 at 1:55 AM, Jan Van Boghout <li...@ma...> wrote:

> Hello List,
>
> When I upgraded to libcurl 7.21.0, I started getting strange timeout errors
> from FTP commands (immediately after sending them). After some research,
> I've concluded that the FTP implementation is missing at least one timestamp
> reset point, which the following patch adds:
>
> http://macrabbit.com/misc/libssh2/ftp.c.patch
>
> If someone who knows the FTP subsystem could review and include, that would
> be most appreciated. There's also a documentation/comment error in
> pingpong.h: response_time is described as a number of seconds when it's
> actually milliseconds. Relevant line:
>
> long response_time; /* When no timeout is given, this is the amount of
> seconds we await for a server response. */
>
> Cheers,
> Jan
>



>
>
> QUESTION/ SUGGESTION/ PATCH:
> Can you please enlighten me as to why my ftp config file is not working?
> Why can i not get close to line rate with either ftp or http?  I tried with
> -r and -t with no luck.
>
>

You can either:

The best - take the most recent curl sanpshot and place to curl-loader and
try to build with it.
http://curl.haxx.se/snapshots/

Or
to incorporate the patch, that  Jan Van Boghout  provided.

Ask me if you need some more detailed guiding in curl-loader making system.



-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................