Re: How do I set send and recv buffers using curl-loader

[Paddy G. <pad...@gm...>]

First off, thank you for the prompt response and supporting this wonderful
tool.

On Wed, Aug 26, 2009 at 11:05 PM, Robert Iakobashvili
<cor...@gm...>wrote:

> I have no idea about httpperf, but may be other users have some experience.


http://httperf.comlore.com/Home gives you some idea of recent advances to
the original paper (
http://www.hpl.hp.com/research/linux/httperf/wisp98/httperf.pdf)

>
>
> Neither we nor libcurl (the HTTP stack) are playing with socket default
> buffers at linux.
> There is no point in playing with them for a web-client.


It is critical for WAN latencies. If the buffers are too small, like they
are when default values are used, the TCP congestion window will never fully
open up (particularly on long fat networks)
http://www.cisco.com/web/about/ac123/ac147/ac174/ac196/about_cisco_ipj_archive_article09186a00800c8417.htmlhas
more information (TCP Volume Transfer section near Figure 7))



>
> You can change the client window size by playing with this parameter in
> Makefile,
> whereas I have doubts, whether it has a serious performance impact, but may
> be with
> some rare web-servers.


I did a small experiment in my lab wherein I get a 100K file with 16K buffer
as well as 64K buffer over a 330ms network.
With 16K the response time was 17.161 s whereas just tweaking the buffer
size gives you 7.589 s

Thank you again.

-Paddy

ps: The lab output for my 100K file so you can simulate the same

16K Buffer
-------------------

httperf --timeout=5 --client=0/1 --server=web.lab --port=80 --uri=/100K
--rate=60 --send-buffer=4096 --recv-buffer=16384 --num-conns=1
--num-calls=10
Maximum connect burst length: 0

Total: connections 1 requests 10 replies 10 test-duration 17.161 s

Connection rate: 0.1 conn/s (17160.7 ms/conn, <=1 concurrent connections)
Connection length [replies/conn]: 10.000

num avg stddev min 10% 25% 50% 75% 90% 95% 99% max
Connection life [ms] 1 17160.7 0.0 17160.7 17160 17160 17160 17160 17160
17160 17160 17160.7
Handshake time [ms] 1 330.1 0.0 330.1 330 330 330 330 330 330 330 330.1
Response time [ms] 10 329.9 0.3 329.7 330 330 330 330 330 330 330 330.8
Transfer time [ms] 10 1353.1 104.4 1320.1 1320 1320 1320 1320 1650 1650 1650
1650.3

Request rate: 0.6 req/s (1716.1 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 0.0 avg 0.5 max 0.6 stddev 0.1 (3 samples)
Reply size [B]: header 234.0 content 106404.0 footer 0.0 (total 106638.0)
Reply status: 1xx=0 2xx=10 3xx=0 4xx=0 5xx=0

CPU time [s]: user 4.88 system 12.27 (user 28.5% system 71.5% total 100.0%)
Net I/O: 60.7 KB/s (0.5*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0


64K Buffer
-------------
 httperf --timeout=5 --client=0/1 --server=web.lab --port=80 --uri=/100K
--rate=60 --send-buffer=4096 --recv-buffer=65535 --num-conns=1
--num-calls=10
httperf --timeout=5 --client=0/1 --server=web.lab --port=80 --uri=/100K
--rate=60 --send-buffer=4096 --recv-buffer=65535 --num-conns=1
--num-calls=10
Maximum connect burst length: 0

Total: connections 1 requests 10 replies 10 test-duration 7.589 s

Connection rate: 0.1 conn/s (7589.1 ms/conn, <=1 concurrent connections)
Connection length [replies/conn]: 10.000

num avg stddev min 10% 25% 50% 75% 90% 95% 99% max
Connection life [ms] 1 7589.1 0.0 7589.1 7590 7590 7590 7590 7590 7590 7590
7589.1
Handshake time [ms] 1 329.2 0.0 329.2 330 330 330 330 330 330 330 329.2
Response time [ms] 10 329.9 0.1 329.6 330 330 330 330 330 330 330 330.0
Transfer time [ms] 10 396.1 208.8 330.0 330 330 330 330 990 990 990 990.4

Request rate: 1.3 req/s (758.9 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 0.0 avg 1.2 max 1.2 stddev 0.0 (1 samples)
Reply size [B]: header 234.0 content 106404.0 footer 0.0 (total 106638.0)
Reply status: 1xx=0 2xx=10 3xx=0 4xx=0 5xx=0

CPU time [s]: user 1.35 system 6.24 (user 17.7% system 82.2% total 100.0%)
Net I/O: 137.3 KB/s (1.1*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

>
>
> Take care!
>
>
> --
> Truly,
> Robert Iakobashvili, Ph.D.
> ......................................................................
> www.ghotit.com
> Assistive technology that understands you
> ......................................................................
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>