RE: Re:

[CHOO C. W. [MY] <che...@vo...>]

Hi Dave,

I've actually tune it accordingly from this website http://curl-loader.sourceforge.net/doc/faq.html#big-load

ulimit -n 100000
sysctl -w tcp_tw_recycle=1
sysctl -w net.ipv4.tcp_tw_recycle=1
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w fs.file-max=100000
sysctl -w net.ipv4.conf.all.arp_filter=0
sysctl -w net.ipv4.conf.all.rp_filter=0
sysctl -w net.ipv4.conf.eth2.rp_filter=0
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sysctl -w net.ipv4.tcp_wmem="4096 87380 16777216"
sysctl -w net.ipv4.tcp_mem="16777216 16777216 16777216"
echo 1024000 > /proc/sys/net/ipv4/neigh/default/gc_thresh1
echo 1024000 > /proc/sys/net/ipv4/neigh/default/gc_thresh2
echo 1024000 > /proc/sys/net/ipv4/neigh/default/gc_thresh3

I manage to get the link usage to around 800mbps but after 5 minutes, the link usage drops to around 200mbps. Any clues to where should I look into?
Also whenever I tried to use the -t switch, it gives out segmentation error. I tried it with -t 4 and get the same segmentation error.

main - RUNNING LOAD, STARTING THREADS

create_thr_subbatches - error: subbatches should deal either with a single shared IP or with at least a single shared IP per subbatch
main - note: Thread 0, started normally
main - note: Thread 1, started normally
Segmentation fault

Regards,

Chee Wei


 Please adopt the “eco-attitude”.

  Print this mail only if really necessary.


-----Original Message-----
From: Dave Seddon [mailto:da...@se...] 
Sent: Monday, March 21, 2011 12:30 PM
To: curl-loader-devel
Subject: Re:

Greetings,

The thought put will obviously depend on many things, some examples (not
exhaustive) include:
- are you requesting the same object form the web server each time (so
it will be cached in RAM by the web server, otherwise you'll be IO bound
on the disk)
- how big is the file(s)?  if you have lots of short lived TCP/HTTP
sessions, then you won't wined out the TCP window, so the throughput is
likely to be low.  If you have several large and long lived
transactions, the throughput is likely to be much more.  e.g.  Several
clients downloading a 100MB file are likely to get more throughput than
lots of clients with 1k file.
- how is the tcp stack tuned? -> tune it :)

Consider using nginx web server.

Kind regards,
Dave Seddon
da...@se...

On Mon, 2011-03-21 at 04:59 +0100, CHOO Chee Wei [MY] wrote:
> Hi,
> 
>  
> 
> I had a 1gb link between curl-loader and a standard apache webserver
> with no tweaks.
> 
> I am just wondering what is the relationship between CLIENTS_NUM_MAX
> and bandwidth?
> 
> I cant seem to generate enough bandwidth when my CLIENTS_NUM_MAX is
> huge, say like 10,000.
> 
> I had 200 ip address which is more than enough to support the 10k
> users, since each ip address is only limited by the number of ports
> available which is around 64k per ip.
> 
>  
> 
> Curl loader conf:
> 
> BATCH_NAME= bulk-load
> 
> CLIENTS_NUM_MAX=10000
> 
> CLIENTS_NUM_START=100
> 
> CLIENTS_RAMPUP_INC=50
> 
> INTERFACE   =eth2
> 
> NETMASK= 24
> 
> IP_ADDR_MIN= 10.0.149.1
> 
> IP_ADDR_MAX= 10.0.149.201
> 
> IP_SHARED_NUM=200
> 
> CYCLES_NUM= -1
> 
> RUN_TIME=3600
> 
> REQ_RATE=500
> 
> URLS_NUM= 30
> 
> ############ URL SECTION ####################################
> 
> URL=http://10.0.97.31/load/load.html
> 
> URL_SHORT_NAME="index-get"
> 
> REQUEST_TYPE=GET
> 
> TIMER_URL_COMPLETION = 0     # In msec. When positive, Now it is
> enforced by cancelling url fetch on timeout
> 
> TIMER_AFTER_URL_SLEEP = 0
> 
> #TRANSFER_LIMIT_RATE=56000
> 
>  
> 
> URL=http://10.0.97.31/load2/load.html
> 
> URL_SHORT_NAME="index"
> 
> REQUEST_TYPE=GET
> 
> TIMER_URL_COMPLETION = 0
> 
> TIMER_AFTER_URL_SLEEP =0
> 
> #TRANSFER_LIMIT_RATE=56000
> 
>  
> 
> URL=http://10.0.97.31/load3/load.html
> 
> URL_SHORT_NAME="index"
> 
> REQUEST_TYPE=GET
> 
> TIMER_URL_COMPLETION = 0
> 
> TIMER_AFTER_URL_SLEEP =0
> 
> #TRANSFER_LIMIT_RATE=56000
> 
>  
> 
> Goes on until…….
> 
> URL=http://10.0.97.31/load30/load.html
> 
> URL_SHORT_NAME="index"
> 
> REQUEST_TYPE=GET
> 
> TIMER_URL_COMPLETION = 0
> 
> TIMER_AFTER_URL_SLEEP =0
> 
> #TRANSFER_LIMIT_RATE=56000
> 
>  
> 
>  
> 
>  
> 
> Curl-loader client info:
> 
> [root@EAF loadtest]# cat /proc/cpuinfo
> 
> processor       : 0
> 
> vendor_id       : GenuineIntel
> 
> cpu family      : 6
> 
> model           : 15
> 
> model name      : Intel(R) Xeon(R) CPU            3060  @ 2.40GHz
> 
> stepping        : 6
> 
> cpu MHz         : 2400.127
> 
> cache size      : 4096 KB
> 
> physical id     : 0
> 
> siblings        : 2
> 
> core id         : 0
> 
> cpu cores       : 2
> 
> apicid          : 0
> 
> fdiv_bug        : no
> 
> hlt_bug         : no
> 
> f00f_bug        : no
> 
> coma_bug        : no
> 
> fpu             : yes
> 
> fpu_exception   : yes
> 
> cpuid level     : 10
> 
> wp              : yes
> 
> flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm
> constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
> 
> bogomips        : 4800.25
> 
>  
> 
> processor       : 1
> 
> vendor_id       : GenuineIntel
> 
> cpu family      : 6
> 
> model           : 15
> 
> model name      : Intel(R) Xeon(R) CPU            3060  @ 2.40GHz
> 
> stepping        : 6
> 
> cpu MHz         : 2400.127
> 
> cache size      : 4096 KB
> 
> physical id     : 0
> 
> siblings        : 2
> 
> core id         : 1
> 
> cpu cores       : 2
> 
> apicid          : 1
> 
> fdiv_bug        : no
> 
> hlt_bug         : no
> 
> f00f_bug        : no
> 
> coma_bug        : no
> 
> fpu             : yes
> 
> fpu_exception   : yes
> 
> cpuid level     : 10
> 
> wp              : yes
> 
> flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca
> cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm
> constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr lahf_lm
> 
> bogomips        : 4800.18
> 
>  
> 
> [root@EAF loadtest]# free -m
> 
>              total       used       free     shared    buffers
> cached
> 
> Mem:          4052        725       3326          0        238
> 411
> 
> -/+ buffers/cache:         76       3975
> 
> Swap:         1983          0       1983
> 
>  
> 
> With the above specification, can you recommend a sample conf that
> will generate 10k-40k users with traffic that will suffocate the 1gb
> link on my network?
> 
>  
> 
> Regards,
> 
>  
> 
> Chee Wei
> 
>  
> 
> cid:image002.jpg@01C93B49.E347BA70
> 
> 
>  Please adopt the “eco-attitude”.
> 
> 
>   Print this mail only if really
> necessary.
> 
> 
> 
>  
> 
>  
> 
> 
> ------------------------------------------------------------------------------
> Colocation vs. Managed Hosting
> A question and answer guide to determining the best fit
> for your organization - today and in the future.
> http://p.sf.net/sfu/internap-sfd2d
> _______________________________________________ curl-loader-devel mailing list cur...@li... https://lists.sourceforge.net/lists/listinfo/curl-loader-devel



------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
curl-loader-devel mailing list
cur...@li...
https://lists.sourceforge.net/lists/listinfo/curl-loader-devel