Thread: Query random paths?
Status: Alpha
Brought to you by:
coroberti
Menu
▾
▴
curl-loader-devel
|
From: Jan W. <J.W...@cs...> - 2009-07-24 15:08:50
|
Hi, Just trying my first steps into curl-loader :-) I looked through the FAQ and config files, but I couldn't find an answer on how to query a *lot* of random paths using a pool of -lets say- 20 clients. I see we have FORM_RECORDS_RANDOM, which seems to do the trick for form-data. I am trying to debug server crashes, where I know it is related to concurrency, but otherwise I have little clue on whether or not it is related to the actual query paths. Just to bash the server, I collected a list of 6,000 paths that have been queried on the server recently and then I use the config below, but curl-loader only seems to use the first 20 paths of the file :-( Did I overlook something trivial? Cheers --- Jan ================================================================ BATCH_NAME= bulk CLIENTS_NUM_MAX=20 CLIENTS_NUM_START=20 CLIENTS_RAMPUP_INC=0 INTERFACE=eth0 NETMASK=16 IP_ADDR_MIN= 192.168.1.1 IP_ADDR_MAX= 192.168.1.200 IP_SHARED_NUM=200 CYCLES_NUM= -1 URLS_NUM= 1 URL_TEMPLATE=http://localhost:3040%s URL_TOKEN_FILE=paths URL_SHORT_NAME="local-server" REQUEST_TYPE=GET TIMER_URL_COMPLETION = 5000 TIMER_AFTER_URL_SLEEP = 0 ================================================================ |
|
From: Val S. <Val...@no...> - 2009-07-24 18:25:46
|
I'm not an expert on curl-loader and I have not used the url templates yet. The way I do it is literally generate all the urls in the urls section and set FETCH_PROBABILITY on each url to a low value, e.g 2., so it looks like this: ########### GENERAL SECTION ################################ BATCH_NAME = random-picked-urls CLIENTS_NUM_MAX = 35 INTERFACE = eth1 NETMASK = 255.255.0.0 IP_ADDR_MIN = 10.2.1.2 IP_ADDR_MAX = 10.2.1.201 CYCLES_NUM = -1 URLS_NUM = 5000 ########### URL SECTION #################################### URL = http://www.XYZ.NET/ URL_SHORT_NAME = "badurl1" REQUEST_TYPE = GET TIMER_URL_COMPLETION = 0 TIMER_AFTER_URL_SLEEP = 0 FETCH_PROBABILITY = 2 FETCH_PROBABILITY_ONCE = 0 URL = http://www.XYZ1.NET/ URL_SHORT_NAME = "badurl2" REQUEST_TYPE = GET TIMER_URL_COMPLETION = 0 TIMER_AFTER_URL_SLEEP = 0 FETCH_PROBABILITY = 2 FETCH_PROBABILITY_ONCE = 0 .... >-----Original Message----- >From: Jan Wielemaker [mailto:J.W...@cs...] >Sent: Friday, July 24, 2009 6:25 AM >To: cur...@li... >Subject: Query random paths? > >Hi, > >Just trying my first steps into curl-loader :-) I looked through the FAQ >and config files, but I couldn't find an answer on how to query a *lot* of >random paths using a pool of -lets say- 20 clients. > >I see we have FORM_RECORDS_RANDOM, which seems to do the trick for >form-data. I am trying to debug server crashes, where I know it is >related to concurrency, but otherwise I have little clue on whether or >not it is related to the actual query paths. > >Just to bash the server, I collected a list of 6,000 paths that have >been queried on the server recently and then I use the config below, but >curl-loader only seems to use the first 20 paths of the file :-( > >Did I overlook something trivial? > > Cheers --- Jan > >================================================================ >BATCH_NAME= bulk >CLIENTS_NUM_MAX=20 >CLIENTS_NUM_START=20 >CLIENTS_RAMPUP_INC=0 >INTERFACE=eth0 >NETMASK=16 >IP_ADDR_MIN= 192.168.1.1 >IP_ADDR_MAX= 192.168.1.200 >IP_SHARED_NUM=200 >CYCLES_NUM= -1 >URLS_NUM= 1 > >URL_TEMPLATE=http://localhost:3040%s >URL_TOKEN_FILE=paths >URL_SHORT_NAME="local-server" >REQUEST_TYPE=GET >TIMER_URL_COMPLETION = 5000 >TIMER_AFTER_URL_SLEEP = 0 >================================================================ > > > > >--------------------------------------------------------------------------- >--- >_______________________________________________ >curl-loader-devel mailing list >cur...@li... >https://lists.sourceforge.net/lists/listinfo/curl-loader-devel |
|
From: Jan W. <J.W...@cs...> - 2009-07-27 11:20:52
|
Dear Val, Thanks for the trick. Maybe not very elegant, but workable :-) --- Jan On Friday 24 July 2009 08:25:28 pm Val Shkolnikov wrote: > I'm not an expert on curl-loader and I have not used the url templates yet. > The way I do it is literally generate all the urls in the urls section and > set FETCH_PROBABILITY on each url to a low value, e.g 2., so it looks like > this: |
|
From: Robert I. <cor...@gm...> - 2009-07-27 05:38:22
|
Hi Jan, On Fri, Jul 24, 2009 at 4:25 PM, Jan Wielemaker <J.W...@cs...>wrote: > Hi, > > Just trying my first steps into curl-loader :-) I looked through the FAQ > and config files, but I couldn't find an answer on how to query a *lot* of > random paths using a pool of -lets say- 20 clients. > > I see we have FORM_RECORDS_RANDOM, which seems to do the trick for > form-data. I am trying to debug server crashes, where I know it is > related to concurrency, but otherwise I have little clue on whether or > not it is related to the actual query paths. > > Just to bash the server, I collected a list of 6,000 paths that have > been queried on the server recently and then I use the config below, but > curl-loader only seems to use the first 20 paths of the file :-( > > Did I overlook something trivial? > > Cheers --- Jan > > ================================================================ > BATCH_NAME= bulk > CLIENTS_NUM_MAX=20 > CLIENTS_NUM_START=20 > CLIENTS_RAMPUP_INC=0 > INTERFACE=eth0 > NETMASK=16 > IP_ADDR_MIN= 192.168.1.1 > IP_ADDR_MAX= 192.168.1.200 > IP_SHARED_NUM=200 > CYCLES_NUM= -1 > URLS_NUM= 1 > > URL_TEMPLATE=http://localhost:3040%s > URL_TOKEN_FILE=paths > URL_SHORT_NAME="local-server" > REQUEST_TYPE=GET > TIMER_URL_COMPLETION = 5000 > TIMER_AFTER_URL_SLEEP = 0 > ================================================================ > It is a rather recent feature developed by Gary and it will be great if Gary will comment. Still, you are keeping 20 clients, which are testing 20 paths. Could it be that 6000 clients will do the job? Give it a try. Just keep in mind, that by default you need for 35K of memory per each client and ramp-up about 100 not to start all them at start-up. -- Truly, Robert Iakobashvili, Ph.D. ...................................................................... www.ghotit.com Assistive technology that understands you ...................................................................... |
|
From: Jan W. <J.W...@cs...> - 2009-07-27 11:13:19
|
On Monday 27 July 2009 07:38:12 am Robert Iakobashvili wrote: > Hi Jan, > > On Fri, Jul 24, 2009 at 4:25 PM, Jan Wielemaker <J.W...@cs...>wrote: > > Hi, > > > > Just trying my first steps into curl-loader :-) I looked through the FAQ > > and config files, but I couldn't find an answer on how to query a *lot* > > of random paths using a pool of -lets say- 20 clients. > > > > I see we have FORM_RECORDS_RANDOM, which seems to do the trick for > > form-data. I am trying to debug server crashes, where I know it is > > related to concurrency, but otherwise I have little clue on whether or > > not it is related to the actual query paths. > > > > Just to bash the server, I collected a list of 6,000 paths that have > > been queried on the server recently and then I use the config below, but > > curl-loader only seems to use the first 20 paths of the file :-( > > > > Did I overlook something trivial? > > > > Cheers --- Jan > > > > ================================================================ > > BATCH_NAME= bulk > > CLIENTS_NUM_MAX=20 > > CLIENTS_NUM_START=20 > > CLIENTS_RAMPUP_INC=0 > > INTERFACE=eth0 > > NETMASK=16 > > IP_ADDR_MIN= 192.168.1.1 > > IP_ADDR_MAX= 192.168.1.200 > > IP_SHARED_NUM=200 > > CYCLES_NUM= -1 > > URLS_NUM= 1 > > > > URL_TEMPLATE=http://localhost:3040%s > > URL_TOKEN_FILE=paths > > URL_SHORT_NAME="local-server" > > REQUEST_TYPE=GET > > TIMER_URL_COMPLETION = 5000 > > TIMER_AFTER_URL_SLEEP = 0 > > ================================================================ > > It is a rather recent feature developed by Gary and it will be great if > Gary will comment. I've seen that the config allows for getting random records from form-data files. Would be nice to have the same for token-files. > Still, you are keeping 20 clients, which are testing 20 paths. > Could it be that 6000 clients will do the job? Give it a try. Probably that will work, but I don't want a lot of concurrent clients. > Just keep in mind, that by default you need for 35K of memory per each > client > and ramp-up about 100 not to start all them at start-up. I must admit that I still don't really get the meaning of the CIENTS_* settings. Is the number of clients expressed by this formula (t is time in seconds). max(CLIENTS_NUM_MAX, CLIENTS_NUM_START+t*CLIENTS_RAMPUP_INC) --- Jan |
|
From: Robert I. <cor...@gm...> - 2009-07-27 15:17:54
|
On Mon, Jul 27, 2009 at 2:13 PM, Jan Wielemaker <J.W...@cs...>wrote: > On Monday 27 July 2009 07:38:12 am Robert Iakobashvili wrote: > > Hi Jan, > > > > On Fri, Jul 24, 2009 at 4:25 PM, Jan Wielemaker > <J.W...@cs...>wrote: > > > Hi, > > > > > > Just trying my first steps into curl-loader :-) I looked through the > FAQ > > > and config files, but I couldn't find an answer on how to query a *lot* > > > of random paths using a pool of -lets say- 20 clients. > > > > > > I see we have FORM_RECORDS_RANDOM, which seems to do the trick for > > > form-data. I am trying to debug server crashes, where I know it is > > > related to concurrency, but otherwise I have little clue on whether or > > > not it is related to the actual query paths. > > > > > > Just to bash the server, I collected a list of 6,000 paths that have > > > been queried on the server recently and then I use the config below, > but > > > curl-loader only seems to use the first 20 paths of the file :-( > > > > > > Did I overlook something trivial? > > > > > > Cheers --- Jan > > > > > > ================================================================ > > > BATCH_NAME= bulk > > > CLIENTS_NUM_MAX=20 > > > CLIENTS_NUM_START=20 > > > CLIENTS_RAMPUP_INC=0 > > > INTERFACE=eth0 > > > NETMASK=16 > > > IP_ADDR_MIN= 192.168.1.1 > > > IP_ADDR_MAX= 192.168.1.200 > > > IP_SHARED_NUM=200 > > > CYCLES_NUM= -1 > > > URLS_NUM= 1 > > > > > > URL_TEMPLATE=http://localhost:3040%s > > > URL_TOKEN_FILE=paths > > > URL_SHORT_NAME="local-server" > > > REQUEST_TYPE=GET > > > TIMER_URL_COMPLETION = 5000 > > > TIMER_AFTER_URL_SLEEP = 0 > > > ================================================================ > > > > It is a rather recent feature developed by Gary and it will be great if > > Gary will comment. > > > I must admit that I still don't really get the meaning of the CIENTS_* > settings. Is the number of clients expressed by this formula (t is > time in seconds). > > max(CLIENTS_NUM_MAX, CLIENTS_NUM_START+t*CLIENTS_RAMPUP_INC) > current clients num = CLIENTS_NUM_START+t*CLIENTS_RAMPUP_INC till it comes the the CLIENTS_NUM_MAX -- Truly, Robert Iakobashvili, Ph.D. ...................................................................... www.ghotit.com Assistive technology that understands you ...................................................................... |
|
From: Gary F. <ga...@iF...> - 2009-07-27 17:36:26
|
I've been away from this for a long time now, and it will be a week or
two before I can take a look, but I'll see what I can do.
Gary
On Jul 26, 2009, at 10:38 PM, Robert Iakobashvili wrote:
Hi Jan,
On Fri, Jul 24, 2009 at 4:25 PM, Jan Wielemaker
<J.W...@cs...> wrote:
Hi,
Just trying my first steps into curl-loader :-) I looked through the
FAQ
and config files, but I couldn't find an answer on how to query a
*lot* of
random paths using a pool of -lets say- 20 clients.
I see we have FORM_RECORDS_RANDOM, which seems to do the trick for
form-data. I am trying to debug server crashes, where I know it is
related to concurrency, but otherwise I have little clue on whether or
not it is related to the actual query paths.
Just to bash the server, I collected a list of 6,000 paths that have
been queried on the server recently and then I use the config below, but
curl-loader only seems to use the first 20 paths of the file :-(
Did I overlook something trivial?
Cheers --- Jan
================================================================
BATCH_NAME= bulk
CLIENTS_NUM_MAX=20
CLIENTS_NUM_START=20
CLIENTS_RAMPUP_INC=0
INTERFACE=eth0
NETMASK=16
IP_ADDR_MIN= 192.168.1.1
IP_ADDR_MAX= 192.168.1.200
IP_SHARED_NUM=200
CYCLES_NUM= -1
URLS_NUM= 1
URL_TEMPLATE=http://localhost:3040%s
URL_TOKEN_FILE=paths
URL_SHORT_NAME="local-server"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 5000
TIMER_AFTER_URL_SLEEP = 0
================================================================
It is a rather recent feature developed by Gary and it will be great
if Gary will comment.
Still, you are keeping 20 clients, which are testing 20 paths.
Could it be that 6000 clients will do the job? Give it a try.
Just keep in mind, that by default you need for 35K of memory per each
client
and ramp-up about 100 not to start all them at start-up.
--
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................
------------------------------------------------------------------------------
_______________________________________________
curl-loader-devel mailing list
cur...@li...
https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
|
|
From: Robert I. <cor...@gm...> - 2009-07-27 18:04:06
|
On Mon, Jul 27, 2009 at 7:35 PM, Gary Fitts <ga...@if...> wrote: > I've been away from this for a long time now, and it will be a week or two > before I can take a look, but I'll see what I can do. > > Gary > > Nice hearing from you, Gary. Take your time! -- Truly, Robert Iakobashvili, Ph.D. ...................................................................... www.ghotit.com Assistive technology that understands you ...................................................................... |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X