Thread: Simultaenous URL request
Status: Alpha
Brought to you by:
coroberti
Menu
▾
▴
curl-loader-devel
|
From: Ciaran M. <cmc...@gm...> - 2012-06-27 17:24:32
|
Hi, Curlloader is a great piece of software - keep up the good work! I was wondering if it's possible to fetch a list of URLS simultaneously? I have a script that automates the generation of large amounts of URLs. I'm looking to transfer a known amount of unique data and a known throughput via a proxy device. At the moment, Curlloader processes each URL sequentially for each Client. I'd like it to download each URL for each specified Client, once. E.g. ########## GENERAL SECTION ########## BATCH_NAME=sim_get CLIENTS_NUM_MAX=1000 CLIENTS_NUM_START=1000 CLIENTS_RAMPUP_INC=1 INTERFACE=eth0 NETMASK=255.255.224.0 IP_ADDR_MIN=172.16.47.100 IP_ADDR_MAX=172.16.47.100 CYCLES_NUM=1 URLS_NUM=1 ########## URLs SECTION ############# URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1 URL_SHORT_NAME="1" REQUEST_TYPE=GET TRANSFER_LIMIT_RATE=1000 URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=2 URL_SHORT_NAME="2" REQUEST_TYPE=GET TRANSFER_LIMIT_RATE=1000 URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=3 URL_SHORT_NAME="3" REQUEST_TYPE=GET TRANSFER_LIMIT_RATE=1000 .. .. .. URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1000 URL_SHORT_NAME="1000" REQUEST_TYPE=GET TRANSFER_LIMIT_RATE=1000 Thanks and regards! |
|
From: Robert I. <cor...@gm...> - 2012-06-28 11:25:11
|
Dear Ciaran, Look at URL-probabality feature in README. Regards, Robert On Wed, Jun 27, 2012 at 8:24 PM, Ciaran McCrisken <cmc...@gm...> wrote: > Hi, > > > > Curlloader is a great piece of software – keep up the good work! > > > > I was wondering if it’s possible to fetch a list of URLS simultaneously? I > have a script that automates the generation of large amounts of URLs. I’m > looking to transfer a known amount of unique data and a known throughput via > a proxy device. > > > > At the moment, Curlloader processes each URL sequentially for each Client. > I’d like it to download each URL for each specified Client, once. > > > > E.g. > > > > ########## GENERAL SECTION ########## > > BATCH_NAME=sim_get > > CLIENTS_NUM_MAX=1000 > > CLIENTS_NUM_START=1000 > > CLIENTS_RAMPUP_INC=1 > > INTERFACE=eth0 > > NETMASK=255.255.224.0 > > IP_ADDR_MIN=172.16.47.100 > > IP_ADDR_MAX=172.16.47.100 > > CYCLES_NUM=1 > > URLS_NUM=1 > > > > ########## URLs SECTION ############# > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1 > > URL_SHORT_NAME="1" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=2 > > URL_SHORT_NAME="2" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=3 > > URL_SHORT_NAME="3" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > .. > > .. > > .. > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1000 > > URL_SHORT_NAME="1000" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > Thanks and regards! > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > curl-loader-devel mailing list > cur...@li... > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel > -- Regards, Robert Iakobashvili, Ph.D. Home: http://www.ghotit.com Blog: http://www.ghotit.com/category/ghotit-blogs/ Twitter: http://twitter.com/ghotit Facebook: http://facebook.com/ghotit ...................................................................... Ghotit Dyslexia Assistive technology that understands you ...................................................................... |
|
From: Ciaran M. <cmc...@gm...> - 2012-07-23 15:53:48
|
Hi Robert, Sorry for the late response. I initially looked at this - I think I've perhaps misunderstood it's functionality, but it looks as though I could use this - I'll take another look at it. In the end, I was able to achieve what I wanted using the URL_RANDOM_RANGE functionality which complements the server-side random byte generator nicely. Thanks again! Ciaran. -----Original Message----- From: Robert Iakobashvili [mailto:cor...@gm...] Sent: 28 June 2012 12:25 To: curl-loader-devel Subject: Re: Simultaenous URL request Dear Ciaran, Look at URL-probabality feature in README. Regards, Robert On Wed, Jun 27, 2012 at 8:24 PM, Ciaran McCrisken <cmc...@gm...> wrote: > Hi, > > > > Curlloader is a great piece of software keep up the good work! > > > > I was wondering if its possible to fetch a list of URLS simultaneously? I > have a script that automates the generation of large amounts of URLs. Im > looking to transfer a known amount of unique data and a known throughput via > a proxy device. > > > > At the moment, Curlloader processes each URL sequentially for each Client. > Id like it to download each URL for each specified Client, once. > > > > E.g. > > > > ########## GENERAL SECTION ########## > > BATCH_NAME=sim_get > > CLIENTS_NUM_MAX=1000 > > CLIENTS_NUM_START=1000 > > CLIENTS_RAMPUP_INC=1 > > INTERFACE=eth0 > > NETMASK=255.255.224.0 > > IP_ADDR_MIN=172.16.47.100 > > IP_ADDR_MAX=172.16.47.100 > > CYCLES_NUM=1 > > URLS_NUM=1 > > > > ########## URLs SECTION ############# > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1 > > URL_SHORT_NAME="1" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=2 > > URL_SHORT_NAME="2" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=3 > > URL_SHORT_NAME="3" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > .. > > .. > > .. > > > > URL=http://172.16.62.110/cgi-bin/bytegenerator?size=100&seed=1000 > > URL_SHORT_NAME="1000" > > REQUEST_TYPE=GET > > TRANSFER_LIMIT_RATE=1000 > > > > Thanks and regards! > > > ---------------------------------------------------------------------------- -- > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > curl-loader-devel mailing list > cur...@li... > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel > -- Regards, Robert Iakobashvili, Ph.D. Home: http://www.ghotit.com Blog: http://www.ghotit.com/category/ghotit-blogs/ Twitter: http://twitter.com/ghotit Facebook: http://facebook.com/ghotit ...................................................................... Ghotit Dyslexia Assistive technology that understands you ...................................................................... ---------------------------------------------------------------------------- -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ curl-loader-devel mailing list cur...@li... https://lists.sourceforge.net/lists/listinfo/curl-loader-devel |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X