curl-loader-devel — curl-loader usage and development issues

how does curl-loader use RTNETLINK

[Javier M. <jam...@in...>]

Hi, first thanks for your excellent program, very useful.
I have some questions about the use of sockets.
When I load a client with a virtual IP I understand curl-loader uses
RTNETLINK for this, and when I run it again with the same config file
(same client ip) appears the following:

add_secondary_ip_addrs - setting secondary IP 10.111.111.150
rtnl_talk(): RTNETLINK answers: File exists
add_secondary_ip_addrs - note: probably, the IP-address
"10.111.111.150/24" already exists.

this is not really a problem when using the loader, but when I use the
same ip used by another machine (e.g. a virtual machine), the virtual
client replaces it (supplanting) and the host became unreachable (even
when curl-loader is down).
Do you know how to delete the file which is referenced by rtnl_talk()
?, or if there is another way to solve this problem.

thanks very much for your help.

greetings from chile!!

-- 
Javier Miranda
jam...@in...
+569 8-552-6707

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Prahav,

On Feb 19, 2008 11:46 PM, Pranav Desai <pra...@gm...> wrote:

>
>
> Well, the log I sent you was for only 5 users, so I dont see how it
> could be a resource issue. In any case I will dig around a bit and see
> if I have messed up something, especially with heavier load testing.


The recommendations  are related to the "heavy" load.
As you see in the FAQs, your memory should be about 35K per client and
there are other issues in the FAQs.

Between, to set a proxy you can simply set the env. variable
> (http_proxy=http://proxy_addr:prxoy_port) and libcurl will do it ...
> pretty easy :-)


So, we are supporting HTTP/FTP proxy right now. :)
I will add this tip to our FAQs.

Thanks for all your help.
>
> -- Pranav


Thank  you.

-- 
Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: TIMER_AFTER_URL_SLEEP question

[Pranav D. <pra...@gm...>]

On Feb 19, 2008 2:27 AM, Robert Iakobashvili <cor...@gm...> wrote:
> Hi Prahav,
>
> On Feb 19, 2008 12:24 PM, Robert Iakobashvili <cor...@gm...> wrote:
>
> Try also to exclude various network bottlenecks,
>  like instead of 100 Mbps your get 10 Mbps network due
> to some weird issues, your router/switch settings, etc.
>

Well, the log I sent you was for only 5 users, so I dont see how it
could be a resource issue. In any case I will dig around a bit and see
if I have messed up something, especially with heavier load testing.

Between, to set a proxy you can simply set the env. variable
(http_proxy=http://proxy_addr:prxoy_port) and libcurl will do it ...
pretty easy :-)

Thanks for all your help.

-- Pranav
>
>
> Sincerely,
> Robert Iakobashvili
> "Light will come from Jerusalem"
> ...........................................................
>  http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>



-- 

------------------------------
http://pd.dnsalias.org

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Hi Prahav,

On Feb 19, 2008 12:24 PM, Robert Iakobashvili <cor...@gm...> wrote:

Try also to exclude various network bottlenecks,
like instead of 100 Mbps your get 10 Mbps network due
to some weird issues, your router/switch settings, etc.

Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Hi Prahav,

On Feb 19, 2008 9:19 AM, Pranav Desai <pra...@gm...> wrote:

> On Feb 18, 2008 10:03 PM, Robert Iakobashvili <cor...@gm...> wrote:
>
> I will look into those issues. One more thing I noticed was that for
> different bind addresses its  using the same connection. Is this
> expected?. Or should different addresses use a new connection
> irrespective of the -r options.
>

Connections re-use is OK. Don't use -r.
I would more suspect shortage of memory and/or lack of
enough open descriptors.

Please, follow the big-load procedure step by step and understand
the memory concerns.

Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: TIMER_AFTER_URL_SLEEP question

[Pranav D. <pra...@gm...>]

On Feb 18, 2008 10:03 PM, Robert Iakobashvili <cor...@gm...> wrote:
> Pranav,
>
>
> On Feb 19, 2008 5:12 AM, Pranav Desai <pra...@gm...> wrote:
> >
> >
> > Will definitely look into that ...
> >
> > Regarding the original questions for TIMER_AFTER_URL_SLEEP, I am
> > trying to increase the number of users/conections (5000+) but keep the
> > BW low since my webserver can only support 100Mbps, and I thought
> > increasing the above timer would help (2 sec), but it seems like the
> > num. of users seems to be very low compared to what i have set it to.
> > Some users just dont seem to send requests for a long time (a lot more
> > than 2 sec), looking at tcpdump.
> >
> > What do you think could be problems. Can you suggest any other way to
> > achieve the above test setup. I dont want to use the
> > TRANSFER_RATE_LIMIT since I want the clients to fetch it at full BW.
> >
>
> Talking in general may be 3 sources of problem:
> a) client load side;
> b) network bottleneck, including NATs, Firewall limits, etc;
> c) web-server side.
>
> If you suspect, that the problem is on the side of curl-loader, please,
>  go through the procedure described in FAQs  here:
>
> http://curl-loader.sourceforge.net/doc/faq.html#big-load
>
> It may be e.g. a limit of open descriptors at your client computer or at
> server, etc problems.
>

Hello Robert,

I will look into those issues. One more thing I noticed was that for
different bind addresses its  using the same connection. Is this
expected?. Or should different addresses use a new connection
irrespective of the -r options.

Here is a log:

0 1 (12.0.0.1)  :== Info: About to connect() to 172.16.55.200 port 80 (#0)
: eff-url: , url:
0 1 (12.0.0.1)  :== Info:   Trying 172.16.55.200... : eff-url: , url:
0 1 (12.0.0.1)  :== Info: Bind local address to 12.0.0.1
: eff-url: , url:
0 1 (12.0.0.1)  :== Info: Local port: 38494
: eff-url: , url:
0 1 (12.0.0.1)  :== Info: Connected to 172.16.55.200 (172.16.55.200)
port 80 (#0)
: eff-url: , url:
0 1 (12.0.0.1)  => Send header: eff-url: , url:
0 1 (12.0.0.1)  <= Recv header: eff-url: , url:
0 1 (12.0.0.1) :!! 200 OK: eff-url: , url:
0 1 (12.0.0.1)  <= Recv header: eff-url: , url:
...
0 1 (12.0.0.1)  <= Recv header: eff-url: , url:
0 1 (12.0.0.1)  <= Recv data: eff-url: , url:
...
0 1 (12.0.0.1)  <= Recv data: eff-url: , url:
0 1 (12.0.0.1)  :== Info: Connection #0 to host 172.16.55.200 left intact
: eff-url: , url:
0 2 (12.0.0.2)  :== Info: Re-using existing connection! (#0) with host
172.16.55.200
: eff-url: , url:
0 2 (12.0.0.2)  :== Info: Connected to 172.16.55.200 (172.16.55.200)
port 80 (#0)
: eff-url: , url:
0 1 (12.0.0.1)  :== Info: About to connect() to 172.16.55.200 port 80 (#1)

Here 12.0.0.2 seems to be using the same connection as for 12.0.0.1

Thanks for your help.

-- Pranav

> If you have a multi-core or multi-cpu at the curl-loader linux computer, you
> can effectively
> increase your loading power, please use -t 2 or -t 4 options according to
> the number of cores/processors.
>
> --
>
>
> Sincerely,
>  Robert Iakobashvili
> "Light will come from Jerusalem"
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>



-- 

------------------------------
http://pd.dnsalias.org

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Pranav,

On Feb 19, 2008 5:12 AM, Pranav Desai <pra...@gm...> wrote:

>
> Will definitely look into that ...
>
> Regarding the original questions for TIMER_AFTER_URL_SLEEP, I am
> trying to increase the number of users/conections (5000+) but keep the
> BW low since my webserver can only support 100Mbps, and I thought
> increasing the above timer would help (2 sec), but it seems like the
> num. of users seems to be very low compared to what i have set it to.
> Some users just dont seem to send requests for a long time (a lot more
> than 2 sec), looking at tcpdump.
>
> What do you think could be problems. Can you suggest any other way to
> achieve the above test setup. I dont want to use the
> TRANSFER_RATE_LIMIT since I want the clients to fetch it at full BW.
>

Talking in general may be 3 sources of problem:
a) client load side;
b) network bottleneck, including NATs, Firewall limits, etc;
c) web-server side.

If you suspect, that the problem is on the side of curl-loader, please,
go through the procedure described in FAQs  here:

http://curl-loader.sourceforge.net/doc/faq.html#big-load

It may be e.g. a limit of open descriptors at your client computer or at
server, etc problems.

If you have a multi-core or multi-cpu at the curl-loader linux computer, you
can effectively
increase your loading power, please use -t 2 or -t 4 options according to
the number of cores/processors.

-- 
Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: TIMER_AFTER_URL_SLEEP question

[Pranav D. <pra...@gm...>]

On Feb 16, 2008 8:38 PM, Robert Iakobashvili <cor...@gm...> wrote:
> Pranav,
>
> On Feb 16, 2008 10:18 PM, Pranav Desai <pra...@gm...> wrote:
> > Great !
> > Thanks for the help.
> >
> > Do you guys have any plans for adding proxy support ... ?
>
> It is rather easy. Since libcurl has such support, what you need to do
> is to add for the new configuration file tags treatment in parser, and
> to set such options to the libcurl handle, CURL* object.
>

Will definitely look into that ...

Regarding the original questions for TIMER_AFTER_URL_SLEEP, I am
trying to increase the number of users/conections (5000+) but keep the
BW low since my webserver can only support 100Mbps, and I thought
increasing the above timer would help (2 sec), but it seems like the
num. of users seems to be very low compared to what i have set it to.
Some users just dont seem to send requests for a long time (a lot more
than 2 sec), looking at tcpdump.

What do you think could be problems. Can you suggest any other way to
achieve the above test setup. I dont want to use the
TRANSFER_RATE_LIMIT since I want the clients to fetch it at full BW.



> If you wish to add such support, I can guide you.
>
>
>
> Sincerely,
> Robert Iakobashvili
> "Light will come from Jerusalem"
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>



-- 

------------------------------
http://pd.dnsalias.org

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Pranav,

On Feb 16, 2008 10:18 PM, Pranav Desai <pra...@gm...> wrote:
> Great !
> Thanks for the help.
>
> Do you guys have any plans for adding proxy support ... ?

It is rather easy. Since libcurl has such support, what you need to do
is to add for the new configuration file tags treatment in parser, and
to set such options to the libcurl handle, CURL* object.

If you wish to add such support, I can guide you.


Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: TIMER_AFTER_URL_SLEEP question

[Pranav D. <pra...@gm...>]

Great !
Thanks for the help.

Do you guys have any plans for adding proxy support ... ?

Thanks again.

-- Pranav

On Feb 16, 2008 8:06 AM, Robert Iakobashvili <cor...@gm...> wrote:
> Hi Pranav,
>
> On 2/16/08, Pranav Desai <pra...@gm...> wrote:
> > Hello All,
> >
> > I am working with version 0.44. I was curious about how the
> > TIMER_AFTER_URL_SLEEP option works. I have set it up fairly high
> > 20000, but I still seem to get like 70+ requests every 3 sec with just
> > one client. I was expecting it to have 1 request every 20 sec. I am
> > trying to control the load, so what am I missing here.
>
> Your intentions are clear. The syntax of your batch file
> should be corrected in the URL section:
>
>
> > ########### URL SECTION ####################################
>
>
> The below syntax is not supported.
>
> The below:
> > URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/
> > URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/swa/c/home.css
> > URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/03.gif
> > URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/17.gif
> >
>
> Y can correct it to the following:
>
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/
> URL_SHORT_NAME="url-main"
> REQUEST_TYPE=GET
> TIMER_URL_COMPLETION = 0
> TIMER_AFTER_URL_SLEEP =0
>
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/swa/c/home.css
> URL_SHORT_NAME="url-css"
> REQUEST_TYPE=GET
> TIMER_URL_COMPLETION = 0
> TIMER_AFTER_URL_SLEEP =0
>
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/03.gif
> URL_SHORT_NAME="url-03-gif"
> REQUEST_TYPE=GET
> TIMER_URL_COMPLETION = 0
> TIMER_AFTER_URL_SLEEP =0
>
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/17.gif
> URL_SHORT_NAME="url-17-gif"
> REQUEST_TYPE=GET
> TIMER_URL_COMPLETION = 0
> TIMER_AFTER_URL_SLEEP =20000
>
> So that you sleep, when fetched the whole page.
>
> Best wishes!
>
> Sincerely,
> Robert Iakobashvili
> "Light will come from Jerusalem"
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>



-- 

------------------------------
http://pd.dnsalias.org

Re: TIMER_AFTER_URL_SLEEP question

[Robert I. <cor...@gm...>]

Hi Pranav,

On 2/16/08, Pranav Desai <pra...@gm...> wrote:
> Hello All,
>
> I am working with version 0.44. I was curious about how the
> TIMER_AFTER_URL_SLEEP option works. I have set it up fairly high
> 20000, but I still seem to get like 70+ requests every 3 sec with just
> one client. I was expecting it to have 1 request every 20 sec. I am
> trying to control the load, so what am I missing here.

Your intentions are clear. The syntax of your batch file
should be corrected in the URL section:


> ########### URL SECTION ####################################


The below syntax is not supported.

The below:
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/swa/c/home.css
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/03.gif
> URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/17.gif
>

Y can correct it to the following:

URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/
URL_SHORT_NAME="url-main"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0
TIMER_AFTER_URL_SLEEP =0

URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/swa/c/home.css
URL_SHORT_NAME="url-css"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0
TIMER_AFTER_URL_SLEEP =0

URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/03.gif
URL_SHORT_NAME="url-03-gif"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0
TIMER_AFTER_URL_SLEEP =0

URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/17.gif
URL_SHORT_NAME="url-17-gif"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0
TIMER_AFTER_URL_SLEEP =20000

So that you sleep, when fetched the whole page.

Best wishes!

Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

TIMER_AFTER_URL_SLEEP question

[Pranav D. <pra...@gm...>]

Hello All,

I am working with version 0.44. I was curious about how the
TIMER_AFTER_URL_SLEEP option works. I have set it up fairly high
20000, but I still seem to get like 70+ requests every 3 sec with just
one client. I was expecting it to have 1 request every 20 sec. I am
trying to control the load, so what am I missing here.

 Thanks
-- Pranav

P.S.
Here is my config.

########### GENERAL SECTION ################################

BATCH_NAME=small
CLIENTS_NUM_MAX=1 # Same as CLIENTS_NUM
CLIENTS_NUM_START=1
CLIENTS_RAMPUP_INC=1
INTERFACE   =eth1
NETMASK=16
IP_ADDR_MIN= 12.0.0.1
IP_ADDR_MAX= 12.0.1.250 #Actually - this is for self-control
CYCLES_NUM=2000
URLS_NUM=4

########### URL SECTION ####################################

URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/
URL=http://172.16.55.200/MJOLNIRRAND/websites/cisco/www.cisco.com/swa/c/home.css
URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/03.gif
URL=http://172.16.55.200/MJOLNIRRAND/websites/cnn/i.l.cnn.net/cnn/.element/img/2.0/weather/03/17.gif

URL_SHORT_NAME="url-http"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0 # In msec. When positive, Now it is enforced
by cancelling url fetch on timeout
TIMER_AFTER_URL_SLEEP =20000

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Jan 10, 2008 5:40 AM, aaron smith <bei...@gm...>
wrote:

> Thanks Robert, I guess that's unfortunate. However jmeter was very easy to
> setup and it works just as expected. Thank you for your time helping me.
> -Aaron
>
> JMeter is a great tool, that we like.
curl-loader is supposed to deliver a high-performance and heavy load.
When the fix for PUT instead POST in either libcurl or our workaround will
be available, I'll inform you.

Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

Thanks Robert, I guess that's unfortunate. However jmeter was very easy to
setup and it works just as expected. Thank you for your time helping me.
-Aaron

On Jan 5, 2008 1:02 AM, Robert Iakobashvili <cor...@gm...> wrote:

> Aaron,
>
> On Jan 5, 2008 1:16 AM, aaron smith <bei...@gm...>
> wrote:
>
> >
> > Basically what's happening is the content from the file is not being put
> > into the body. But It does grab the content-length correctly. It seems
> to me
> > everything is ok, except the binary data isn't being attached..
>
> You are correct on your side. The problem seen in wireshark is that
> curl-loader was sending PUT and not POST.
>
> This is due to verification by libcurl, the HTTP stack library, that
> we are using.
> I am sorry to conclude, that it will require patching of libcurl and
> more time on our side to fix the issue.
>
> Y can try to use JMeter in meanwhile.
>
>
> Sincerely,
> Robert Iakobashvili
> "Light will come from Jerusalem"
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Jan 5, 2008 1:16 AM, aaron smith <bei...@gm...> wrote:

>
> Basically what's happening is the content from the file is not being put
> into the body. But It does grab the content-length correctly. It seems to me
> everything is ok, except the binary data isn't being attached..

You are correct on your side. The problem seen in wireshark is that
curl-loader was sending PUT and not POST.

This is due to verification by libcurl, the HTTP stack library, that
we are using.
I am sorry to conclude, that it will require patching of libcurl and
more time on our side to fix the issue.

Y can try to use JMeter in meanwhile.


Sincerely,
Robert Iakobashvili
"Light will come from Jerusalem"
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

also, one other thing. the domain www.lvr.com is a hosts file entry that
points to 206.188.30.110..

On Jan 4, 2008 3:16 PM, aaron smith <bei...@gm...> wrote:

> Hello Robert.
>
> So I finally got around to testing this all out..
>
> I've got curl-loader built from SVN. I've attached the configuration file
> I'm using for curl-loader(amf.conf). A wireshark session capture of just
> one request. And the binary file that I'm trying to attach in the body.
>
> Basically what's happening is the content from the file is not being put
> into the body. But It does grab the content-length correctly. It seems to me
> everything is ok, except the binary data isn't being attached..
>
> Can you check out what I've attached and see if you see anything obvious
> that is incorrect?
>
> Thanks much.
> Aaron
>
>
> ---------- Forwarded message ----------
> From: Robert Iakobashvili <cor...@gm...>
> Date: Dec 16, 2007 4:54 AM
> Subject: Re: a few questions.. huge load testing job
> To: web loading and performance testing tool <
> cur...@li...>
>
>
> Aaron,
>
> On Dec 10, 2007 7:04 PM, aaron smith < bei...@gm...>
> wrote:
>
> > Thanks Robert! I''ll be trying this out sometime today or tomorrow and
> > let you know what happens..
> >
> >
> > > POST //amfphp/gateway.php HTTP/1.1
> > > Accept: */*
> > > Accept-Language: en-US
> > > Referer: http://www.petmoustache.com/global/swf/main.swf
> > > x-flash-version<http://www.petmoustache.com/global/swf/main.swfx-flash-version>:
> > > 9,0,115,0
> > > Content-Type: application/x-amf
> > > Content-Length: 55561
> > > Accept-Encoding: gzip, deflate
> > > User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
> > > Host: petmoustache.com
> > > Connection: Keep-Alive
> > > Cache-Control: no-cache
> > > Cookie: __utma=110968324.1943311580.1197250082.1197250082.1197250082.1
> > > ;
> > > __utmb=110968324; __utmc=110968324;
> > > __utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
> > > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff
> > >
> > > -----------------------------------------------------------------------------
> > > body
> > > -----------------------------------------------------------------------------
> > >
> > >
> > > This is the POST, which we see.
> > >
> > > The url to be used seems to be:
> > > http://petmoustache.com/amfphp/gateway.php
> > >
> > > The essential header to be added via HEADER-tag up to me are:
> > >
> > > HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf "
> > > HEADER="x-flash-version: 9,0,115,0"
> > > HEADER="Content-Type: application/x-amf"
> > > HEADER="Cache-Control: no-cache"
> > > HEADER="Cookie:
> > > __utma=110968324.1943311580.1197250082.1197250082.1197250082.1 ;
> > > __utmb=110968324; __utmc=110968324;
> > > __utmz=110968324.1197250082.1.1.utmccn=
> > > (direct)|utmcsr=(direct)|utmcmd=(none);
> > > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
> > >
> > > Note, that Cookie is a separate story, which is generated per user and
> > >
> > > it depends on
> > > the policy of you web-server. Y may need to play with it or without
> > > it, etc.
> > >
> > > What the POST is doing is actually uploading some binary file.
> > > We have such an example for the PUT method, namely put.conf.
> > >
> > > Your url section may be similar:
> > >
> > > --------------------------------------------------------------------------------------------------------
> > > URL= http://petmoustache.com/amfphp/gateway.php
> > > URL_SHORT_NAME="POST-file"
> > > REQUEST_TYPE=POST
> > > UPLOAD_FILE=Atterntion!!!** place a path to your file from
> > > curl-loader, like ./myfile***
> > > HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
> > > HEADER="x-flash-version: 9,0,115,0"
> > > HEADER="Content-Type: application/x-amf"
> > > HEADER="Cache-Control: no-cache"
> > > HEADER="Cookie:
> > > __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> > > __utmb=110968324; __utmc=110968324;
> > > __utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
> > >
> > > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
> > > TIMER_URL_COMPLETION = 0  # In msec. When positive, Now it is enforced
> > > by cancelling url fetch on timeout
> > > TIMER_AFTER_URL_SLEEP =3000
> > >
> > > Please, run that (sure on linux) with a single client
> > > and a single cycle with -v and -u options also added (verbose
> > > debugging) to curl-loader
> > > command line (learn for -f <batch file> and other options) and
> > > wireshark capture to see, if there are any problems.
> > >
> > > If it works, start increasing, etc.
> > > If not, send to me:
> > > a) the log of curl-loader <your-batch-name>.log;
> > > b) the wireshark capture.
> > >
> > >
> Better late, than nether.
> Occasionally, I seen, that our checks do not allow POST-ing
> of a binary file without any POST-form.
>
> The error message will be something like: error: post_data is NULL.
>
> Therefore, I have committed a small patch.
> Y can either checkout the latest svn code, or apply the patch attached.
>
> Best wishes.
>
> --
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
>
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>
>
>

Re: curl-loader enhancement

[Jeremy B. <jb...@no...>]

Robert,

I am just getting back from vacation so I have put in on the hold for a =
little bit. But I do have lots of questions to ask different features and =
the organization of the code that I will get back to you.=20

Thanks
Jeremy

>>> On Thu, Dec 20, 2007 at  6:51 AM, in message
<7e6...@ma...>, "Robert
Iakobashvili" <cor...@gm...> wrote:=20
> Jeremy,
>=20
> On Nov 29, 2007 8:56 PM, Jeremy Brown <jb...@no...> wrote:
>=20
>> Robert,
>>
>> That gives me a good place to start but I am thinking of having a file =
of
>> thousands of urls. So I think I will just use some of the functionality =
from
>> the RECORDS_FROM_FILE.
>=20
>=20
> How it goes? Y are welcome to ask an advise.
>=20
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.

Re: curl-loader enhancement

[Robert I. <cor...@gm...>]

Jeremy,

On Nov 29, 2007 8:56 PM, Jeremy Brown <jb...@no...> wrote:

> Robert,
>
> That gives me a good place to start but I am thinking of having a file of
> thousands of urls. So I think I will just use some of the functionality from
> the RECORDS_FROM_FILE.


How it goes? Y are welcome to ask an advise.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: HTTP Soap Post

[Robert I. <cor...@gm...>]

My Dear Huy,

>From the recent message to Aaron to the list
you may guess, that the latest svn version is supposed
do POST-ing upload of a SOAP or any other file.

URL=your url <http://petmoustache.com/amfphp/gateway.php>
URL_SHORT_NAME="POST-file"
REQUEST_TYPE=POST
UPLOAD_FILE=place a path to your file from
curl-loader process, like ./my_SOAP_FILE

Customize your request HTTP headers using
HEADER tag as described in FAQs.

With my best wishes.


Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Dec 10, 2007 7:04 PM, aaron smith <bei...@gm...>
wrote:

> Thanks Robert! I''ll be trying this out sometime today or tomorrow and let
> you know what happens..
>
>
> > POST //amfphp/gateway.php HTTP/1.1
> > Accept: */*
> > Accept-Language: en-US
> > Referer: http://www.petmoustache.com/global/swf/main.swf
> > x-flash-version<http://www.petmoustache.com/global/swf/main.swfx-flash-version>:
> > 9,0,115,0
> > Content-Type: application/x-amf
> > Content-Length: 55561
> > Accept-Encoding: gzip, deflate
> > User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
> > Host: petmoustache.com
> > Connection: Keep-Alive
> > Cache-Control: no-cache
> > Cookie: __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> > __utmb=110968324; __utmc=110968324;
> > __utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
> > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff
> >
> > -----------------------------------------------------------------------------
> > body
> > -----------------------------------------------------------------------------
> >
> >
> > This is the POST, which we see.
> >
> > The url to be used seems to be:
> > http://petmoustache.com/amfphp/gateway.php
> >
> > The essential header to be added via HEADER-tag up to me are:
> >
> > HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf "
> > HEADER="x-flash-version: 9,0,115,0"
> > HEADER="Content-Type: application/x-amf"
> > HEADER="Cache-Control: no-cache"
> > HEADER="Cookie:
> > __utma=110968324.1943311580.1197250082.1197250082.1197250082.1 ;
> > __utmb=110968324; __utmc=110968324;
> > __utmz=110968324.1197250082.1.1.utmccn=
> > (direct)|utmcsr=(direct)|utmcmd=(none);
> > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
> >
> > Note, that Cookie is a separate story, which is generated per user and
> > it depends on
> > the policy of you web-server. Y may need to play with it or without it,
> > etc.
> >
> > What the POST is doing is actually uploading some binary file.
> > We have such an example for the PUT method, namely put.conf.
> >
> > Your url section may be similar:
> >
> > --------------------------------------------------------------------------------------------------------
> > URL= http://petmoustache.com/amfphp/gateway.php
> > URL_SHORT_NAME="POST-file"
> > REQUEST_TYPE=POST
> > UPLOAD_FILE=Atterntion!!!** place a path to your file from
> > curl-loader, like ./myfile***
> > HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
> > HEADER="x-flash-version: 9,0,115,0"
> > HEADER="Content-Type: application/x-amf"
> > HEADER="Cache-Control: no-cache"
> > HEADER="Cookie:
> > __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> > __utmb=110968324; __utmc=110968324;
> > __utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
> >
> > PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
> > TIMER_URL_COMPLETION = 0  # In msec. When positive, Now it is enforced
> > by cancelling url fetch on timeout
> > TIMER_AFTER_URL_SLEEP =3000
> >
> > Please, run that (sure on linux) with a single client
> > and a single cycle with -v and -u options also added (verbose
> > debugging) to curl-loader
> > command line (learn for -f <batch file> and other options) and
> > wireshark capture to see, if there are any problems.
> >
> > If it works, start increasing, etc.
> > If not, send to me:
> > a) the log of curl-loader <your-batch-name>.log;
> > b) the wireshark capture.
> >
> >
Better late, than nether.
Occasionally, I seen, that our checks do not allow POST-ing
of a binary file without any POST-form.

The error message will be something like: error: post_data is NULL.

Therefore, I have committed a small patch.
Y can either checkout the latest svn code, or apply the patch attached.

Best wishes.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: Full Page rendering

[Robert I. <cor...@gm...>]

Rick,

On Dec 11, 2007 11:17 PM, Richard Parker <rp...@ce...> wrote:
> How many objects does curl-loader currently support in the config file?

It's called URL. Unlimited. Y can even mix HTTP. HTTPS. FTP. FTPS urls
in one batch (testing configuration plan) file.

Actually. limited by the memory and "long" in C-lang limits.
:-)

> Rick

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

RE: Full Page rendering

[Richard P. <rp...@ce...>]

Hello Robert,
How many objects does curl-loader currently support in the config file?
Thanks,
Rick



rick parker : rp...@ce... :  d 781 425 5200 x224 : www.certeon.com
certeon - Accelerate Your Business


-----Original Message-----
From: cur...@li... [mailto:curl-loader-d=
eve...@li...] On Behalf Of Robert Iakobashvili
Sent: Tuesday, December 11, 2007 3:51 PM
To: web loading and performance testing tool
Subject: Re: Full Page rendering

Hi Rick
On Dec 11, 2007 9:41 PM, Richard Parker <rp...@ce...> wrote:
> I would like to know if there is a way to point curl-loader to download a=
nd
> activate the full asp page when told to go to it.

If you mean not to fetch a single url, but to analyse all links and crawl t=
hem,
this is the feature in our RoadMap, but not developed yet.

People are work-arounding it by placing each image or internal link as
a separate
URL.

> Rick

--
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
curl-loader-devel mailing list
cur...@li...
https://lists.sourceforge.net/lists/listinfo/curl-loader-devel

Re: Full Page rendering

[Robert I. <cor...@gm...>]

Hi Rick
On Dec 11, 2007 9:41 PM, Richard Parker <rp...@ce...> wrote:
> I would like to know if there is a way to point curl-loader to download and
> activate the full asp page when told to go to it.

If you mean not to fetch a single url, but to analyse all links and crawl them,
this is the feature in our RoadMap, but not developed yet.

People are work-arounding it by placing each image or internal link as
a separate
URL.

> Rick

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Full Page rendering

[Richard P. <rp...@ce...>]

Hello,
I would like to know if there is a way to point curl-loader to download and=
 activate the full asp page when told to go to it.
I have searched through the archives and did not find anything regarding it=
.
Thanks,
Rick

Re: compiling curl-loader for mac.

[aaron s. <bei...@gm...>]

yeah no worries, I was just trying to get around having to screw with linux,
it's always so much more work than it should be.

On Dec 10, 2007 9:34 AM, Robert Iakobashvili <cor...@gm...> wrote:

> Aaron,
>
> On Dec 10, 2007 7:03 PM, aaron smith <bei...@gm...>
> wrote:
> > yeah my problem was that there is no documentation for bugnux. I can
> > boot/login fine, but anything beyond that is past me at this point. I
> wasn't
> > even sure how to get on the internet with it. dah. I'll be trying it out
> > again today.
>
> You are going to test a rather serious web-site.
> Don't you think that you need at least a good testing PC-computer?
>
> We would recommend a PC with Intel Quad-core CPU and 2G memory, whereas
> dual-core PC can be found for a really bargain prices (2G memory is
> recommended).
> Y can make a dual-boot computer with XP and some linux on the same HD
> (read
> HOW-TOWS, windows to be installed first) or a pure linux.
>
> With all our experience in the field we have nether even heard about
> using MAC for load/stress
> testing. (I do personally like MACs and was using them since 1993 for
> desktop).
>
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>