curl-loader-devel — curl-loader usage and development issues

Re: compiling curl-loader for mac.

[Robert I. <cor...@gm...>]

Aaron,

On Dec 10, 2007 7:03 PM, aaron smith <bei...@gm...> wrote:
> yeah my problem was that there is no documentation for bugnux. I can
> boot/login fine, but anything beyond that is past me at this point. I wasn't
> even sure how to get on the internet with it. dah. I'll be trying it out
> again today.

You are going to test a rather serious web-site.
Don't you think that you need at least a good testing PC-computer?

We would recommend a PC with Intel Quad-core CPU and 2G memory, whereas
dual-core PC can be found for a really bargain prices (2G memory is
recommended).
Y can make a dual-boot computer with XP and some linux on the same HD (read
HOW-TOWS, windows to be installed first) or a pure linux.

With all our experience in the field we have nether even heard about
using MAC for load/stress
testing. (I do personally like MACs and was using them since 1993 for desktop).


Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

Thanks Robert! I''ll be trying this out sometime today or tomorrow and let
you know what happens..

-A

On Dec 9, 2007 11:54 PM, Robert Iakobashvili <cor...@gm...> wrote:

> Aaron,
>
> On Dec 10, 2007 3:45 AM, aaron smith <bei...@gm...>
> wrote:
> > I would appreciate the help so much, and a patch would be awesome!
> Thanks!
> > Here is a wireshark dump.
>
> POST //amfphp/gateway.php HTTP/1.1
> Accept: */*
> Accept-Language: en-US
> Referer: http://www.petmoustache.com/global/swf/main.swf
> x-flash-version: 9,0,115,0
> Content-Type: application/x-amf
> Content-Length: 55561
> Accept-Encoding: gzip, deflate
> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
> Host: petmoustache.com
> Connection: Keep-Alive
> Cache-Control: no-cache
> Cookie: __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> __utmb=110968324; __utmc=110968324;
> __utmz=110968324.1197250082.1.1.utmccn=
> (direct)|utmcsr=(direct)|utmcmd=(none);
> PHPSESSID=70624e053d197b61ce5e704d45f1a5ff
>
> -----------------------------------------------------------------------------
> body
>
> -----------------------------------------------------------------------------
>
> This is the POST, which we see.
>
> The url to be used seems to be:
> http://petmoustache.com/amfphp/gateway.php
>
> The essential header to be added via HEADER-tag up to me are:
>
> HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
> HEADER="x-flash-version: 9,0,115,0"
> HEADER="Content-Type: application/x-amf"
> HEADER="Cache-Control: no-cache"
> HEADER="Cookie:
> __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> __utmb=110968324; __utmc=110968324;
> __utmz=110968324.1197250082.1.1.utmccn=
> (direct)|utmcsr=(direct)|utmcmd=(none);
> PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
>
> Note, that Cookie is a separate story, which is generated per user and
> it depends on
> the policy of you web-server. Y may need to play with it or without it,
> etc.
>
> What the POST is doing is actually uploading some binary file.
> We have such an example for the PUT method, namely put.conf.
>
> Your url section may be similar:
>
> --------------------------------------------------------------------------------------------------------
> URL=http://petmoustache.com/amfphp/gateway.php
> URL_SHORT_NAME="POST-file"
> REQUEST_TYPE=POST
> UPLOAD_FILE=Atterntion!!!** place a path to your file from
> curl-loader, like ./myfile***
> HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
> HEADER="x-flash-version: 9,0,115,0"
> HEADER="Content-Type: application/x-amf"
> HEADER="Cache-Control: no-cache"
> HEADER="Cookie:
> __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
> __utmb=110968324; __utmc=110968324;
> __utmz=110968324.1197250082.1.1.utmccn=
> (direct)|utmcsr=(direct)|utmcmd=(none);
> PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
> TIMER_URL_COMPLETION = 0  # In msec. When positive, Now it is enforced
> by cancelling url fetch on timeout
> TIMER_AFTER_URL_SLEEP =3000
>
> Please, run that (sure on linux) with a single client
> and a single cycle with -v and -u options also added (verbose
> debugging) to curl-loader
> command line (learn for -f <batch file> and other options) and
> wireshark capture to see, if there are any problems.
>
> If it works, start increasing, etc.
> If not, send to me:
> a) the log of curl-loader <your-batch-name>.log;
> b) the wireshark capture.
>
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: compiling curl-loader for mac.

[aaron s. <bei...@gm...>]

yeah my problem was that there is no documentation for bugnux. I can
boot/login fine, but anything beyond that is past me at this point. I wasn't
even sure how to get on the internet with it. dah. I'll be trying it out
again today.

On Dec 9, 2007 10:39 PM, Robert Iakobashvili <cor...@gm...> wrote:

> On Dec 10, 2007 7:39 AM, aaron smith <bei...@gm...>
> wrote:
> > shoot. I didn't see that. hm. will it literally not work on mac or just
> no
> > one has tried compiling it? either way I guess bugnux it is..
>
> curl-loader is linux only.
>
> If you can simulate linux on MAC, or install linux on MAC HW, you can do
> it.
>
> Bugnux is a linux Live-CD distribution, containing already built
> curl-loader.
>
> Y can boot from the Bugnux CD (at least on windows) and work as you are
> at linux-installed machine. I do not think, that somebody tried Bugnux on
> MAC,
> but booting from windows-installed PC HW works.
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Dec 10, 2007 3:45 AM, aaron smith <bei...@gm...> wrote:
> I would appreciate the help so much, and a patch would be awesome! Thanks!
> Here is a wireshark dump.

POST //amfphp/gateway.php HTTP/1.1
Accept: */*
Accept-Language: en-US
Referer: http://www.petmoustache.com/global/swf/main.swf
x-flash-version: 9,0,115,0
Content-Type: application/x-amf
Content-Length: 55561
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: petmoustache.com
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: __utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
__utmb=110968324; __utmc=110968324;
__utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
PHPSESSID=70624e053d197b61ce5e704d45f1a5ff
-----------------------------------------------------------------------------
body
-----------------------------------------------------------------------------

This is the POST, which we see.

The url to be used seems to be:
http://petmoustache.com/amfphp/gateway.php

The essential header to be added via HEADER-tag up to me are:

HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
HEADER="x-flash-version: 9,0,115,0"
HEADER="Content-Type: application/x-amf"
HEADER="Cache-Control: no-cache"
HEADER="Cookie:
__utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
__utmb=110968324; __utmc=110968324;
__utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"

Note, that Cookie is a separate story, which is generated per user and
it depends on
the policy of you web-server. Y may need to play with it or without it, etc.

What the POST is doing is actually uploading some binary file.
We have such an example for the PUT method, namely put.conf.

Your url section may be similar:
--------------------------------------------------------------------------------------------------------
URL=http://petmoustache.com/amfphp/gateway.php
URL_SHORT_NAME="POST-file"
REQUEST_TYPE=POST
UPLOAD_FILE=Atterntion!!!** place a path to your file from
curl-loader, like ./myfile***
HEADER="Referer: http://www.petmoustache.com/global/swf/main.swf"
HEADER="x-flash-version: 9,0,115,0"
HEADER="Content-Type: application/x-amf"
HEADER="Cache-Control: no-cache"
HEADER="Cookie:
__utma=110968324.1943311580.1197250082.1197250082.1197250082.1;
__utmb=110968324; __utmc=110968324;
__utmz=110968324.1197250082.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
PHPSESSID=70624e053d197b61ce5e704d45f1a5ff"
TIMER_URL_COMPLETION = 0  # In msec. When positive, Now it is enforced
by cancelling url fetch on timeout
TIMER_AFTER_URL_SLEEP =3000

Please, run that (sure on linux) with a single client
and a single cycle with -v and -u options also added (verbose
debugging) to curl-loader
command line (learn for -f <batch file> and other options) and
wireshark capture to see, if there are any problems.

If it works, start increasing, etc.
If not, send to me:
a) the log of curl-loader <your-batch-name>.log;
b) the wireshark capture.


Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: compiling curl-loader for mac.

[Robert I. <cor...@gm...>]

On Dec 10, 2007 7:39 AM, aaron smith <bei...@gm...> wrote:
> shoot. I didn't see that. hm. will it literally not work on mac or just no
> one has tried compiling it? either way I guess bugnux it is..

curl-loader is linux only.

If you can simulate linux on MAC, or install linux on MAC HW, you can do it.

Bugnux is a linux Live-CD distribution, containing already built curl-loader.

Y can boot from the Bugnux CD (at least on windows) and work as you are
at linux-installed machine. I do not think, that somebody tried Bugnux on MAC,
but booting from windows-installed PC HW works.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: compiling curl-loader for mac.

[aaron s. <bei...@gm...>]

shoot. I didn't see that. hm. will it literally not work on mac or just no
one has tried compiling it? either way I guess bugnux it is..

thanks

On Dec 9, 2007 9:31 PM, Robert Iakobashvili <cor...@gm...> wrote:

> I
>
> On Dec 10, 2007 4:31 AM, aaron smith <bei...@gm...>
> wrote:
> > robert, I thought I'd open a new thread for mac install, in case anyone
> else
> > comes across this. I think I just about got it..
> >
> > So anyway, I'm trying to install curl-loader on a Mac Intel. So far i've
> > done the following:
>
> It is explicitly stated, that curl-loader is linux only.
>
> On MAC try to use Bagnux LiveCD with a performance penalty.
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: compiling curl-loader for mac.

[Robert I. <cor...@gm...>]

I

On Dec 10, 2007 4:31 AM, aaron smith <bei...@gm...> wrote:
> robert, I thought I'd open a new thread for mac install, in case anyone else
> comes across this. I think I just about got it..
>
> So anyway, I'm trying to install curl-loader on a Mac Intel. So far i've
> done the following:

It is explicitly stated, that curl-loader is linux only.

On MAC try to use Bagnux LiveCD with a performance penalty.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: compiling curl-loader for mac.

[aaron s. <bei...@gm...>]

hm. strange. it's getting through all compilations correctly except when it
starts the compile for curl-loader, the last compile at the end of the make
file.. ($(OBJ_DIR)/%.o: %.c
    $(CC) $(CFLAGS) $(PROF_FLAG) $(OPT_FLAGS) $(DEBUG_FLAGS) $(INCDIR) -c -o
$(OBJ_DIR)/$*.o $<).

batch.c, which requires fdsetsize.h, the first include of bites/types.h
kills it. Which file is that supposed to be referencing? From curl /
openssl?

thanks for any help.


On Dec 9, 2007 6:31 PM, aaron smith <bei...@gm...> wrote:

> robert, I thought I'd open a new thread for mac install, in case anyone
> else comes across this. I think I just about got it..
>
> So anyway, I'm trying to install curl-loader on a Mac Intel. So far i've
> done the following:
>
> 1. update "openssl.sl" -> (OPENSSLDIR=/usr/include/openssl)
> 2. Update MakeFile (line 158)
>    from: cp -a $(CURL_BUILD)/include/curl ./inc/curl
>    to: cp -RL $(CURL_BUILD)/include/curl ./inc/curl
>
> I now run:
> sudo make cleanall
> sudo make
>
> The make is getting hung up on::
>
> gcc -W -Wall -Wpointer-arith -pipe -DCURL_LOADER_FD_SETSIZE=20000
> -D_FILE_OFFSET_BITS=64   -g -I. -I./inc -I/usr/include/openssl/include -c -o
> obj/batch.o batch.c
> In file included from batch.c:23:
> fdsetsize.h:27:24: error: bits/types.h: No such file or directory
> make: *** [obj/batch.o] Error 1
>
> In the file fdsetsize.h, it includes a bits/types.h, but that isn't here?
>
> Any ideas?
>
> Thanks all.
>

compiling curl-loader for mac.

[aaron s. <bei...@gm...>]

robert, I thought I'd open a new thread for mac install, in case anyone else
comes across this. I think I just about got it..

So anyway, I'm trying to install curl-loader on a Mac Intel. So far i've
done the following:

1. update "openssl.sl" -> (OPENSSLDIR=/usr/include/openssl)
2. Update MakeFile (line 158)
   from: cp -a $(CURL_BUILD)/include/curl ./inc/curl
   to: cp -RL $(CURL_BUILD)/include/curl ./inc/curl

I now run:
sudo make cleanall
sudo make

The make is getting hung up on::

gcc -W -Wall -Wpointer-arith -pipe -DCURL_LOADER_FD_SETSIZE=20000
-D_FILE_OFFSET_BITS=64   -g -I. -I./inc -I/usr/include/openssl/include -c -o
obj/batch.o batch.c
In file included from batch.c:23:
fdsetsize.h:27:24: error: bits/types.h: No such file or directory
make: *** [obj/batch.o] Error 1

In the file fdsetsize.h, it includes a bits/types.h, but that isn't here?

Any ideas?

Thanks all.

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

On Dec 7, 2007 9:32 AM, aaron smith <bei...@gm...> wrote:
> >Please, read about HTTP multipart form data POST-ing as in RFC1867
>
> I took a look at this. The key difference, and where the flash player (I
> think)breaks this standard, is with posting data. It doesn't conform to this
> multi-part rfc

It means, that it is not multipart form data, but rather a regular POST-ing.

I tried to make Charles Debugging Proxy working on my linux spending some time,
but it fails. Probably, jdk-6 is the reason, but nether mind, send to
me a wireshark
traffic capture for a single HTTP session, and I will tell what to do,
or, if necessary,
will patch curl-loader to support it.

Waiting for your wireshark capture.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

ok. thanks for all the help. I'm writing up a pretty big article about what
I'm doing for this kind of stress testing and remoting gateways. Their
aren't any for Flex that are good, so this could be huge for our community..
Thanks again for any help. -Aaron

On Dec 7, 2007 1:23 AM, Robert Iakobashvili <cor...@gm...> wrote:

> Aaron,
>
> On Dec 7, 2007 9:32 AM, aaron smith <bei...@gm...>
> wrote:
> I will look at it on Sunday or Mozei Shabbat, as we are on the eve of
> Shabbat
> here in Israel.
>
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Dec 7, 2007 9:32 AM, aaron smith <bei...@gm...> wrote:
I will look at it on Sunday or Mozei Shabbat, as we are on the eve of Shabbat
here in Israel.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

>Please, read about HTTP multipart form data POST-ing as in RFC1867

I took a look at this. The key difference, and where the flash player (I
think)breaks this standard, is with posting data. It doesn't conform to this
multi-part rfc. Specifically in section 3.3 it says "the HTTP protocol can
transport arbitrary BINARY data". That's what the flash player does. It
arbitrarily puts AMF into the body, and gives the content-type /
content-length. hmm.

Here is a sample conf I'm thinking will maybe do the trick.. Based off of
one of the examples..

########### URL SECTION ##################################

# GET-part
URL= http://10.30.6.42:8888/vax/root/Admin
URL_SHORT_NAME="Login-GET"
#URL_DONT_CYCLE = 1
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 4000
TIMER_AFTER_URL_SLEEP =0

# POST-part
URL=""
URL_SHORT_NAME="Login-POST"
URL_USE_CURRENT= 1
#URL_DONT_CYCLE = 1
REQUEST_TYPE=POST
MULTIPART_FORM_DATA="htmlcode=<HTML></HTML>;type=application/x-amf"
MULTIPART_FORM_DATA="file=@myamf.amf"

I'm also wondering about taking out the file=? so like so?
MULTIPART_FORM_DATA="@myamf.amf"

To try and get arbitrary binary in the body..

What do you think?

Anyway, I'm gonna try some of this out tomorrow. hopefully i'll be able to
get something working..

-Thanks
-Aaron


On Dec 6, 2007 11:01 PM, aaron smith <bei...@gm...>
wrote:

> >Examples were inside from the day one.
> They're not in 0.4.4. There aren't any make files either, It looks like
> 0.4.4. is a patch, lot's of files missing.
>
> >By the way, you can use Bugnux linux Live-CD curl-loader version
> >just to start from any computer, even Windows.
> >The Bugnux version, however, might have some performance penalty.
> Sweet! That's perfect.
>
> >Doe it have a linux version
> Yep. It's java.
>
> >It should be an example of posting a file. I will look into it later.
> >If it is not (somehow) supported
> >by curl-loader, it will be rather easy to add.
>
> Ah I think I got it. So basically when you post a file, it's just posting
> the binary data. So I can just post whatever file the AMF is in and that
> will be the AMF. Right? I don't see why that wouldn't work.
>
>
>
>
>
>
> On Dec 6, 2007 10:42 PM, Robert Iakobashvili <cor...@gm...> wrote:
>
> > Aaron,
> >
> > On Dec 7, 2007 8:33 AM, aaron smith <bei...@gm...>
> > wrote:
> > > >There are two ways of doing POST-ing:
> > > >a) a regular one;
> > > >b) a multiform POST-ing.
> > > What is the difference?
> >
> > Please, read about HTTP multipart form data POST-ing as in RFC1867
> >
> > > >Try to capture the traffic, remove all private info and send here to
> > see,
> > > >what happens.
> > >
> > > I've attached a Charles Debugging Proxy tool session. It can sniff and
> > > deserialize AMF. If you download Charles, you can go to file->open
> > session
> > > and open that file. It'll show you one request I made to a remoting
> > gateway.
> > > as well as the response.
> >
> > Doe it have a linux version
> >
> > >I can save the AMF to a file, but from
> > > curl-loader, how do I tell it to load that file as the content to
> > send?
> >
> > It should be an example of posting a file. I will look into it later.
> > If it is not (somehow) supported
> > by curl-loader, it will be rather easy to add.
> >
> > >
> > > Thanks for those examples btw. I didn't realize that 1.4.3 had them in
> > it. I
> > > DL 1.4.4, not in there.. 1.4.4 must be a patch eh. Anyway, for the
> > most part
> > > I see through the examples of what has to be done. Just this one issue
> > of
> > > associating the AMF as the content with it..
> >
> > Examples were inside from the day one.
> >
> > By the way, you can use Bugnux linux Live-CD curl-loader version
> > just to start from any computer, even Windows.
> > The Bugnux version, however, might have some performance penalty.
> > --
> > Sincerely,
> > Robert Iakobashvili,
> > coroberti %x40 gmail %x2e com
> > ...........................................................
> > http://curl-loader.sourceforge.net
> > An open-source web testing and traffic generation.
> >
> >
> > -------------------------------------------------------------------------
> > SF.Net email is sponsored by:
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > curl-loader-devel mailing list
> > cur...@li...
> > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
> >
>
>

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

>Examples were inside from the day one.
They're not in 0.4.4. There aren't any make files either, It looks like
0.4.4. is a patch, lot's of files missing.

>By the way, you can use Bugnux linux Live-CD curl-loader version
>just to start from any computer, even Windows.
>The Bugnux version, however, might have some performance penalty.
Sweet! That's perfect.

>Doe it have a linux version
Yep. It's java.

>It should be an example of posting a file. I will look into it later.
>If it is not (somehow) supported
>by curl-loader, it will be rather easy to add.

Ah I think I got it. So basically when you post a file, it's just posting
the binary data. So I can just post whatever file the AMF is in and that
will be the AMF. Right? I don't see why that wouldn't work.





On Dec 6, 2007 10:42 PM, Robert Iakobashvili <cor...@gm...> wrote:

> Aaron,
>
> On Dec 7, 2007 8:33 AM, aaron smith <bei...@gm...>
> wrote:
> > >There are two ways of doing POST-ing:
> > >a) a regular one;
> > >b) a multiform POST-ing.
> > What is the difference?
>
> Please, read about HTTP multipart form data POST-ing as in RFC1867
>
> > >Try to capture the traffic, remove all private info and send here to
> see,
> > >what happens.
> >
> > I've attached a Charles Debugging Proxy tool session. It can sniff and
> > deserialize AMF. If you download Charles, you can go to file->open
> session
> > and open that file. It'll show you one request I made to a remoting
> gateway.
> > as well as the response.
>
> Doe it have a linux version
>
> >I can save the AMF to a file, but from
> > curl-loader, how do I tell it to load that file as the content to send?
>
> It should be an example of posting a file. I will look into it later.
> If it is not (somehow) supported
> by curl-loader, it will be rather easy to add.
>
> >
> > Thanks for those examples btw. I didn't realize that 1.4.3 had them in
> it. I
> > DL 1.4.4, not in there.. 1.4.4 must be a patch eh. Anyway, for the most
> part
> > I see through the examples of what has to be done. Just this one issue
> of
> > associating the AMF as the content with it..
>
> Examples were inside from the day one.
>
> By the way, you can use Bugnux linux Live-CD curl-loader version
> just to start from any computer, even Windows.
> The Bugnux version, however, might have some performance penalty.
> --
> Sincerely,
> Robert Iakobashvili,
> coroberti %x40 gmail %x2e com
> ...........................................................
> http://curl-loader.sourceforge.net
> An open-source web testing and traffic generation.
>
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Aaron,

On Dec 7, 2007 8:33 AM, aaron smith <bei...@gm...> wrote:
> >There are two ways of doing POST-ing:
> >a) a regular one;
> >b) a multiform POST-ing.
> What is the difference?

Please, read about HTTP multipart form data POST-ing as in RFC1867

> >Try to capture the traffic, remove all private info and send here to see,
> >what happens.
>
> I've attached a Charles Debugging Proxy tool session. It can sniff and
> deserialize AMF. If you download Charles, you can go to file->open session
> and open that file. It'll show you one request I made to a remoting gateway.
> as well as the response.

Doe it have a linux version

>I can save the AMF to a file, but from
> curl-loader, how do I tell it to load that file as the content to send?

It should be an example of posting a file. I will look into it later.
If it is not (somehow) supported
by curl-loader, it will be rather easy to add.

>
> Thanks for those examples btw. I didn't realize that 1.4.3 had them in it. I
> DL 1.4.4, not in there.. 1.4.4 must be a patch eh. Anyway, for the most part
> I see through the examples of what has to be done. Just this one issue of
> associating the AMF as the content with it..

Examples were inside from the day one.

By the way, you can use Bugnux linux Live-CD curl-loader version
just to start from any computer, even Windows.
The Bugnux version, however, might have some performance penalty.
-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

Ok, i answered the question of associating amf in the body myself. Just
reaind some HTTP specs duh. http://www.jmarshall.com/easy/http/#structure

So the part that is the "body" will be some raw AMF. Is there a way to
attach a raw body from a file in curl-loader?. I'll keep reading too.
thanks.

On Dec 6, 2007 10:33 PM, aaron smith <bei...@gm...>
wrote:

> >There are two ways of doing POST-ing:
> >a) a regular one;
> >b) a multiform POST-ing.
>
> What is the difference?
>
> >Try to capture the traffic, remove all private info and send here to see,
> >what happens.
>
> I've attached a Charles Debugging Proxy tool session. It can sniff and
> deserialize AMF. If you download Charles, you can go to file->open session
> and open that file. It'll show you one request I made to a remoting gateway.
> as well as the response.
>
> It's pretty basic, all that's happening is the content-type is set to
> application/x-amf. I'm just unsure of how to attach the correct request AMF
> with a request from curl loader.. I can save the AMF to a file, but from
> curl-loader, how do I tell it to load that file as the content to send?
>
> Thanks for those examples btw. I didn't realize that 1.4.3 had them in it.
> I DL 1.4.4, not in there.. 1.4.4 must be a patch eh. Anyway, for the most
> part I see through the examples of what has to be done. Just this one issue
> of associating the AMF as the content with it..
>
> Thanks..
>
>
>
>
>
> On Dec 6, 2007 10:02 PM, Robert Iakobashvili <cor...@gm... > wrote:
>
> > Hi Aaron,
> >
> > On Dec 7, 2007 7:32 AM, aaron smith < bei...@gm...>
> > wrote:
> > > -This should work on a mac right? (intel mac)
> >
> > It works on linux. If you have managed to run linux on some MAC HW
> > - you can try.
> >
> > > -Are there any other documentation/examples of using curl-loader?
> >
> > Examples of the batches (testing plans) are inside the tarball in
> > conf-examples directory. The FAQs are here:
> > http://curl-loader.sourceforge.net/doc/faq.html
> > even more docs are here:
> > http://curl-loader.sourceforge.net
> >
> > > The request from the flash player comes over as a POST, but i'm not
> > sure if
> > > the AMF comes over in a form variable? Usually in a remoting gateway I
> > can
> > > get the AMF from the request.raw_body. How does that correlate to a
> > request?
> >
> > Y can try to sniff the traffic, using wireshark (ethereal) and know,
> > what are
> > the HTTP request/responses.
> >
> > > The workflow that I see with curl loader is the batch file, changing
> > the
> > > header to supply the content-type of application/x-amf, and the
> > > content-length. Then associating the AMF from a file with the
> > content/body
> > > of the request.. The site is expected to be getting millions of hits
> > daily
> > > possibly weekly. In any case huge loads.
> >
> > There are two ways of doing POST-ing:
> > a) a regular one;
> > b) a multiform POST-ing.
> >
> > Try to capture the traffic, remove all private info and send here to
> > see,
> > what happens.
> >
> > > Has anything like this been accomplished with curl-loader?
> >
> > Let's say 10 mln of hits per day are only about 400 000 per hour, which
> > means only 7000 heats/minute. I do not see any problem for curl-loader f
> > rom the point of number of request.
> > Your scenario, however, may require high bandwidth, high CPU,
> > but a dual core Core2 Intel CPU with some 2G of memory should manage it.
> >
> >
> > The question is about your application part.
> >
> > > Thanks much for any help!
> > > -Aaron
> >
> > Sincerely,
> > Robert Iakobashvili,
> > coroberti %x40 gmail %x2e com
> > ...........................................................
> > http://curl-loader.sourceforge.net
> > An open-source web testing and traffic generation.
> >
> > -------------------------------------------------------------------------
> >
> > SF.Net email is sponsored by:
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > curl-loader-devel mailing list
> > cur...@li...
> > https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
> >
>
>

Re: a few questions.. huge load testing job

[Robert I. <cor...@gm...>]

Hi Aaron,

On Dec 7, 2007 7:32 AM, aaron smith <bei...@gm...> wrote:
> -This should work on a mac right? (intel mac)

It works on linux. If you have managed to run linux on some MAC HW
- you can try.

> -Are there any other documentation/examples of using curl-loader?

Examples of the batches (testing plans) are inside the tarball in
conf-examples directory. The FAQs are here:
http://curl-loader.sourceforge.net/doc/faq.html
even more docs are here:
http://curl-loader.sourceforge.net

> The request from the flash player comes over as a POST, but i'm not sure if
> the AMF comes over in a form variable? Usually in a remoting gateway I can
> get the AMF from the request.raw_body. How does that correlate to a request?

Y can try to sniff the traffic, using wireshark (ethereal) and know, what are
the HTTP request/responses.

> The workflow that I see with curl loader is the batch file, changing the
> header to supply the content-type of application/x-amf, and the
> content-length. Then associating the AMF from a file with the content/body
> of the request.. The site is expected to be getting millions of hits daily
> possibly weekly. In any case huge loads.

There are two ways of doing POST-ing:
a) a regular one;
b) a multiform POST-ing.

Try to capture the traffic, remove all private info and send here to see,
what happens.

> Has anything like this been accomplished with curl-loader?

Let's say 10 mln of hits per day are only about 400 000 per hour, which
means only 7000 heats/minute. I do not see any problem for curl-loader f
rom the point of number of request.
Your scenario, however, may require high bandwidth, high CPU,
but a dual core Core2 Intel CPU with some 2G of memory should manage it.

The question is about your application part.

> Thanks much for any help!
> -Aaron

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

a few questions.. huge load testing job

[aaron s. <bei...@gm...>]

Hey Guys,

Thanks for the work on curl loader. I've got a couple questions..

-This should work on a mac right? (intel mac)

-Are there any other documentation/examples of using curl-loader?

I've got a pretty big testing task ahead of me for a soon to be released
website. The site is built with Flex and uses the AMF binary protocol.
(Flash Remoting). The way AMF is transmitted from the player is over regular
HTTP, but the entire AMF message is in the body of the request. And the
content-type is application/x-amf.

The Remoting/AMF Gateway sits on the webserver, and handles Remote Object
calls from Flex. What I need to test is this remoting gateway, with a
combination of pre-saved AMF messages directly from the flash player, and
curl-loader.

I've got a generic solution for capturing an AMF request to a file so that I
can use the pre-generated AMF requests in combination with curl-loader. Is
it possible to specify a file's contents as the contents/body of a request?

The request from the flash player comes over as a POST, but i'm not sure if
the AMF comes over in a form variable? Usually in a remoting gateway I can
get the AMF from the request.raw_body. How does that correlate to a request?

The workflow that I see with curl loader is the batch file, changing the
header to supply the content-type of application/x-amf, and the
content-length. Then associating the AMF from a file with the content/body
of the request.. The site is expected to be getting millions of hits daily
possibly weekly. In any case huge loads.

Has anything like this been accomplished with curl-loader?

Thanks much for any help!
-Aaron

curl-loader gets donation from opensourcetesting

[Robert I. <cor...@gm...>]

Gentlemen,

curl-loader project is thankful to www.opensourcetesting.org
Mark Aberdour for his donation, great appreciation and support.

The donation will be used for VScript facility development.

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

[ANN] - curl-loader virtual scripting in SVN

[Robert I. <cor...@gm...>]

Gentlemen,

Michael Moser has checked-in a new system for HTTP/FTP response body analyses,
which have WebLoad and JMeter tools.

Normally, body analyses in testing tools is done using JavaScript or
any other script,
which occupies a total thread, is blocking and is bad for performance.

Since curl-loader is about performance (written in C, using C-written
stacks of libcurl and openssl)
 Michael is developing a new scripting language, where the script will be:

- compiled to an object code,
- possessing its own VM, not occupying a whole thread;
- enjoying several entrance and exit points.

Y can see the major development of the "Virtual Script -VScript", yet
without integration to the
curl-loader configurationand practical usage and without support for
64-bit linux here:

http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/script/

or even better by importing the svn repository and looking at curl-loader/script
directories: http://sourceforge.net/svn/?group_id=179599

Your comments would be appreciated, but, please, keep Michael's  e-mail
in the list.

curl-loader has been configured to receive donations via sourceforge, which
will be used to accelerate development and integration of VScript:
http://sourceforge.net/project/project_donations.php?group_id=179599


Have a nice CURL-ing.
:)

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: curl-loader enhancement

[Robert I. <cor...@gm...>]

On Nov 29, 2007 8:56 PM, Jeremy Brown <jb...@no...> wrote:

> That gives me a good place to start but I am thinking of having a file of thousands of urls. So I think I will just use some of the functionality from the RECORDS_FROM_FILE.

Good lack and mem, that you can always post patches for integration to
the mainstream!

> Jeremy

-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: curl-loader enhancement

[Jeremy B. <jb...@no...>]

Robert,

That gives me a good place to start but I am thinking of having a file of =
thousands of urls. So I think I will just use some of the functionality =
from the RECORDS_FROM_FILE.=20

thanks
Jeremy

>>> On Thu, Nov 29, 2007 at 11:17 AM, in message
<7e6...@ma...>, "Robert
Iakobashvili" <cor...@gm...> wrote:=20
> Hi Jeremy,
>=20
> On Nov 29, 2007 7:38 PM, Jeremy Brown <jb...@no...> wrote:
>> Robert,
>=20
> Please, address all corresponsence to the mailing list. Thanks.
>=20
>> I just used the login with random users with curl loader and it works =
great.
>=20
> Great :-)
>=20
>>Does it run with the same random username for the duration or does it
> get a random one each time it runs? I don't mind the >reusing of
> usernames but just want them to keep cycling through.
>=20
> Each time. Y can look in the code, are not you?
>=20
>> I wondered if you can add the ability to have random urls with a simple =
get=20
> request.
>=20
> Please, download the recent version:
>=20
> Y can look at examples fetch-probability-once.conf and fetch-probability.=
conf
> to use the sa-called URL fetching probability.
>=20
> Y can read the FAQs at:
> http://curl-loader.sourceforge.net/doc/faq.html
> namely:
> FETCH_PROBABILITY - allows to fetch a url not as a must, but with a
> certain run-time probability. The allowed values are in the range from
> 1 to 100 percents.
>=20
> FETCH_PROBABILITY_ONCE when set to 1 configures each client to make
> the decision regarding whether to fetch a URL marked by a
> FETCH_PROBABILITY or not, to be done only once, namely, at the first
> cycle.
>=20
> Define a list of URL, which will be fetched either with a fixed or
> with a per-cycle calculated
> probability
>=20
>>I am willing to donate $50 to you if you can get this fix into
> curl-loader as a one off for now and maybe as an enhancement >later. I
> just haven't had time to start working on the code yet. But I think
> you have done a fantastic job with it so far.
>=20
> Thanks for your appreciation.
> Please, note, that curl-loader developement is driven by open-source
> community needs, not by money.
>=20
> I am not accepting donations. curl-loader project does accept
> donations, where we have been informed by www.opensourcetesting.org ,
> that they are also willing to donate to us as well.
>=20
> It can be done here and for the purpose of vscript development, and
> not nothing else:
> http://sourceforge.net/project/project_donations.php?group_id=3D179599
>=20
> Take care.

Re: curl-loader enhancement

[Robert I. <cor...@gm...>]

Hi Jeremy,

On Nov 29, 2007 7:38 PM, Jeremy Brown <jb...@no...> wrote:
> Robert,

Please, address all corresponsence to the mailing list. Thanks.

> I just used the login with random users with curl loader and it works great.

Great :-)

>Does it run with the same random username for the duration or does it
get a random one each time it runs? I don't mind the >reusing of
usernames but just want them to keep cycling through.

Each time. Y can look in the code, are not you?

> I wondered if you can add the ability to have random urls with a simple get request.

Please, download the recent version:

Y can look at examples fetch-probability-once.conf and fetch-probability.conf
to use the sa-called URL fetching probability.

Y can read the FAQs at:
http://curl-loader.sourceforge.net/doc/faq.html
namely:
FETCH_PROBABILITY - allows to fetch a url not as a must, but with a
certain run-time probability. The allowed values are in the range from
1 to 100 percents.

FETCH_PROBABILITY_ONCE when set to 1 configures each client to make
the decision regarding whether to fetch a URL marked by a
FETCH_PROBABILITY or not, to be done only once, namely, at the first
cycle.

Define a list of URL, which will be fetched either with a fixed or
with a per-cycle calculated
probability

>I am willing to donate $50 to you if you can get this fix into
curl-loader as a one off for now and maybe as an enhancement >later. I
just haven't had time to start working on the code yet. But I think
you have done a fantastic job with it so far.

Thanks for your appreciation.
Please, note, that curl-loader developement is driven by open-source
community needs, not by money.

I am not accepting donations. curl-loader project does accept
donations, where we have been informed by www.opensourcetesting.org ,
that they are also willing to donate to us as well.

It can be done here and for the purpose of vscript development, and
not nothing else:
http://sourceforge.net/project/project_donations.php?group_id=179599

Take care.
-- 
Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Re: Is it possible to use data from the response header to a POST in a subsequent GET?

[Robert I. <cor...@gm...>]

Hi Ken,

On Nov 26, 2007 10:46 PM, Ken Mamitsuka <ke...@sh...> wrote:

> Hi, we have a web app that first takes a POST to a generic URL and returns
> an id in the POST response header. I'd like to use curl-loader to grab that
> unique id and the subsequently PUT to that id. >From a straight curl
> perspective, it'd look like:

If you know C and willing to write a patch, I can guide you about what is
necessary to do for that.

Sincerely,
Robert Iakobashvili,
coroberti %x40 gmail %x2e com
...........................................................
http://curl-loader.sourceforge.net
An open-source web testing and traffic generation.

Is it possible to use data from the response header to a POST in a subsequent GET?

[Ken M. <ke...@sh...>]

Hi, we have a web app that first takes a POST to a generic URL and
returns an id in the POST response header. I'd like to use curl-loader
to grab that unique id and the subsequently PUT to that id. From a
straight curl perspective, it'd look like:

=20

Curl -v -X POST <my url>

=20

The response header would have a line:

=20

< Location: http://<my url>/files/<my id>

=20

So, I'd grab <my id>

=20

And do:

=20

Curl -v -X PUT http://<my url>/files/<my id> -T <file to put>

=20

Is there any way to do this?

=20

Another option would be to pre-create all the <my id>'s and have them
read from a file, but it seems that curl-loader only likes reading
username/password pairs from a file.

=20

Thanks for any info!