curl-loader-devel — curl-loader usage and development issues

Re: explicit proxy setting for curl-loader-0.47

[Robert I. <cor...@gm...>]

Hi Frank,

On Wed, Jan 14, 2009 at 3:46 AM, <Fra...@ma...> wrote:

>
> Dear Curl-Loader maintainers,
>
> I needed a way to switch off or explicitly set the curl proxy string for
> curl-loader, without touching the environment or .curlrc.
> I couldn't find a hint in the docs or FAQ, so I implemented the "-x" option
> from curl in a quick and dirty way to curl-loader.
> Now I am able to do:
>
> ./curl-loader -x "" -f ssl-loadtest
>
> Where '-x ""' unsets the proxy. It might be of interest for others and I
> believe it is worth implementing, so I attach the diffs.
>
> I couldn't find quick enough the size for the proxy string, I simply set it
> to char[255] but it might be worth to look up or use the definition from the
> curl sources.
> I am also not sure if I placed the curl_easy_setopt (handle, CURLOPT_PROXY,
> config_proxy); call optimal in loader.c, But it works fine for me at the
> moment.
>
> Cheers,
> Frank
>
> fm@mlp09325:~/dev/curl-loader-0.47> diff conf.h.orig conf.h
> 114a115,120
> >    Name of the configuration proxy.
> > */
> > extern char config_proxy[254 + 1];
> >
> >
> > /*
>
>
> fm@mlp09325:~/dev/curl-loader-0.47> diff conf.c.orig conf.c
> 74a75,77
> > /* Name of the configuration file */
> > char config_proxy[254 + 1];
> >
> 87c90
> <     while ((rget_opt = getopt (argc, argv, "c:dehf:i:l:m:op:rst:vuw")) !=
> EOF)
> ---
> >     while ((rget_opt = getopt (argc, argv, "c:dehf:i:l:m:op:rst:vuwx:"))
> != EOF)
> 187a191,200
> >         case 'x': /* set/unset proxy */
> >           if (optarg)
> >             strcpy(config_proxy, optarg);
> >           else
> >             {
> >               fprintf (stderr, "%s error: -x option should be followed by
> a proxy IP or name.\n", __func__);
> >               return -1;
> >             }
> >           break;
> >
> 223a237
> >   fprintf (stderr, " -x[set|unset proxy]\n");
>
>
> fm@mlp09325:~/dev/curl-loader-0.47> diff loader.c loader.c.orig
> 483,485d482
> <
> <   /* set|unset the curl proxy */
> <   curl_easy_setopt (handle, CURLOPT_PROXY, config_proxy);
>

Thanks, looks good.

I would keep the buffer 1024.

Could you add explanations of the option to documentation, like README, man
page
and send to me a unified diff from diff -Naru ?
Thanks and your patch is appreciated.

Sincerely,
Robert

explicit proxy setting for curl-loader-0.47

[<Fra...@ma...>]

Dear Curl-Loader maintainers,

I needed a way to switch off or explicitly set the curl proxy string for 
curl-loader, without touching the environment or .curlrc.
I couldn't find a hint in the docs or FAQ, so I implemented the "-x" 
option from curl in a quick and dirty way to curl-loader.
Now I am able to do:

./curl-loader -x "" -f ssl-loadtest

Where '-x ""' unsets the proxy. It might be of interest for others and I 
believe it is worth implementing, so I attach the diffs. 

I couldn't find quick enough the size for the proxy string, I simply set 
it to char[255] but it might be worth to look up or use the definition 
from the curl sources.
I am also not sure if I placed the curl_easy_setopt (handle, 
CURLOPT_PROXY, config_proxy); call optimal in loader.c, But it works fine 
for me at the moment.

Cheers,
Frank

fm@mlp09325:~/dev/curl-loader-0.47> diff conf.h.orig conf.h
114a115,120
>    Name of the configuration proxy.
> */
> extern char config_proxy[254 + 1];
>
>
> /*


fm@mlp09325:~/dev/curl-loader-0.47> diff conf.c.orig conf.c
74a75,77
> /* Name of the configuration file */
> char config_proxy[254 + 1];
>
87c90
<     while ((rget_opt = getopt (argc, argv, "c:dehf:i:l:m:op:rst:vuw")) 
!= EOF)
---
>     while ((rget_opt = getopt (argc, argv, "c:dehf:i:l:m:op:rst:vuwx:")) 
!= EOF)
187a191,200
>         case 'x': /* set/unset proxy */
>           if (optarg)
>             strcpy(config_proxy, optarg);
>           else
>             {
>               fprintf (stderr, "%s error: -x option should be followed 
by a proxy IP or name.\n", __func__);
>               return -1;
>             }
>           break;
>
223a237
>   fprintf (stderr, " -x[set|unset proxy]\n");


fm@mlp09325:~/dev/curl-loader-0.47> diff loader.c loader.c.orig
483,485d482
<
<   /* set|unset the curl proxy */
<   curl_easy_setopt (handle, CURLOPT_PROXY, config_proxy);

Re: Does curl-loader support HTTP 1.0 ?

[Robert I. <cor...@gm...>]

Hi Meenakshi,

On Mon, Dec 29, 2008 at 12:52 PM, meenakshi syamkumar
<mee...@gm...> wrote:
> Hi,
>
> I tried what you had suggested. But still only HTTP 1.1 requests are being
> sent.
>
> Can you please explain it more. I need to konw what should my configuration
> file contain [if it should be compatible with HTTP 1.0].
> Thanks

It is not an issue of a configuration file.
If you have placed an option to force 1.0,
curl_easy_setopt (handle, CURLOPT_HTTP_VERSION , CURL_HTTP_VERSION_1_0);

and still 1.0 is not sent it means, that some other curl_easy_setopt
option in:

setup_curl_handle_init ()
or
setup_curl_handle_appl ()

forces it back to 1.1.

You may wish to try and comment them one by one to to find, which
makes a problem.


-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
Assistive technology that understands you
......................................................................

Re: Does curl-loader support HTTP 1.0 ?

[meenakshi s. <mee...@gm...>]

Hi,

I tried what you had suggested. But still only HTTP 1.1 requests are being
sent.

Can you please explain it more. I need to konw what should my configuration
file contain [if it should be compatible with HTTP 1.0].
Thanks

Meenakshi
On Wed, Dec 24, 2008 at 11:35 AM, Robert Iakobashvili
<cor...@gm...>wrote:

> Hi Meenakshi,
>
> On Wed, Dec 24, 2008 at 7:05 AM, meenakshi syamkumar
> <mee...@gm...> wrote:
> > Hi,
> >
> > I need to perform HTTP 1.0 testing as well as HTTP 1.1 in my test suite.
> >
> > So, I'd like to know whethercurl-loader supports HTTP 1.0. That is, can
> it
> > generate and send HTTP 1.0 traffic?
> >
> > Thanks,
> >
> > Meenakshi
>
> Since libcurl has such option, the code can be modified to do that.
>
> Y can add in loader.c function setup_curl_handle_init ()
> somewhere at the beginning:
>
> curl_easy_setopt (handle, CURLOPT_HTTP_VERSION , CURL_HTTP_VERSION_1_0);
>
> which means Enforce HTTP 1.0 requests.
>
> A possible issue is that it can potentially conflict with other options set
> in
> setup_curl_handle_init () or
> setup_curl_handle_appl ().
>
> Y might wish to investigate, whether is just works after the above
> modification,
> which is fast.
>
>
>
> --
> Truly,
> Robert Iakobashvili, Ph.D.
> ......................................................................
> www.ghotit.com
> Assistive technology that understands you
> ......................................................................
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>



-- 
Live LIFE Lively to make it Lovely

Re: Does curl-loader support HTTP 1.0 ?

[Robert I. <cor...@gm...>]

Hi Meenakshi,

On Wed, Dec 24, 2008 at 7:05 AM, meenakshi syamkumar
<mee...@gm...> wrote:
> Hi,
>
> I need to perform HTTP 1.0 testing as well as HTTP 1.1 in my test suite.
>
> So, I'd like to know whethercurl-loader supports HTTP 1.0. That is, can it
> generate and send HTTP 1.0 traffic?
>
> Thanks,
>
> Meenakshi

Since libcurl has such option, the code can be modified to do that.

Y can add in loader.c function setup_curl_handle_init ()
somewhere at the beginning:

curl_easy_setopt (handle, CURLOPT_HTTP_VERSION , CURL_HTTP_VERSION_1_0);

which means Enforce HTTP 1.0 requests.

A possible issue is that it can potentially conflict with other options set in
setup_curl_handle_init () or
setup_curl_handle_appl ().

Y might wish to investigate, whether is just works after the above modification,
which is fast.



-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Does curl-loader support HTTP 1.0 ?

[meenakshi s. <mee...@gm...>]

Hi,

I need to perform HTTP 1.0 testing as well as HTTP 1.1 in my test suite.

So, I'd like to know whethercurl-loader supports HTTP 1.0. That is, can it
generate and send HTTP 1.0 traffic?

Thanks,

Meenakshi


--
Live Life Lively to make it Lovely

Re: Test result summary - details

[Robert I. <cor...@gm...>]

Hi Alo,

On Thu, Dec 18, 2008 at 6:25 PM, alo sinnathamby <asi...@ic...>wrote:

> Hi Robert,
>
> Test total duration was 21 seconds and CAPS average 39:
> H/F
>
> Req:1788,1xx:0,2xx:1785,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:40ms,D-2xx:40ms,Ti:764258B/s,To:25762B/s
> H/F/S
>
> Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
> Operations: Success Failed Timed out
> URL0:my-GET *136 *914 0 0 0 0
> URL1:my-POST *136 *869 0 0 0 0
>
> when i get a test result summary as above, what does 136 mean? I gone
> thru all online docs including FAQs but unable to find any explanation
> for it. Please give me some clues.
>
>

Y can look in the sources, like in statistics.c:
http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/statistics.c?revision=484&view=markup

It means:

Success - 136 in the latest time interval (default 3 sec), 869 - total
Falied - 0 in the latest time interval (default 3 sec), 0 - total
Timed Out - 0 in the latest time interval (default 3 sec), 0 - total

If you think, that there are bugs in this reporting, or it can be improved,
please, fill free to fix it and propose patches for the mainstream
integration.

Best wishes!
-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Test result summary - details

[alo s. <asi...@ic...>]

Hi Robert,

Test total duration was 21 seconds and CAPS average 39:
H/F
Req:1788,1xx:0,2xx:1785,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:40ms,D-2xx:40ms,Ti:764258B/s,To:25762B/s
H/F/S
Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
Operations: Success Failed Timed out
URL0:my-GET *136 *914 0 0 0 0
URL1:my-POST *136 *869 0 0 0 0

when i get a test result summary as above, what does 136 mean? I gone
thru all online docs including FAQs but unable to find any explanation
for it. Please give me some clues.

Greatly appreciate your help.

Thanks,
Alo
NOTE: This message, and any attached files, contain information that is privileged, confidential, proprietary or otherwise protected from disclosure.  Any disclosure, copying or distribution of, or reliance upon, this message by anyone else is strictly prohibited.  If you received this communication in error, please notify the sender immediately by reply e-mail message or by telephone to one of the numbers above and deleting it from your computer.  Thank you.

curl-loader Presentation and Starter Guide in Chinese

[Robert I. <cor...@gm...>]

Hi folks,


You can find as a very good starter presentation in Chinese:

在线观看： http://www.boobooke.com/v/bbk1809 Online viewing:
http://www.boobooke.com/v/bbk1809

视频下载： http://www.boobooke.com/v/bbk1809.zip Download video:
http://www.boobooke.com/v/bbk1809.zip



Our best wishes with regards to Hanukka, Kurban and Christmas holidays,
whatever is appropriate.

-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Re: make error on Ubuntu8.04

[Robert I. <cor...@gm...>]

Hi Jian,

On Sun, Dec 14, 2008 at 11:42 PM, Jian Wang <ji...@gm...> wrote:
> oops, another error occurs:
>
> gcc  -g  -o curl-loader  obj/batch.o  obj/cl_alloc.o  obj/client.o
> obj/conf.o  obj/environment.o  obj/heap.o  obj/ip_secondary.o
> obj/loader.o  obj/loader_fsm.o  obj/loader_hyper.o
> obj/loader_smooth.o  obj/mpool.o  obj/parse_conf.o  obj/screen.o
> obj/ssl_thr_lock.o  obj/statistics.o  obj/timer_queue.o  obj/url.o
> -L./lib -L/usr/local/ssl/lib -ldl -lpthread -lrt -lcurl -levent -lz
> -lssl -lcrypto
> /usr/bin/ld: cannot find -lz
> collect2: ld returned 1 exit status
> make: *** [curl-loader] Error 1
>
> Any more help?
>
> Thanks,
> Jian


Please, install libz in some standard path.
This is a classical library, which is used for compression.

Take care!


-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
Assistive technology that understands you
......................................................................

Re: make error on Ubuntu8.04

[Jian W. <ji...@gm...>]

oops, another error occurs:

gcc  -g  -o curl-loader  obj/batch.o  obj/cl_alloc.o  obj/client.o
obj/conf.o  obj/environment.o  obj/heap.o  obj/ip_secondary.o
obj/loader.o  obj/loader_fsm.o  obj/loader_hyper.o
obj/loader_smooth.o  obj/mpool.o  obj/parse_conf.o  obj/screen.o
obj/ssl_thr_lock.o  obj/statistics.o  obj/timer_queue.o  obj/url.o
-L./lib -L/usr/local/ssl/lib -ldl -lpthread -lrt -lcurl -levent -lz
-lssl -lcrypto
/usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
make: *** [curl-loader] Error 1

Any more help?

Thanks,
Jian


On Sun, Dec 14, 2008 at 3:15 PM, Jian Wang <ji...@gm...> wrote:
> Thanks a lot, Robert,
>
> That solved the problem.
>
> My best regards,
> Jian
> On Sun, Dec 14, 2008 at 3:11 PM, Robert Iakobashvili
> <cor...@gm...> wrote:
>> Hi Jian Wang,
>>
>> On Sun, Dec 14, 2008 at 11:05 PM, Jian Wang <ji...@gm...> wrote:
>>> Hi,
>>>
>>> I'm trying to install curl-loader on Ubuntu8.04. However, I got error
>>> on make command and don't know how to fix it. Here is the error
>>> message and my OS info:
>>>
>>> root@DELL1:/usr/include$ uname -a
>>> Linux DELL1 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux
>>>
>>> $make
>>> .......
>>> curl-7.19.0/m4/curl-reentrant.m4
>>> curl-7.19.0/Makefile.in
>>> curl-7.19.0/buildconf
>>> /bin/sh: patch: not found
>>> make: *** [lib/libcurl.a] Error 127
>>>
>>> Thanks in advance.
>>>
>>> Jian
>>>
>>
>> Install the command named
>> patch
>>
>> It may be a package named patch, or package with another name,
>> just look at your distro packaging system.
>>
>> --
>> Truly,
>> Robert Iakobashvili, Ph.D.
>> ......................................................................
>> Assistive technology that understands you
>> ......................................................................
>>
>> ------------------------------------------------------------------------------
>> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
>> The future of the web can't happen without you.  Join us at MIX09 to help
>> pave the way to the Next Web now. Learn more and register at
>> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
>> _______________________________________________
>> curl-loader-devel mailing list
>> cur...@li...
>> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>>
>

Re: make error on Ubuntu8.04

[Jian W. <ji...@gm...>]

Thanks a lot, Robert,

That solved the problem.

My best regards,
Jian
On Sun, Dec 14, 2008 at 3:11 PM, Robert Iakobashvili
<cor...@gm...> wrote:
> Hi Jian Wang,
>
> On Sun, Dec 14, 2008 at 11:05 PM, Jian Wang <ji...@gm...> wrote:
>> Hi,
>>
>> I'm trying to install curl-loader on Ubuntu8.04. However, I got error
>> on make command and don't know how to fix it. Here is the error
>> message and my OS info:
>>
>> root@DELL1:/usr/include$ uname -a
>> Linux DELL1 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux
>>
>> $make
>> .......
>> curl-7.19.0/m4/curl-reentrant.m4
>> curl-7.19.0/Makefile.in
>> curl-7.19.0/buildconf
>> /bin/sh: patch: not found
>> make: *** [lib/libcurl.a] Error 127
>>
>> Thanks in advance.
>>
>> Jian
>>
>
> Install the command named
> patch
>
> It may be a package named patch, or package with another name,
> just look at your distro packaging system.
>
> --
> Truly,
> Robert Iakobashvili, Ph.D.
> ......................................................................
> Assistive technology that understands you
> ......................................................................
>
> ------------------------------------------------------------------------------
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
> _______________________________________________
> curl-loader-devel mailing list
> cur...@li...
> https://lists.sourceforge.net/lists/listinfo/curl-loader-devel
>

Re: make error on Ubuntu8.04

[Robert I. <cor...@gm...>]

Hi Jian Wang,

On Sun, Dec 14, 2008 at 11:05 PM, Jian Wang <ji...@gm...> wrote:
> Hi,
>
> I'm trying to install curl-loader on Ubuntu8.04. However, I got error
> on make command and don't know how to fix it. Here is the error
> message and my OS info:
>
> root@DELL1:/usr/include$ uname -a
> Linux DELL1 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux
>
> $make
> .......
> curl-7.19.0/m4/curl-reentrant.m4
> curl-7.19.0/Makefile.in
> curl-7.19.0/buildconf
> /bin/sh: patch: not found
> make: *** [lib/libcurl.a] Error 127
>
> Thanks in advance.
>
> Jian
>

Install the command named
patch

It may be a package named patch, or package with another name,
just look at your distro packaging system.

-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
Assistive technology that understands you
......................................................................

make error on Ubuntu8.04

[Jian W. <ji...@gm...>]

Hi,

I'm trying to install curl-loader on Ubuntu8.04. However, I got error
on make command and don't know how to fix it. Here is the error
message and my OS info:

root@DELL1:/usr/include$ uname -a
Linux DELL1 2.6.24-22-generic #1 SMP Mon Nov 24 18:32:42 UTC 2008 i686 GNU/Linux

$make
.......
curl-7.19.0/m4/curl-reentrant.m4
curl-7.19.0/Makefile.in
curl-7.19.0/buildconf
/bin/sh: patch: not found
make: *** [lib/libcurl.a] Error 127

Thanks in advance.

Jian

Re: Need some clarification for running a test with 1000+ URLs

[Robert I. <cor...@gm...>]

Hi Pranav,

On Thu, Dec 4, 2008 at 5:20 AM, Pranav Desai <pra...@gm...> wrote:

>
> I am running version 0.44.


Recent version is 0.47.

>
>
> I would like to use 1000+ URLs with 300 or so clients.


Please, read the docs, like README, man pages from
curl-loader-0.47 for the URL_TEMPLATE, e.g.

>
>
> I am seeing that URL300 onwards doesn't seem to be used (assuming that
> the 2nd and 3rd columns are number of times the URL is accessed). So
> is this tied to number of users, since I am using 300 clients. If so
> how can I use a large number of URLs.


We cannot say nothing unless you explain, what you are doing
by sending a duly filled PROBLEM-REPORTING-FORM and either inlining
or attaching your configuration file.
Y may wish to amend/decrypt the real URLs, however.

>
>
> I was assuming that over a period of time all URLs would be used, but
> I have ran this same test overnight and it still shows the same thing.
> The test below also has been running for quite some time.
>
> Thanks
> -- Pranav
>


Could it be, that URLS_NUM is 300?


-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Need some clarification for running a test with 1000+ URLs

[Pranav D. <pra...@gm...>]

Hello,

I am running version 0.44.

I would like to use 1000+ URLs with 300 or so clients.

I am seeing that URL300 onwards doesn't seem to be used (assuming that
the 2nd and 3rd columns are number of times the URL is accessed). So
is this tied to number of users, since I am using 300 clients. If so
how can I use a large number of URLs.

I was assuming that over a period of time all URLs would be used, but
I have ran this same test overnight and it still shows the same thing.
The test below also has been running for quite some time.

Thanks
-- Pranav

------------------------------------------------------------------------------
URL289:                 0      2400             0      0                0      0
URL290:                 0      2400             0      0                0      0
URL291:                 0      2400             0      0                0      0
URL292:                 0      2400             0      0                0      0
URL293:                 0      2400             0      0                0      0
URL294:                 0      2400             0      0                0      0
URL295:                 0      2400             0      0                0      0
URL296:                 0      2400             0      0                0      0
URL297:                 0      2400             0      0                0      0
URL298:                 0      2400             0      0                0      0
URL299:                 0      0                0      2400             0      0
URL300:                 0      0                0      0                0      0
URL301:                 0      0                0      0                0      0
...
URL993:                 0      0                0      0
 0      0
URL994:                 0      0                0      0
 0      0
URL995:                 0      0                0      0
 0      0
URL996:                 0      0                0      0
 0      0
URL997:                 0      0                0      0
 0      0
URL998:                 0      0                0      0
 0      0
URL999:                 0      0                0      0
 0      0
--------------------------------------------------------------------------------
Interval stats (latest:10 sec, clients:300, CAPS-curr:0):
H/F   Req:750,1xx:0,2xx:675,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:1623ms,D-2xx:1623ms,Ti:92014B/s,To:17536B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
--------------------------------------------------------------------------------
Summary stats (runs:6995 secs, CAPS-average:0):
H/F   Req:793340,1xx:0,2xx:790865,3xx:0,4xx:2400,5xx:0,Err:0,T-Err:0,D:443ms,D-2xx:443ms,Ti:698921B/s,To:27965B/s
H/F/S Req:0,1xx:0,2xx:0,3xx:0,4xx:0,5xx:0,Err:0,T-Err:0,D:0ms,D-2xx:0ms,Ti:0B/s,To:0B/s
=================================================================================
 Manual: clients:max[300],curr[300]. Inc num: [+|*].
=================================================================================

curl-loader version 0.47 released

[Robert I. <cor...@gm...>]

Gentlemen,

version 0.47, unstable, December 2, 2008, svn 551

* Fixed a POST-binary bug, where an extra check was preventing direct
posting.
   Thanks to Alex <ale...@gm...> and Gary Fitts <ga...@in...
>

* Fixed a bug on 64-bit ubuntoo, originating most probably from the
   wrong compiler optimization and running on an SMP machine, where
   configuration files has failed to be loaded.
   Thanks to Sergei Shnaidman <ser...@qi...> for
   reporting, debugging and testing the fix.

* Integrated a great contribution from Gary Fitts <ga...@in...>
  - template URL, where throusands URLs specifying by a single template is
an option;
  - fetching of response tokens from the bodies of responses and usage them
in template URL;
  - usage of statically defined in file tokens for template URL;
  - fixing of several bugs in that area
  - introduced new tags:
   URL_TEMPLATE, RESPONSE_TOKEN, URL_TOKEN, URL_TOKEN_FILE

* UPLOAD_FILE bug fixed, where the same file-pointer for uploading was
shared
   among mulitple clients. Thanks to  Gary Fitts <ga...@in...>.

*  Added for "Template URL" README and man pages, usage examples.

-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Re: building urls from server responses

[Robert I. <cor...@gm...>]

Hi Gary,

On Mon, Dec 1, 2008 at 10:11 PM, Gary Fitts <ga...@in...> wrote:

> Hello Robert,
>
> Here is the latest (hopefully last) patch file, except for bug fixes.
> And everything else -- the README and the sample files -- looks OK.
>
> I've added a bit to the README to explain two new tags that I'm using
> (ERROR_TOKEN and PRINT_TIMING_STATS) in case others want to use them too.
>
> Gary
>
>

Thank you very much.

I will be doing release very soon, therefore, the new tags
will be added after release. We are also postponing code improvements, etc,
since this release will incorporate a number of good bug-fixes, that are
better
to release ASAP.


-- 
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Re: building urls from server responses

[Gary F. <ga...@in...>]

Hello Robert,

Here is the latest (hopefully last) patch file, except for bug fixes.
And everything else -- the README and the sample files -- looks OK.

I've added a bit to the README to explain two new tags that I'm using
(ERROR_TOKEN and PRINT_TIMING_STATS) in case others want to use them  
too.

Gary

Re: building urls from server responses

[Gary F. <ga...@in...>]

Hello Robert,

Here is the patch file for my latest source:
diff -Naru    my-source-tree    version-547-source-tree    > Nov28.patch

I've tried to conform to your coding conventions and style. If I've  
missed
anywhere I apologize.

Please take a look at the code in load_fsm.c that you referred to,
where I call curl_easy_cleanup. You pointed out that I shouldn't call  
this
if the client might be scheduled again. But I think that at the point  
where
this is called, the functions in load_state_func_table have returned
CSTATE_ERROR, and I think this means that the client will never be
scheduled again even if error_recovery_client is true. I may be  
wrong ...

I'll check the document files tomorrow or Sunday.

Thanks,
Gary

Re: building urls from server responses

[Gary F. <ga...@in...>]

We've had a major crunch at work for the last week,
but I'll try to get this done soon,
hopefully this weekend.


On Nov 27, 2008, at 2:41 PM, Robert Iakobashvili wrote:

> Hi Gary,
>
> Had you a chance to look at the below changes?
> We would like to add the new tags also to the man and web-pages and to
> make a new release.
>
> Thanks,
> Robert
>
>
>
> ==========================================================
> - explaining to potential users how to use the new tags.
> Please, look at README changes and review them:
> http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/doc/README?r1=534&r2=546
>
> Added example files:
>
> http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-fr-file.conf?revision=546&view=markup
> http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-resp-dynamic.conf?view=markup
> http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-file.txt?revision=546&view=markup
>
> Afterwards, the man-pages and the web-site pages to be updates
>
> - code reviewing and potential bugs tracking.
>
> In meanwhile, I have detected only a single potential issue:
> in load_fsm.c
>
> Calling of newly added close_connections() with a curl_easy_cleanup ()
> for a client CURL handle is a bugfix for clients in  
> CSTATE_FINISHED_OK state.
> However, it make create a bug for clients in state CSTATE_ERROR.
>
> Such clients with an error in one cycle may be re-scheduled once more
> for another cycle, thus, they will be attempted to be added to the  
> multi-handle,
> rejected as null, and, therefore, such clients will be never  
> scheduled again for
> more loading cycles (there is a command-line option to configure for
> such behavior,
> which is not the default).
>
> Calling curl_easy_cleanup () is a good hint to libcurl multi-handle to
> decrease the pool of connections to the server. Added for clients in
> error-state one more curl_easy_init() as you can see in svn.
>
> - bringing the newly added code to the coding conventions
>  and coding style of the project (I started that, note, that a couple
>  functions have been re-named to make their names more clear);
> - removing various extern declarations (done for functions);
> - functions should have function caption with description in  
> appropriate
>  format;
>
>>
>> Since I submitted these changes I've found the need for a couple more
>> tags.
>> I'll send those along in a day or so when I've been able to test  
>> them.
>
> Great!
> If you can work against the code in svn and submit your code as  
> patches
> diff -Naru newcode-tree old-core-tree > somepatch.patch, it may make
> integration easier.
>
> Try, where it is possible to follow the project's coding style and  
> conventions.
> Thanks!
>
> --
> Truly,
> Robert Iakobashvili, Ph.D.
> ......................................................................
> Assistive technology that understands you
> ......................................................................

Re: building urls from server responses

[Robert I. <cor...@gm...>]

Hi Gary,

Had you a chance to look at the below changes?
We would like to add the new tags also to the man and web-pages and to
make a new release.

Thanks,
Robert



==========================================================
- explaining to potential users how to use the new tags.
Please, look at README changes and review them:
http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/doc/README?r1=534&r2=546

Added example files:

http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-fr-file.conf?revision=546&view=markup
http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-resp-dynamic.conf?view=markup
http://curl-loader.svn.sourceforge.net/viewvc/curl-loader/trunk/curl-loader/conf-examples/url-template-file.txt?revision=546&view=markup

Afterwards, the man-pages and the web-site pages to be updates

- code reviewing and potential bugs tracking.

In meanwhile, I have detected only a single potential issue:
in load_fsm.c

Calling of newly added close_connections() with a curl_easy_cleanup ()
for a client CURL handle is a bugfix for clients in CSTATE_FINISHED_OK state.
However, it make create a bug for clients in state CSTATE_ERROR.

Such clients with an error in one cycle may be re-scheduled once more
for another cycle, thus, they will be attempted to be added to the multi-handle,
rejected as null, and, therefore, such clients will be never scheduled again for
more loading cycles (there is a command-line option to configure for
such behavior,
which is not the default).

Calling curl_easy_cleanup () is a good hint to libcurl multi-handle to
decrease the pool of connections to the server. Added for clients in
error-state one more curl_easy_init() as you can see in svn.

- bringing the newly added code to the coding conventions
  and coding style of the project (I started that, note, that a couple
  functions have been re-named to make their names more clear);
- removing various extern declarations (done for functions);
- functions should have function caption with description in appropriate
  format;

>
> Since I submitted these changes I've found the need for a couple more
> tags.
> I'll send those along in a day or so when I've been able to test them.

Great!
If you can work against the code in svn and submit your code as patches
diff -Naru newcode-tree old-core-tree > somepatch.patch, it may make
integration easier.

Try, where it is possible to follow the project's coding style and conventions.
Thanks!

--
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
Assistive technology that understands you
......................................................................

Re: curl-loader needs a big WARNING message

[Robert I. <cor...@gm...>]

Hi,

On Wed, Nov 26, 2008 at 8:24 PM, Edward Lott <ed...@je...> wrote:
>
> I recently tried to use curl-loader to test a production server.  I didn't realize (until looking at the source) that curl-loader made kernel calls to modify the network configuration of the server it runs on (i.e. adding static IP addresses to server and changing the routing table).

The extract from the README file in curl-loader-<version> directory:

================================================================
 IP-addresses from IP_ADDR_MIN up to IP_ADDR_MAX are the interval to
be used for
 loading clients. The interval of the addresses should be large enough
to supply
 a dedicated IP for each client. Another mode that can be used is "single IP
 address" for all clients. When IP_ADDR_MIN and IP_ADDR_MAX have the same valid
 IP-address, it will be used for all clients. If the IP-addresses specified by
 the tags do not exist, curl-loader adds them as the secondary IPs to
the loading
network interface.
Remember, that you should ensure smooth routing from the addresses used to
server and from server back to the client addresses. Another route to ensure,
 when is appropriate, is to/from DNS server for resolving. Firewalling
(netfilter/iptables) settings are also among those to check.

=============================================================

Still, you might wish to improve the documentation, and send your patches for
README file, FAQs. man-pages, web-site, etc as the unified diff to the list.

Note, that we do not have any doc writers, therefore, such assistance would
be very much appreciated.

--
Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

Re: curl-loader needs a big WARNING message

[Robert I. <cor...@gm...>]

Hi Edward,

On Wed, Nov 26, 2008 at 8:24 PM, Edward Lott <ed...@je...> wrote:

>  I recently tried to use curl-loader to test a production server.  I
> didn't realize (until looking at the source) that curl-loader made kernel
> calls to modify the network configuration of the server it runs on (i.e.
> adding static IP addresses to server and changing the routing table).
>

It is a testing tool, isn't it? The place for testing tools are
at testing machines in the lab.

>
>
> I assumed that the INTERFACE, IP_ADDR_MIN and IP_ADDR_MAX worked with
> definitions that already existed, vs actually making them.  Creating these
> definitions is potentially VERY DANGEROUS, especially if the program is run
> on a production system.  Most folks will want to run tools like cur-loader
> on systems that are on the same segment as the server they're testing.  For
> me, this meant running it on another production server since we work with a
> hosting provider.  The resulting curl-loader changes to the IP configuration
> of the machine it ran on caused 1 week of general mayhem.
>
>
>
> Suggestions:
>
>
>
> 1)      Add a BIG warning about IP network configuration by curl-loader
>
I am in disagreement with you.

> 2)      Create a –changeNetworkIfNeeded option.  Without this option,
> curl-loader will attempt to satisfy INTERFACE, IP_ADDR_MIN and IP_ADDR_MAX
> without reconfiguring the network of the host it runs on.  If the host can't
> satisfy what's requested, curl-loader errors out, telling the user to use
> the changeNetworkIfNeeded option to make the test work.
>

There are options for doing that, please read the FAQs.

Truly,
Robert Iakobashvili, Ph.D.
......................................................................
www.ghotit.com
Assistive technology that understands you
......................................................................

curl-loader needs a big WARNING message

[Edward L. <ed...@je...>]

I recently tried to use curl-loader to test a production server.  I
didn't realize (until looking at the source) that curl-loader made
kernel calls to modify the network configuration of the server it runs
on (i.e. adding static IP addresses to server and changing the routing
table).

 

I assumed that the INTERFACE, IP_ADDR_MIN and IP_ADDR_MAX worked with
definitions that already existed, vs actually making them.  Creating
these definitions is potentially VERY DANGEROUS, especially if the
program is run on a production system.  Most folks will want to run
tools like cur-loader on systems that are on the same segment as the
server they're testing.  For me, this meant running it on another
production server since we work with a hosting provider.  The resulting
curl-loader changes to the IP configuration of the machine it ran on
caused 1 week of general mayhem.

 

Suggestions:

 

1)      Add a BIG warning about IP network configuration by curl-loader

2)      Create a -changeNetworkIfNeeded option.  Without this option,
curl-loader will attempt to satisfy INTERFACE, IP_ADDR_MIN and
IP_ADDR_MAX without reconfiguring the network of the host it runs on.
If the host can't satisfy what's requested, curl-loader errors out,
telling the user to use the changeNetworkIfNeeded option to make the
test work.