Re: url probability bug fix and random number use update

[Val S. <va...@nv...>]

Hi, Robert,


>
>From: Nadia and Val Shkolnikov <nv...@ya...>
>To: curl-loader-devel <cur...@li...>
>Sent: Fri, June 11, 2010 8:43:13 PM
>Subject: Re: url probability bug fix and random number use update
>
>
>Hi, Robert,
>
>
>>
>>From: Robert Iakobashvili <cor...@gm...>
>>To: curl-loader-devel <cur...@li...>
>>Sent: Wed, June 9, 2010 11:22:27 PM
>>Subject: Re: url probability bug fix and random number use update
>>
>>>>Hi, Val,
>>
>>
>>On Wed, Mar 24, 2010 at 3:36 AM, Val Shkolnikov <va...@nv...> wrote:
>>
>>Hi Robert,
>>>
>>>
>>>>
>>>>From: Val Shkolnikov <va...@nv...>
>>>>
>>>>To: curl-loader-devel <cur...@li...>
>>>>Sent: Sun, March 21, 2010 9:13:04 PM
>>>>
>>>>Subject: Re: url probability bug fix and random number use update
>>>>
>>>>
>>>>
>>>>Hi Robert,
>>>>
>>>>From: Robert Iakobashvili <cor...@gm...>
>>>>>To: curl-loader-devel <cur...@li...>
>>>>>Sent: Sun, March 21, 2010 12:02:05 AM
>>>>>Subject: Re: url probability bug fix and random number use update
>>>>>
>>>>>>>>>>Hi Val,
>>>>>
>>>>>
>>>>>On Sun, Mar 14, 2010 at 8:54 AM, Robert Iakobashvili <cor...@gm...> wrote:
>>>>>
>>>>>>>>>>>Hi Val,
>>>>>>
>>>>>>
>>>>>>On Mon, Mar 8, 2010 at 7:16 AM, Val Shkolnikov <va...@nv...> wrote:
>>>>>>
>>>>>>>>>>>>>Hi Robert,
>>>>>>>>>>>>>>   There is a major bug in the curl-loader regarding use of the FETCH_PROBABILITY tag.  In loader_fsm.c, svn rev 574, line 1359 is
>>>>>>>
>>>>>>>>>>>>>>cctx->url_curr_index =  (size_t) url_next;
>>>>>>>
>>>>>>>>>>>>>>The url index is updated but not the url pointer.  The result of this bug is that all urls are selected regardless of the probability somewhat uniformly.
>>>>>>>
>>>>>>>>>>>>>>While debugging this I also reworked the use of probability in the curl-loader to produce better pseudo-random numbers and also added a tag RANDOM_SEED that lets you control the seed.  I am attaching the patch against svn rev 574.  To use it
>>>>>>>
>>>>>>>>>>>>>>cd curl-loader
>>>>>>>>>>>>>>patch -p3 <curl-loader.patch.574+vs6
>>>>>>>
>>>>>>
>>>>>>Applied the patch, thanks!
>>>>>>
>>>>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>>>>>>I also noticed that the tag FETCH_PROBAILITY_ONCE is not used, but will let you decide whether it's worth to keep.
>>>>>>>
>>>>>>
>>>>>>Please, look at the latest version in svn. Thanks!
>>>>>>
>>>>>>The array of url_fetch_decision was allocated per client only, when FETCH_PROBAILITY_ONCE was defined. 
>>>>>>>>>>>>
>>>>>>
>>>>>>The array is used to cache the fetching decision and to decrease
>>>>>>the calls to random() for high-load testing.
>>>>>>
>>>>>>In most of the cases, however, it will be of no use, and it was corrected
>>>>>>not to use url_fetch_decision array, when it is not supposed to be allocated.
>>>>>>>>>>>>
>>>>>>
>>>>>> 
>>>>>Have you a chance to look at the version in svn and try the changes made?
>>>>>Thanks!
>>>>>
>>>>>-- 
>>>>>Truly,
>>>>>Robert Iakobashvili, Ph.D.
>>>>>......................................................................
>>>>>www.ghotit.com
>>>>>Assistive technology that understands you
>>>>>......................................................................
>>>>>
>>>>Sorry, a bit busy.  I might have a chance to integrate the new code into mine next week.
>>>>/Val
>>>>
>>>I reviewed your code and it looks good.  One minor change I suggest which is not terribly important if you think your code is clearer.  The patch is
>>> attached.
>>>/Val
>>>
>>Could you, please, try the code from subversion and say, if it works 
>>for your settings?
>>Thank you in advance.
>>
>>-- 
>>Truly,
>>Robert Iakobashvili, Ph.D.
>>......................................................................
>>www.ghotit.com
>>Assistive technology that understands you
>>......................................................................
>>I'll get to it this weekend, promise :-).
>/Val
>
OK, loaded rev 577, compiled from scratch and tested with my usual load, everything looks good.
/Val