Re: [Grinder-use] Dynamically replacing a parameter in the URL

[<edb...@gm...>]

Sean,

Yes, you defined those headers before any requests weer made -- indeed,
even before the test class was defined.  There are a few ways you could go
about this, but to make it maintainable long-term, it might be worth taking
some time to develop a better understanding of Python.

I recommend the free HTML eBook at http://learnpythonthehardway.org/book/

Enjoy,
-Ed


-Ed Brannin
edb...@gm...
cell: 585-261-0279


On Tue, Sep 24, 2013 at 3:38 PM, Sean Tiley <sea...@gm...> wrote:

> Thanks Ed, that does help, I now have the value getting set properly on
> request#4.
> I achieved this by defining a variable mySRN
>
> class TestRunner:
>   """A TestRunner instance is created for each worker thread."""
>   mySRN=""
> .....
>
> The value of mySRN gets set in request 3
>
> self.mySRN=extractSRN(result,"Result3")
>
> Then in request 4 I do as you suggest...
>  def page4(self):
>     " .......
>       self.token_SWEPL +
>       '&SRN=' +
>       self.mySRN +
>       .....
>
>
>
> So far excellent.  Now What I am not sure about is the SRN value is set on
> the headers
>
> headers2= \
>   [ NVPair('Accept', '*/*'),
>     NVPair('Referer', '
> https://abc.def.com/finsecustomer_enu/start.swe?SWECmd=Login&SWEPL=1&*
> SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb*&SWETS=1380027072171'), ]
>
> headers3= \
>   [ NVPair('Accept', '*/*'),
>     NVPair('Referer', 'https://abc.def.com/finsecustomer_enu/start.swe?*
> SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb*&SWEContainer=&SWETS=&SWETA='),
> ]
>
> headers4= \
>   [ NVPair('Accept', '*/*'),
>     NVPair('Referer', '
> https://abc.def.com/finsecustomer_enu/start.swe?SWENeedContext=false&SWECmd=Logoff&SWEC=3&SWEBID=-1&
> *SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb*&SWETS='), ]
>
> So I am not sure how to replace the hardcoded value in those headers.  Are
> they initialized prior to the requests even being made?
>
> Alternatively, I think I could remove the SRN from the defined headers and
> add them in as necessary in the def page(N) construction, just not clear on
> how to do that.
>
> Thanks again for your help.
>
> Sean
>
>
>
> On Tue, Sep 24, 2013 at 2:17 PM, edb...@gm... <edb...@gm...>wrote:
>
>> Sean,
>>
>> To set a class-instance variable in Python, prefex the name with "self."
>>
>> in page3(self):
>>     self.srn = extract_SRN(result, "Result3")
>>
>> in page4(self):
>>    self.token_SRN = self.srn
>>
>> ...scratch that.  You can just set self.token_SRN from page3(self), as
>> shown above, then remove the "self.token_SRN = "..."" line from page4(self).
>>
>> A few other thoughts:
>>
>>    1. Unless you're using all these other "token_FOO" values from
>>    methods other than page4(), you can leave them as local variables by
>>    dropping "self."
>>    2. Alternately: It looks like you might want to replace all these
>>    "token_FOO" variables with a "self.token = dict()" in
>>    TestRunner.__init__(self).  Then you could use them all like
>>    "self.token['SRN'] = extract_srn(...)" and build the parameters in that
>>    page4 URL with a loop like:
>>    url = "http://...."
>>    first = True
>>    for token, value in self.token.items():
>>        if first:
>>            url += "?"
>>            first = False
>>        else:
>>            url += "&"
>>        url += "%s=%s" % (token, value)
>>    # ...and then extract the above into build_url(base, tokens) and
>>    consider URLencoding token names for future sanity. :)
>>
>>
>> Is that what you were looking for?
>>
>> -Ed
>>
>> -Ed Brannin
>> edb...@gm...
>> cell: 585-261-0279
>>
>>
>> On Tue, Sep 24, 2013 at 1:33 PM, Sean Tiley <sea...@gm...> wrote:
>>
>>> Thank you Ravi,
>>> Originally I mentioned I was using 3.1.1, should have said 3.11.
>>> Anyways, really good info on the site, definitely helpful.
>>> I do not think I was very clear on my initial posting. I have used some
>>> of your info to get at what I need now I have another question(s).
>>>
>>> My snipped file has been included here for reference as it is clear I am
>>> not really sure how it all works (see the comments with #************)
>>>
>>> I have created a helper method that will extract the SRN parameter on
>>> the apropriate response (the third response).
>>>
>>> In request 4, the recorded value of
>>> ''HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb'' is what I want to replace
>>> with the value that was extracted with my helper method.
>>>
>>> First Question, how can I assign the value I extracted in my method to a
>>> variable that is accessible from the next request?  Is it a member
>>> variable?  If so, where does the definition of that member variable belong
>>> and how do I assign my value to it?
>>>
>>> Once I can do this, the next question is how can I replace the recorded
>>> value for the SRN parameter in the headers2, headers3, headers4?
>>>
>>>
>>>
>>> # The Grinder 3.11
>>> # HTTP script recorded by TCPProxy at 24-Sep-2013 8:50:22 AM
>>> from net.grinder.script import Test
>>> from net.grinder.script.Grinder import grinder
>>> from net.grinder.plugin.http import HTTPPluginControl, HTTPRequest
>>> from HTTPClient import NVPair
>>> from ValidationUtils import *
>>> import re
>>> connectionDefaults = HTTPPluginControl.getConnectionDefaults()
>>> httpUtilities = HTTPPluginControl.getHTTPUtilities()
>>> # To use a proxy server, uncomment the next line and set the host and
>>> port.
>>> # connectionDefaults.setProxyServer("localhost", 8001)
>>> def createRequest(test, url, headers=None):
>>>     """Create an instrumented HTTPRequest."""
>>>     request = HTTPRequest(url=url)
>>>     if headers: request.headers=headers
>>>     test.record(request, HTTPRequest.getHttpMethodFilter())
>>>     return request
>>> # These definitions at the top level of the file are evaluated once,
>>> # when the worker process is started.
>>> connectionDefaults.defaultHeaders = \
>>>   [ NVPair('Accept-Language', 'en-CA'),
>>>     NVPair('User-Agent', 'Mozilla/4.0 (compatible; MSIE 8.0; Windows NT
>>> 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729;
>>> .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)'),
>>> ]
>>> headers0= \
>>>   [ NVPair('Accept', 'application/x-ms-application, image/jpeg,
>>> application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap,
>>> application/vnd.ms-excel, application/vnd.ms-powerpoint,
>>> application/msword, */*'), ]
>>> headers1= \
>>>   [ NVPair('Accept', '*/*'),
>>>     NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SWECmd=Start&SWEHo=abc.def.com'),
>>> ]
>>> headers2= \
>>>   [ NVPair('Accept', '*/*'),
>>>     NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SWECmd=Login&SWEPL=1&SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb&SWETS=1380027072171'),
>>> ]
>>> headers3= \
>>>   [ NVPair('Accept', '*/*'),
>>>     NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb&SWEContainer=&SWETS=&SWETA='),
>>> ]
>>> headers4= \
>>>   [ NVPair('Accept', '*/*'),
>>>     NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SWENeedContext=false&SWECmd=Logoff&SWEC=3&SWEBID=-1&SRN=HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb&SWETS='),
>>> ]
>>> headers5= \
>>>   [ NVPair('Accept', '*/*'),
>>>     NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SWECmd=GotoView&SWEView=FINS+EPW+Homepage+View+%28Anonymous%29&SRN=&SWEHo=abc.def.com&SWETS=1380027089'),
>>> ]
>>> url0 = 'https://devscrm12.abc.def.com:443'
>>> request101 = createRequest(Test(101, 'GET /'), url0, headers0)
>>> request201 = createRequest(Test(201, 'GET start.swe'), url0, headers0)
>>> request202 = createRequest(Test(202, 'GET swecommon.js'), url0, headers1)
>>> request203 = createRequest(Test(203, 'GET dCCmain.css'), url0, headers1)
>>> request204 = createRequest(Test(204, 'GET start.swe'), url0, headers1)
>>> request205 = createRequest(Test(205, 'GET swecommon_top.js'), url0,
>>> headers1)
>>> request206 = createRequest(Test(206, 'GET start.swe'), url0, headers1)
>>> request207 = createRequest(Test(207, 'GET swecmn_li.js'), url0, headers1)
>>> request208 = createRequest(Test(208, 'GET start.swe'), url0, headers1)
>>> request209 = createRequest(Test(209, 'GET swemessages_enu.js'), url0,
>>> headers1)
>>> request210 = createRequest(Test(210, 'GET spacer.gif'), url0, headers1)
>>> request211 = createRequest(Test(211, 'GET splash_top_banner.jpg'), url0,
>>> headers1)
>>> request212 = createRequest(Test(212, 'GET icon_req.gif'), url0, headers1)
>>> request213 = createRequest(Test(213, 'GET Logos-Home.jpg'), url0,
>>> headers1)
>>> request214 = createRequest(Test(214, 'GET button_register_now.png'),
>>> url0, headers1)
>>> request215 = createRequest(Test(215, 'GET button_signin.png'), url0,
>>> headers1)
>>> request301 = createRequest(Test(301, 'POST start.swe'), url0)
>>> request401 = createRequest(Test(401, 'GET start.swe'), url0, headers0)
>>> request402 = createRequest(Test(402, 'GET dCCmain.css'), url0, headers2)
>>> request403 = createRequest(Test(403, 'GET swecommon.js'), url0, headers2)
>>> request404 = createRequest(Test(404, 'GET swecommon_top.js'), url0,
>>> headers2)
>>> request405 = createRequest(Test(405, 'GET swemessages_enu.js'), url0,
>>> headers2)
>>> ........
>>>
>>> #the regex to extract the SRN value
>>> pattern = re.compile("SRN=[a-zA-Z0-9]{1,60}[^&]")
>>> class TestRunner:
>>>   """A TestRunner instance is created for each worker thread."""
>>>
>>>   # A method for each recorded page.
>>>   def page1(self):
>>>     """GET / (request 101)."""
>>>     result = request101.GET('/finsecustomer_enu/')
>>>     return result
>>>   def page2(self):
>>>     .....
>>>     return result
>>>   def page3(self):
>>>     """POST start.swe (request 301)."""
>>>     result = request301.POST('/finsecustomer_enu/start.swe',
>>>       ( NVPair('s_2_1_11_0', 'XXXX'),
>>>         NVPair('s_2_1_12_0', 'XXSS'),
>>>         NVPair('SWEFo', 'SWEForm2_0'),
>>>         NVPair('SWEField', 's_2_1_14_0'),
>>>         NVPair('SWENeedContext', 'true'),
>>>         NVPair('SWENoHttpRedir', 'true'),
>>>         NVPair('W', 't'),
>>>         NVPair('SWECmd', 'InvokeMethod'),
>>>         NVPair('SWEMethod', 'AppletLogin'),
>>>         NVPair('SWERowIds', ''),
>>>         NVPair('SWESP', 'false'),
>>>         NVPair('SWEVI', ''),
>>>         NVPair('SWESPNR', ''),
>>>         NVPair('SWEPOC', ''),
>>>         NVPair('SWESPNH', ''),
>>>         NVPair('SWEH', ''),
>>>         NVPair('SWETargetView', ''),
>>>         NVPair('SWEDIC', 'false'),
>>>         NVPair('SWEReqRowId', '0'),
>>>         NVPair('SWEView', 'FINS EPW Homepage View (Anonymous)'),
>>>         NVPair('SWEC', '1'),
>>>         NVPair('SWERowId', 'VRId-0'),
>>>         NVPair('SWETVI', ''),
>>>         NVPair('SWEW', ''),
>>>         NVPair('SWEBID', '-1'),
>>>         NVPair('SWEM', ''),
>>>         NVPair('SRN', ''),
>>>         NVPair('SWESPa', ''),
>>>         NVPair('SWETS', '1380027072171'),
>>>         NVPair('SWEContainer', ''),
>>>         NVPair('SWEWN', ''),
>>>         NVPair('SWEApplet', 'Login Applet'),
>>>         NVPair('SWETA', ''), ),
>>>       ( NVPair('Accept', 'application/x-ms-application, image/jpeg,
>>> application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap,
>>> application/vnd.ms-excel, application/vnd.ms-powerpoint,
>>> application/msword, */*'),
>>>         NVPair('Referer', '
>>> https://abc.def.com/finsecustomer_enu/start.swe?SWECmd=Start&SWEHo=abc.def.com'
>>> ),
>>>         NVPair('Content-Type', 'application/x-www-form-urlencoded'),
>>>         NVPair('Cache-Control', 'no-cache'), ))
>>>
>>>  #*******************************************
>>>  # THIS METHOD SUCCESSFULLY EXTRACTS THE SRN PARAMETER
>>>  # What I want to do is assign the value extracted to a variable
>>>  # so the recorded value can be replaced each time the script runs
>>>  #*******************************************
>>>     extractSRN(result,"Result3")
>>>     return result
>>>   def page4(self):
>>>     """GET start.swe (requests 401-419)."""
>>>     self.token_SWECmd = \
>>>       'Login'
>>>     self.token_SWEPL = \
>>>       '1'
>>>  #*******************************************
>>>  # WANT TO REPLACE WITH THE EXTRACTED VALUE
>>>  #*******************************************
>>>     self.token_SRN = \
>>>       'HE1YHZvKcjjBo8TMBY5cdfVweOyMRbfYKfE33mJL1bYb'
>>>     self.token_SWETS = \
>>>       '1380027072171'
>>>     result = request401.GET('/finsecustomer_enu/start.swe' +
>>>       '?SWECmd=' +
>>>       self.token_SWECmd +
>>>       '&SWEPL=' +
>>>       self.token_SWEPL +
>>>       '&SRN=' +
>>>       self.token_SRN +
>>>       '&SWETS=' +
>>>       self.token_SWETS)
>>>  .....
>>>     return result
>>>   .....
>>>   def __call__(self):
>>>     """Called for every run performed by the worker thread."""
>>>     self.page1()      # GET / (request 101)
>>>     grinder.sleep(31)
>>>     self.page2()      # GET start.swe (requests 201-215)
>>>     grinder.sleep(5712)
>>>     self.page3()      # POST start.swe (request 301)
>>>     grinder.sleep(31)
>>>     self.page4()      # GET start.swe (requests 401-419)
>>>     ....
>>>
>>> # Instrument page methods.
>>> Test(100, 'Page 1').record(TestRunner.page1)
>>> Test(200, 'Page 2').record(TestRunner.page2)
>>> Test(300, 'Page 3').record(TestRunner.page3)
>>> Test(400, 'Page 4').record(TestRunner.page4)
>>> .....
>>> # extract SRN
>>> def extractSRN(HttpResponse,requestNumber):
>>>   target = HttpResponse.text
>>>   #infolog ("HttpResponse.text for response %s in %s" %
>>> (requestNumber,target))
>>>   result2 = pattern.search(target, 0)
>>>   value = result2.group(0)
>>>   val = value[4:]
>>>   #infolog ("SRN value: %s" % val)
>>>   mySRN = val
>>>   return val
>>>
>>>
>>>
>>>
>>>  Any thoughts greatly appreciated.
>>> Many thanks.
>>> Sean
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Sep 20, 2013 at 12:55 PM, Ravi Joshi <rav...@ya...>wrote:
>>>
>>>>  Hi Sean,****
>>>>
>>>> ****
>>>> I would strongly recommend you to upgrade the Grinder. Also I think
>>>> what you are taking is "Parameterizing the script", which is pretty simple.
>>>> Please check the "Parameterizing the test" section in
>>>> http://vivin.net/2009/07/27/performance-testing-using-the-grinder/****
>>>> ** **
>>>> In case it doesn't clear to you, please provide the more details about
>>>> each services.****
>>>> ** **
>>>> -****
>>>> Regards****
>>>>  Ravi Joshi****
>>>>
>>>>   ------------------------------
>>>> *From:* Sean Tiley <sea...@gm...>
>>>> *To:* grinder-use <gri...@li...>
>>>> *Sent:* Friday, 20 September 2013 7:19 PM
>>>> *Subject:* [Grinder-use] Dynamically replacing a parameter in the URL
>>>>
>>>>  Hello,
>>>> I am using Grinder 3.1.1 with java 1.7
>>>>
>>>> I have successfully used the TCPProxy to record many scripts in the
>>>> past and all has been working just fine.  Recently the application, (third
>>>> party), that I have been testing with the Grinder has added a parameter on
>>>> the URL that it uses in conjunction with cookeies for authentication.
>>>>
>>>> This new parameter (SRN) comes in on the 4 response from the site.  I
>>>> need a way to capture it when it comes in and dynamically replace it on all
>>>> subsequent requests in my scripts.  I have done some searching in the
>>>> mailing list archive with limited success and was wondering if there is any
>>>> documentation or examples on how to do this
>>>>
>>>> I have seen this in the lists from 2005
>>>> https://sourceforge.net/p/grinder/mailman/message/9154872/  but I do
>>>> not know how to use the "-newhttpplugin" flag
>>>>
>>>> I've also seen a number of references to a function in a file called
>>>> WebUtils that could potentially help but cannot find such a file anywhere
>>>>
>>>> Any thoughts / guidance would be greatly appreciated
>>>>
>>>> --
>>>> Sean Tiley
>>>> sea...@gm...
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
>>>> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
>>>> SharePoint
>>>> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack
>>>> includes
>>>> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> grinder-use mailing list
>>>> gri...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/grinder-use
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Sean Tiley
>>> sea...@gm...
>>>
>>> ------------------------------------------------------------------------------
>>> October Webinars: Code for Performance
>>> Free Intel webinars can help you accelerate application performance.
>>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>>> from
>>> the latest Intel processors and coprocessors. See abstracts and register
>>> >
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>>>
>>> _______________________________________________
>>> grinder-use mailing list
>>> gri...@li...
>>> https://lists.sourceforge.net/lists/listinfo/grinder-use
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> grinder-use mailing list
>> gri...@li...
>> https://lists.sourceforge.net/lists/listinfo/grinder-use
>>
>>
>
>
> --
> Sean Tiley
> sea...@gm...
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
> from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
> _______________________________________________
> grinder-use mailing list
> gri...@li...
> https://lists.sourceforge.net/lists/listinfo/grinder-use
>
>