opensta-devel — OpenSTA developer discussions

Re: [OpenSTA-devel] Re: Re: Codemax

[Olaf K. <ok...@ab...>]

Federico Toledo wrote:
> I know that LoadRunner has a facility to know what the server answered.
> This tool save the pages that didn't pass the test case.
> So we can view the page returned by the server and then we can debug th=
e error.
>=20
> Resuming: i want to save the pages returned with error by the server
> to know why fails.

I'd like to request a slightly different feature (if I understand you
correctly): Saving a page should be explicitly triggered by the script
whenever the script would like to save a response - not on HTTP errors
in general: I believe that load tests can easily and correctly test that
state 404 is returned by the application. On the other hand if the page
is returned with http-state 200 but contains unexpected content (e.g. an
application error message) the script should be able to save this
unexpected content.

> I have to take a time to do all this 'cause i have to get Visual Studio=
.
> I have a question: Could I use dotnet?

Not having developed any program component for OpenSTA my opinion is
probably inferior to others, but I'd prefer not adding the huge
dependence on dot net (or worse a particular dot net version) to
implement "just an additional feature". This'd be different once lots of
features would require dot net (or be easier to add using dot net) or at
least if it's planned to move into that direction rather soon. The
current stable release definitely should not add the dependency on dot ne=
t.

My 2 cents (=80uro-cent, of course :-)
Olaf

[OpenSTA-devel] Re: Re: Codemax

[Federico T. <ft...@fi...>]

Ok. thanks for the fast answer.

When I use OpenSTA and run a TaskGroup with validations in the scripts
(using tests case sentence)
if a validation fails I didn't know why fails.

I know that LoadRunner has a facility to know what the server answered.
This tool save the pages that didn't pass the test case.
So we can view the page returned by the server and then we can debug the er=
ror.

Resuming: i want to save the pages returned with error by the server
to know why fails.
this is the first idea.

Then I want to investigate how to work with SOA and WebService, but
this will be in the future.

I have implement a couple of tools in Java to do some activities that
can be done automaticaly.
For example, a tool to refresh the script's parameters files
(because after runing a test there are data that is no more valid, so
we have to repeat the sql and export the result to the script's
parameters files to can run again)
I would like to integrate this funcionality to OpenSTA.

Tell me what do you think.

I have to take a time to do all this 'cause i have to get Visual Studio.
I have a question: Could I use dotnet?

ok, thks
bye
FedeFede

[OpenSTA-devel] Re: Codemax

[Daniel S. <da...@Op...>]

Federico Toledo wrote:
> I'm Fede from Uruguay.  I will try to make some improvement to
> OpenSTA.  And for that I require Codemax (according to which I read
> in building prerequisits page).  Could anyone help me in getting
> Codemax?

I've made sure that Fede can get hold of CodeMax - all I ask in
return for this (and future) help, is that you make sure to "give
back" to the OpenSTA project and community ;-)

> I have use OpenSTA in a Load Testing project, and after that I have
> some ideas to do...

Please share your ideas with us before you go ahead implementing
anything.  This is important for a number of reasons:
 - Someone else may have already done what you intend, or something
   similar, that isn't yet in the publically accessible source
   archives or CVS.
 - Someone may have the same desires as you, and be willing to help
   you achieve your goals quicker.
 - It would be great to get any improvements or bug fixes you make
   back into the published source as quick as possible - and that 
   can only really happen if you work within the goals and standards
   of this project from the start, communicating with the other
   project developers is the best way to achieve this.

Hope this helps, welcome aboard.
Cheers
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] Codemax

[Federico T. <ft...@fi...>]

Hi:
I'm Fede from Uruguay. I will try to make some improvement to OpenSTA.
And for that I require Codemax (according to which I read in building
prerequisits page). Could anyone help me in getting Codemax?
I have use OpenSTA in a Load Testing proyect, and after that I have
some ideas to do...
bye

pd: sorry for my english

[OpenSTA-devel] Re: When is Random not random?

[Daniel S. <da...@Op...>]

Mark Elam wrote:
> Yes, but we're calling srand(A1),rand(),srand(a2),rand(),srand(a3)
> ...  So the 'initialize with seed' is occuring before each call to
> rand, which should reset the seed before each call.  However if
> another thread called srand() before the first thread rand() had
> occured I can see the problem you are describing which is why I
> wondered whether just making the srand(),rand() pair atomic (with a
> mutex) would solve the issue.  As rand() would always be called
> after a reset with srand() using the same seed this should give the
> same sequence.

I think you are correct Mark.  I've tried to stimulate this potential
concurrency and timing issue to occur using some contrived scripts
and I can't seem to - that's not to say it won't happen, just that it
seems it is unlikely to happen.  Maybe it's something to do with how
the work is assigned to the threads in a Test Executer, or the
potential points of context switch on my single CPU test host.

The above doesn't matter though, as the whole method of always
reseeding is obviously a really bad way of generating "random"
numbers...  I created a little script like this:
  Definitions
    CHARACTER*512 USER_AGENT
    INTEGER USE_PAGE_TIMERS
    INTEGER RandInt(1-10), RANDOM
    INTEGER Occurrence[10]
    INTEGER Idx
  Code
    Entry[USER_AGENT,USE_PAGE_TIMERS]
    DO Idx = 1, 1000
        GENERATE RandInt
        SET Occurrence[RandInt] = Occurrence[RandInt] + 1
    ENDDO
    DO Idx = 1, 10
       LOG Idx, " occurred ", Occurrence[Idx], " times"
    ENDDO
Running this, very simple, test several times was enough to tell me
that there is definitely not an equal probabiliy of getting each of
the possible returns of RandInt.  I found by, unscientifically,
eyeing the results that there was always much more chance of getting
a 1, 3, or 8; and much less of a chance of getting 2, 5, or 7... not
good!

Obviously the behavior of RANDOM variables needs fixing - it looks
like there's 3 seperate bugs that can be addressed in one update:
 - limited range of returns (bug#415144)
 - less than random distribution of returns (not logged)
 - small chance of REPEATABLE failing due to timing (not logged)
I'm still concerned about keeping the old (bad) behavior of
REPEATABLE as default, with a logged warning, that can then have the
good behavior, or warning, defeated using the .ini file.  My feeling
is that it is not worthwhile keeping the behavior of regular RANDOM
but other opinions are welcome.

I'm starting to put a shortlist together for a 1.4.4 release and it
would be great if Peter (or someone) could donate some code to get
these problems, at least partially, fixed for this release.

Cheers
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] What version to build - Was: When is Random not random?

[Daniel S. <da...@Op...>]

> if I were interested in trying to implement this, am I better
> off grabbing the 1.4.3 source and getting it built than playing
> with the CVS HEAD?

There's currently not very much different between the CVS HEAD and
the 1.4.3 source package - just bug fixes.  Use the CVS HEAD if you
can.

> I'll search the archives on the devel list about building the
> source.  I have been looking at the web page and grabbing the
> various 3rd party libraries, but it seems like some are
> unaccessible (CodeMax?).

CodeMax is the only you should really have a problem with, and I'll
make sure you can get a copy of that.

I hope by "Web site" you are referring to the 2 pages in the Wiki,
starting with this one:
  http://portal.opensta.org/faq.php?topic=BuildingPrerequisites
That is the most current info.

I have updates that I have yet to commit that completely remove use
of STLport - with some pretty amazing speed improvements.  I am
contemplating commiting these to a branch - at least the changes
that only work with no STLport, the ones that work both ways I'll
commit to the HEAD as they are harmless.  I'll keep interested
parties informed of what I'm up to using this mailing list.  Building
without STLport simplifies things because you can just use the
pre-built OmniORB, etc.  It seems to be the way forward for many
reasons ...

I'm fairly confident that using the latest Platform SDK (Server 2003
sp1) works fine - so go-ahead and use that, I intend to for the 1.4.4
release.

> Do I need all of them to build a patch for this, or do I just need
> to build the CyrVDK011 DLL?

If you want to build just the DLL (or whatever you're interested in)
to use with the 1.4.3 release, then you really ought to duplicate the
exact build environment for that release - otherwise you might get
something that works, or you might get unpredictable errors.  Much
safer to build yourself a complete toolset and installation using
whatever build environment you come up with.

Hope this helps,
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

Re: [OpenSTA-devel] Re: When is Random not random?

[Mark E. <ma...@bo...>]

Wickersham, Peter wrote:
> The issue as I see it is the following. Let's say we have defined two
> SCRIPT variables as REPEATABLE RANDOM. A and B with 2 different seeds A1
> and B1
> 
> Now, lets say that if you called srand(A1), you would see the following
> sequence of numbers from rand(). a2, a3, a4, a5, a6. 
<snip>
> <initialize A with seed>
> GENERATE A => a2
> GENERATE A => a3
> <initialize B with seed>
> GENERATE B => b2
> GENERATE B => b3
> GENERATE A => b4


Yes, but we're calling srand(A1),rand(),srand(a2),rand(),srand(a3)...
So the 'initialize with seed' is occuring before each call to rand, 
which should reset the seed before each call. However if another thread 
called  srand() before the first thread rand() had occured I can see the 
problem you are describing which is why I wondered whether just making 
the srand(),rand() pair atomic (with a mutex) would solve the issue.
As rand() would always be called after a reset with srand() using the 
same seed this should give the same sequence.

Or should it? I haven't written anything like this for a few years now 
so I could be way off the mark:-)

On an earlier note, I can't see anything really wrong with having a 
class like you suggested earlier I'm just wondering what I've missed here.

Mark

RE: [OpenSTA-devel] Re: When is Random not random?

[Wickersham, P. <Pet...@di...>]

The issue as I see it is the following. Let's say we have defined two
SCRIPT variables as REPEATABLE RANDOM. A and B with 2 different seeds A1
and B1

Now, lets say that if you called srand(A1), you would see the following
sequence of numbers from rand(). a2, a3, a4, a5, a6.=20

Now, let's say that you called srand(B1) and would normally see the
following sequence of numbers on successive calls to rand(): b2, b3, b4,
b5

First off, srand() will reset the seed for all calls to rand() for all
threads in the process. So, a sequence like below would occur

<initialize A with seed>
GENERATE A =3D> a2
<initialize B with seed>
GENERATE B =3D> b2
GENERATE A =3D> b3
GENERATE B =3D> b4

OR

<initialize A with seed>
GENERATE A =3D> a2
GENERATE A =3D> a3
<initialize B with seed>
GENERATE B =3D> b2
GENERATE B =3D> b3
GENERATE A =3D> b4

You can see how the actual sequences for A and B could often not repeat
just depending on when one or the other was called.

I would imagine that you would want to always be able to count the
sequence of values generated for A and for B to be the same across
multiple runs. I would like to always see A =3D> a2, a3, a4, a5 and B =
=3D>
b2, b3, b4, b5=20

Now, some *nix systems have a rand_r() which is meant to allow a process
to have multiple sequences seeded at different starting points. But, I
don't see this in Windows.

-peter

PS. Looking at the current code, I am guessing that they had this
constant calling of srand() in order to deal with this issue. However,
the result of rand() is not usually the next seed in the sequence within
the actual system call. So, this workaround doesn't work very well.

-----Original Message-----
From: ope...@li...
[mailto:ope...@li...] On Behalf Of Mark
Elam
Sent: Friday, January 06, 2006 1:14 PM
To: ope...@li...
Subject: Re: [OpenSTA-devel] Re: When is Random not random?

Wickersham, Peter wrote:
>=20
> In fact, there already is an issue with this code for REPEATABLE
RANDOM
> and that is the fact that we are using rand() in the first place. This
> is a global function and the seed is a global, as well. So, if you
want
> more than one REPEATABLE RANDOM sequence, this won't work.

I see what you mean, but if the two lines
srand (_cur_seed);
_cur_seed =3D rand();
were an atomic operation, wouldn't that take care of the problem as far=20
as repeatable random is concerned?

Then the #define VOREPRAND could be checked for in _options as to=20
whether repeatable or otherwise (no srand() call) should be used.

Or have I missed something obvious?

Mark




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick
_______________________________________________
OpenSTA-devel mailing list
Ope...@li...
https://lists.sourceforge.net/lists/listinfo/opensta-devel

Re: [OpenSTA-devel] Re: When is Random not random?

[Mark E. <ma...@bo...>]

Wickersham, Peter wrote:
> 
> In fact, there already is an issue with this code for REPEATABLE RANDOM
> and that is the fact that we are using rand() in the first place. This
> is a global function and the seed is a global, as well. So, if you want
> more than one REPEATABLE RANDOM sequence, this won't work.

I see what you mean, but if the two lines
srand (_cur_seed);
_cur_seed = rand();
were an atomic operation, wouldn't that take care of the problem as far 
as repeatable random is concerned?

Then the #define VOREPRAND could be checked for in _options as to 
whether repeatable or otherwise (no srand() call) should be used.

Or have I missed something obvious?

Mark

RE: [OpenSTA-devel] Re: When is Random not random?

[Wickersham, P. <Pet...@di...>]

Yup, it is in both of those files that we use srand() and rand() in the
ways that I describe. I think that the seed generated by GetRandomSeed()
is fine as it is based on time, which is the usual way people seed
rand() for best randomness. However, the issue comes from the fact that
we continually reseed every iteration.=20

In fact, there already is an issue with this code for REPEATABLE RANDOM
and that is the fact that we are using rand() in the first place. This
is a global function and the seed is a global, as well. So, if you want
more than one REPEATABLE RANDOM sequence, this won't work.

The best approach would be create a random generator class and then
associate an instance with each variable that is defined to be either
RANDOM or REPEATABLE RANDOM. That way each sequence can be seeded with
the provided seed in the SCL, or the default (0?).=20

Here is one class that I found in a quick dig around the internet that
is based on a different kind of random generator than the standard
rand() functions.

http://www.coyotegulch.com/products/libcoyotl/twisted_road/

-peter

BTW, if I were interested in trying to implement this, am I better off
grabbing the 1.4.3 source and getting it built than playing with the CVS
HEAD? I'll search the archives on the devel list about building the
source. I have been looking at the web page and grabbing the various 3rd
party libraries, but it seems like some are unaccessible (CodeMax?). Do
I need all of them to build a patch for this, or do I just need to build
the CyrVDK011 DLL?

-----Original Message-----
From: ope...@li...
[mailto:ope...@li...] On Behalf Of Mark
Elam
Sent: Friday, January 06, 2006 12:21 AM
To: ope...@li...
Subject: Re: [OpenSTA-devel] Re: When is Random not random?

Daniel Sutcliffe wrote:
> Peter Wickersham wrote:
>>So, I looked at the source and the code appears to use srand()
>>everytime it calls rand() with the last result from calling rand().
>>However, this actually ends up producing more collisions more
>>quickly than just calling srand() once.=20
>=20
>=20
> Can you give us a source file and line number to save me a little bit
> of work?  I'm only just getting into the inner workings of SCL - the
> previous SCL internals expert no longer has time to give to this=20
> project.
>=20

This will be the variable handling at runtime, in both CECRangeVar.cpp=20
and CECListVar.cpp. The seed generated at the start of the variable use=20
should bring in a semi random element (see comments in=20
CECVar::GetRandomSeed()). Obviously not enough :-) Can't remember if the

options passed at variable contruction will allow a runtime descision as

to if this is REPEATABLE RANDOM or not though. Can probably have a look=20
later if no one else gets a chance.

Mark


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=3D7637&alloc_id=3D16865&op=3Dclick
_______________________________________________
OpenSTA-devel mailing list
Ope...@li...
https://lists.sourceforge.net/lists/listinfo/opensta-devel

Re: [OpenSTA-devel] Re: When is Random not random?

[Mark E. <ma...@bo...>]

Daniel Sutcliffe wrote:
> Peter Wickersham wrote:
>>So, I looked at the source and the code appears to use srand()
>>everytime it calls rand() with the last result from calling rand().
>>However, this actually ends up producing more collisions more
>>quickly than just calling srand() once. 
> 
> 
> Can you give us a source file and line number to save me a little bit
> of work?  I'm only just getting into the inner workings of SCL - the
> previous SCL internals expert no longer has time to give to this 
> project.
> 

This will be the variable handling at runtime, in both CECRangeVar.cpp 
and CECListVar.cpp. The seed generated at the start of the variable use 
should bring in a semi random element (see comments in 
CECVar::GetRandomSeed()). Obviously not enough :-) Can't remember if the 
options passed at variable contruction will allow a runtime descision as 
to if this is REPEATABLE RANDOM or not though. Can probably have a look 
later if no one else gets a chance.

Mark

[OpenSTA-devel] Re: When is Random not random?

[Daniel S. <da...@Op...>]

Peter Wickersham wrote:
> It would appear that the random function used within OpenSTA is not
> very random.
> 
> I was having some issues with using RANDOM on a range of 1 - 120000.

Have you seen/found this bug in the bug DB:
  http://sourceforge.net/tracker/?group_id=10857&atid=110857&func=detail&aid=415144
It looks like you'll never get anything over 32768 anyway.

> It didn't appear to me that it was very random since I seemed to
> get repeating numbers pretty quickly. 

I seem to remember someone talking about this behavior on the Users
list or somewhere a long time ago ... can't find it though and
there's no bug logged.

> So, I looked at the source and the code appears to use srand()
> everytime it calls rand() with the last result from calling rand().
> However, this actually ends up producing more collisions more
> quickly than just calling srand() once. 

Can you give us a source file and line number to save me a little bit
of work?  I'm only just getting into the inner workings of SCL - the
previous SCL internals expert no longer has time to give to this 
project.

> I took the same approach in a sample app and ran a loop for 1000
> times generating random numbers between 1 and 30000.  Using srand()
> once produced 974 unique numbers.  Using srand() every time with
> the last result of rand() produce 173 unique numbers.

Sounds pretty conculsive - the problem is that changing this behavior
means that the REPEATABLE RANDOMs would no longer be ... repeatable,
and this obviously isn't a good thing for a potential users existing
regression test.  We can get around this by making the REPEATABLE
RANDOMs keep the old behavior unless a config file is updated
somehow, so it isn't such a big deal but it does need baring in mind.

> I have not had the opportunity to configure the source to build it
> and test any changes.  In fact, I have the CVS HEAD, so that might
> be problematic if I read the site correctly.  Not to mention that I
> currently have Visual Studio 2003

I'm currently having success using a version built with the current
Platform SDK and no STLport (contemplating dropping my changes into
CVS) but I am using VS6sp6 and have never tried using VS2003 (don't
have it).  If you want to suggest specific code changes then I can
do a test version to make sure this gets sorted.  Please feel free to
log a bug for this on SF.

Cheers
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] When is Random not random?

[Wickersham, P. <Pet...@di...>]

Well,

It would appear that the random function used within OpenSTA is not very
random.

I was having some issues with using RANDOM on a range of 1 - 120000. It
didn't appear to me that it was very random since I seemed to get
repeating numbers pretty quickly.=20

So, I looked at the source and the code appears to use srand() everytime
it calls rand() with the last result from calling rand(). However, this
actually ends up producing more collisions more quickly than just
calling srand() once.=20

I took the same approach in a sample app and ran a loop for 1000 times
generating random numbers between 1 and 30000. Using srand() once
produced 974 unique numbers. Using srand() every time with the last
result of rand() produce 173 unique numbers.

I have not had the opportunity to configure the source to build it and
test any changes. In fact, I have the CVS HEAD, so that might be
problematic if I read the site correctly. Not to mention that I
currently have Visual Studio 2003

-peter=20

[OpenSTA-devel] Re: HTTPS recording issue

[Daniel S. <da...@Op...>]

You are posting to the wrong OpenSTA mailing list - this question
really belongs on the Users mailing list and not the developer list.
Please see this page in the FAQ:
  http://portal.opensta.org/faq.php?topic=MailingLists

There is also plenty of help in the FAQ on recording HTTPS.

Thanks
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] HTTPS recording issue

[PRIYA S. <SP...@co...>]

Hi,
=20
We have problems with recording https URL, using openSTA 1.4.3.2 =
release, XP with IE 6.0.
=20
When IE is opened from opensta gateway, instead of getting the login =
page, we are getting a response http://URL/needs_cookies. The page =
content was also displayed as "needs_cookies". There are no issues when =
we access the application URL from IE 6.0 directly. (We have made sure =
to do the gateway settings correctly while recording.)
=20
Though the URL gets redirected to https, there seems to be a problem =
with displaying response in the browser.=20
We say this beacuse
1. In the gateway.log we can clearly see that gateway gets the relevant =
response for the login page=20
2. After stopping the recording, on examining the primary URL details, =
the page was getting fully displayed. =20
=20
However the page contents do not get displayed in IE browser opened by =
the gateway.
=20
When we tried the opensta 1.4.2 version (with gwhttp.dll patched with =
http://- for https), we had a similar issue but after proceeding for 4 =
pages.=20
=20
Please help.
=20
Thanks
Priya S,  Covansys
Confidentiality Statement:

This message is intended only for the individual or entity to which it =
is addressed. It may contain privileged, confidential information which =
is exempt from disclosure under applicable laws. If you are not the =
intended recipient, please note that you are strictly prohibited from =
disseminating or distributing this information (other than to the =
intended recipient) or copying this information. If you have received =
this communication in error, please notify us immediately by return =
email.

[OpenSTA-devel] [ opensta-Bugs-1389081 ] Gateway does not munge HTTPS URLs in CSS content

[SourceForge.net <no...@so...>]

Bugs item #1389081, was opened at 2005-12-23 13:56
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1389081&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
Group: Behavioral
Status: Open
>Resolution: Accepted
Priority: 5
Submitted By: Daniel Sutcliffe (dansut)
Assigned to: Daniel Sutcliffe (dansut)
Summary: Gateway does not munge HTTPS URLs in CSS content

Initial Comment:
While recording HTTPS traffic the gateway must
translate all HTTPS URLs in the content returned to the
client being recorded to be http://{ - this is to stop
the client attempting to make SSL/TLS connections to
the Gateway which cannot be recorded and will stop the
halt the gateway.  You can see this happen if you have
a "Unsupported protocol ..." comment in your created
script and your recording browser stops being able to
access any site.

In the OpenSTA 1.4.3 release and earlier the only
content they had this munging done was HTML and
javascript.  The most obvious other type of content
that may need munging is CSS ... but there may be others.

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2005-12-23 14:42

Message:
Logged In: YES 
user_id=19748

A possible workaround for this problem is to use the
Remote(manual) recording method and not specify any proxy
for HTTPS content.  This obviously means that the failure
case requests will not be recorded but it may at least allow
some people to make partial recordings of their sites, to
which they can add the problem requests later.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1389081&group_id=10857

[OpenSTA-devel] [ opensta-Bugs-1389081 ] Gateway does not munge HTTPS URLs in CSS content

[SourceForge.net <no...@so...>]

Bugs item #1389081, was opened at 2005-12-23 13:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1389081&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
Group: Behavioral
Status: Open
Resolution: None
Priority: 5
Submitted By: Daniel Sutcliffe (dansut)
Assigned to: Daniel Sutcliffe (dansut)
Summary: Gateway does not munge HTTPS URLs in CSS content

Initial Comment:
While recording HTTPS traffic the gateway must
translate all HTTPS URLs in the content returned to the
client being recorded to be http://{ - this is to stop
the client attempting to make SSL/TLS connections to
the Gateway which cannot be recorded and will stop the
halt the gateway.  You can see this happen if you have
a "Unsupported protocol ..." comment in your created
script and your recording browser stops being able to
access any site.

In the OpenSTA 1.4.3 release and earlier the only
content they had this munging done was HTML and
javascript.  The most obvious other type of content
that may need munging is CSS ... but there may be others.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1389081&group_id=10857

[OpenSTA-devel] [ opensta-Bugs-1386047 ] Pb Default_headers / Host: field when working with >1 site

[SourceForge.net <no...@so...>]

Bugs item #1386047, was opened at 2005-12-20 05:50
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1386047&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
>Group: Inconvenience
>Status: Deleted
>Resolution: Accepted
>Priority: 2
Submitted By: Marc MAZAS (mmazas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Pb Default_headers / Host: field when working with >1 site

Initial Comment:
When I capture a script involving more than one 
distant site (let's say 2, site1 and site2 on <host1> 
and <host2>, IBM WebSphere 5.0 on Windows 2000), the 
capture :
- creates only one DEFAULT_HEADERS constant 
containing "Host: <host1>",
- does not record the Host: header field in the WITH 
{...} clause of the URIs to site1
- does record the "Host: <host2>" header field in the 
WITH {...} clause of the URIs to site2 (as the last 
field).

a) When I replay the exact captured script, it seems 
to work fine.

b) Then when I change the order of the fields in the 
WITH clause of the URIs to site2, I get 400 errors 
(BAD Req).

c) Then when I add a new DEFAULT_HEADERS2 constant not 
containing any "Host: " field and I use it in all the 
URIs to site2 instead of the first one, it works fine.

I don't really know why it works in a) and not b) 
(there is or there is not a conflict on the duplicate 
Host: fields depending on they position) ; however I 
believe there should not be any duplicate "Host:" 
field.
IHMO the best would be that the capture creates two 
distinct DEFAULT_HEADERS constants with their 
respective "Host:" field, and no "Host:" field in the 
WITH clauses.

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2005-12-20 08:53

Message:
Logged In: YES 
user_id=19748

This is not a bug, and certainly not a "Serious" one!
The script works as it is recorded, if you edit and it fails
then it is just a scripting error.

Please discuss problems on the OpenSTA Users Mailing list to
verify if and what the actual bug report should be before
making entries into the bug database.  This specifically
goes for information on why your a) case works but b) doesn't.

This report will be deleted.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1386047&group_id=10857

[OpenSTA-devel] [ opensta-Bugs-1386047 ] Pb Default_headers / Host: field when working with >1 site

[SourceForge.net <no...@so...>]

Bugs item #1386047, was opened at 2005-12-20 11:50
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1386047&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
Group: Serious
Status: Open
Resolution: None
Priority: 5
Submitted By: Marc MAZAS (mmazas)
Assigned to: Nobody/Anonymous (nobody)
Summary: Pb Default_headers / Host: field when working with >1 site

Initial Comment:
When I capture a script involving more than one 
distant site (let's say 2, site1 and site2 on <host1> 
and <host2>, IBM WebSphere 5.0 on Windows 2000), the 
capture :
- creates only one DEFAULT_HEADERS constant 
containing "Host: <host1>",
- does not record the Host: header field in the WITH 
{...} clause of the URIs to site1
- does record the "Host: <host2>" header field in the 
WITH {...} clause of the URIs to site2 (as the last 
field).

a) When I replay the exact captured script, it seems 
to work fine.

b) Then when I change the order of the fields in the 
WITH clause of the URIs to site2, I get 400 errors 
(BAD Req).

c) Then when I add a new DEFAULT_HEADERS2 constant not 
containing any "Host: " field and I use it in all the 
URIs to site2 instead of the first one, it works fine.

I don't really know why it works in a) and not b) 
(there is or there is not a conflict on the duplicate 
Host: fields depending on they position) ; however I 
believe there should not be any duplicate "Host:" 
field.
IHMO the best would be that the capture creates two 
distinct DEFAULT_HEADERS constants with their 
respective "Host:" field, and no "Host:" field in the 
WITH clauses.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1386047&group_id=10857

[OpenSTA-devel] Re: Addition of Bandwidth Limiting Functionality

[Daniel S. <da...@Op...>]

Swathy Kamal wrote:
> We have identified and implemented a Bandwidth Limiting
> Functionality for OpenSTA,the details of which are listed below.
> Also we are interested in contributing this back to the OpenSTA
> project.  Kindly let us know the procedures for the same.

Always good to hear this kind of news - help improving OpenSTA of any
kind is always appreciated, although more bug fixes are really what's
most needed.

As for how to get your changes into the released OpenSTA, I've
written a bit about this in the FAQ Wiki:
  http://portal.opensta.org/faq.php?topic=DevelopmentContributions

Basically, I'll need to get your changed sources, know the exact
source version you started with, and the build environment you are
using ...

It sounds like a fairly big set of changes - so it is unlikely that
we'll get it into the 1.4 release series.  It can definitely go into
1.5 series once that gets forked - until this time, I'm sure we will
be able to make your changes available to get wider testing in some
fashion, something like an addon in the way we do for BView.

Looking forward to hear more,
Cheers,
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] [ opensta-Bugs-1375234 ] Replay of SSL requests over 16K fail with timeout

[SourceForge.net <no...@so...>]

Bugs item #1375234, was opened at 2005-12-07 07:00
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1375234&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Replay
>Group: Behavioral
Status: Open
>Resolution: Accepted
Priority: 5
Submitted By: Sarah Williams (williams_sarah)
>Assigned to: Daniel Sutcliffe (dansut)
>Summary: Replay of SSL requests over 16K fail with timeout

Initial Comment:
Am currently using OpenSTA 1.4.3
Was browsing the forum for a solution to this problem
and found that a few people have reported it too.
 
I'll detail what I've tried here:
-good-
. I've recorded a script to (upload a 10Kb file) in the
Modeler (https).
. The script get recorded correctly. It compiles. I
play it back. It works fine.
. Entire content reaches the server.
-/good-
 
-good-
. I increase the content (by 1KB) to be uploaded within
the script.
. It compiles; play back is fine. Content-length is
correctly sent by OpenSTA.
. Entire content reaches the server.
-/good-
 
-bad-
. I increase the content to around 15.7Kb
. It compiles.
. Content-length is correctly sent by OpenSTA.
. Play back times out on the upload.
. I've printed the content of the request. Only about
15227 bytes get printed.
-/bad-
 
As a side note, this  works on http

Tested on:
1. WEBLOGIC on Intel
2. Oracle 10gAS on Sun Sparc

Any workaround would be appreciated

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2005-12-18 19:53

Message:
Logged In: YES 
user_id=19748

Reproduced this problem in replay from the Modeler, and
narrowed it down any HTTPS requests whose total content is
over 16384 byte (16K).

Seems to be easily reproducible, so only really want more
info posted in this bug report about situations that
contradict my above diagnosis, or any workaround.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1375234&group_id=10857

[OpenSTA-devel] [ opensta-Bugs-897856 ] Recording HTTPS fails because newer IE escapes URL

[SourceForge.net <no...@so...>]

Bugs item #897856, was opened at 2004-02-16 02:43
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=897856&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
Group: Serious
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Alain Ribault (ribault)
Assigned to: Daniel Sutcliffe (dansut)
Summary: Recording HTTPS fails because newer IE escapes URL

Initial Comment:
HTTPS recording :
With my browser (IE6, latest patches), when the opensta
receives an http page with https links, it transforms
the link like http://{ ..., which is correct by opensta
implementation. The problem is that IE escapes '{' by
'%7D' and the gateway does not understand http://%7D,
so the capture stops by 'page not found'.

Hisashi Iwashimizu wrote that it is after applying
MS04-004 
 security patch to IE6. 

I did some tests with Mozilla and it worked fine
(because no escape code). 

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2005-12-18 13:54

Message:
Logged In: YES 
user_id=19748

This problem had NOTHING to do with script playback - it was
entirely a recording issue.  This fact does not matter
anyway, as the problem is fixed in the current release of
OpenSTA (1.4.3).

Also, please do not use this bug database as a type of
support forum - this just clutters up bug reports and makes
them more confusing.

----------------------------------------------------------------------

Comment By: kikibug (kikibug)
Date: 2005-12-07 04:42

Message:
Logged In: YES 
user_id=1314392

Should recording the script OK with '%7D' instead of '{'
affect playback? I have a problem that I do not know for
sure to be a bug, I have entered it in the mailing lists - I
cannot play back HTTPS script (which was recorded OK and
shows https:// in the requests) when going from the second
to the third HTTPS page in a row. So I am wondering if this
occurence with IE6 may affect playback in any way?

----------------------------------------------------------------------

Comment By: Lance (stampy)
Date: 2005-10-19 14:16

Message:
Logged In: YES 
user_id=1137295

hm, i too also get the %7B when using IE6. Im gathering from 
your previous response that its a problem in IE. Any other 
ideas?

im using osta 1.4.3.20

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2005-06-15 12:18

Message:
Logged In: YES 
user_id=19748

mvemers wrote:
> This problem seems to have re-occurred - IE is showing %7B
> in the location bar again:

This actually doesn't mean that this bug has re-occurred,
the fix to this problem was to allow the gateway to
understand the urlencoded form of the { character as well. 
It is only a problem if you cannot record HTTPS because of
this...

----------------------------------------------------------------------

Comment By: Mark Vevers (mvevers)
Date: 2005-06-15 11:26

Message:
Logged In: YES 
user_id=13500

This problem seems to have re-occurred - IE is showing %7B
in the location bar again:

gwhttp.dll: 1.4.3.6 
IE Version:  6.0.2900.2180.xpsp_sp2_gdr.050301-1519

Any suggestions?

Thanks


----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2005-01-24 13:37

Message:
Logged In: YES 
user_id=19748

And addendum to this fix has been merged into the CVS HEAD.
The case were a Location: header was specified from the root
during a HTTPS redirect was not dealt with.  Thanks to Thierry
for spotting this.  This will be included in the OpenSTA 1.4.3
release.

gwhttp.dll: 1.4.3.5
OpenSTA: 1.4.3.18

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2004-12-20 17:29

Message:
Logged In: YES 
user_id=19748

The most recently mentioned fix has been merged into the CVS
HEAD.  It will become generally available in the OpenSTA 1.4.3
release.

gwhttp.dll: 1.4.3.4
OpenSTA: 1.4.3.13

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2004-12-20 14:33

Message:
Logged In: YES 
user_id=19748

I now have a "proper" fix for this that treats the URL encoded
version of { (which is %7b not %7d as Alain initially reported)
as equivilant to the {.

The fix had to be extended with a little fudge because it
uncovered and issue with IE and redirects.  This is best shown
by an example:  If I go to record https://osta.lan/mail/ I'll 
get http://{osta.lan/mail/ which the new "fixed" IE will rewrite
to http://%7Bosta.lan/mail/ - this will work fine, but if the
Web server then returns with a HTTP 302 and "Location:
src/login.php" then IE will get confused and try to redirect to
http://%7Bosta.lanan/mail/src/login.php - obviously the extra
2 chars and the domain name changing length confuse IE.  It
is unlikely that anyone in the "real world" will find this IE
buglet so I worked around it by having the gateway rewrite
the "Location:" header so that it is always fully specified for
SSL recordings (the recording .all gets the original value).

I was convinced this was something the gateway was doing
wrong for a long while but hours spent with a sniffer and
tests with other browsers showed that it was IE.

The fix also "enhances" the console and trace output from
the gateway.

This gateway code is ugly...

----------------------------------------------------------------------

Comment By: John L (johnleij)
Date: 2004-09-21 13:34

Message:
Logged In: YES 
user_id=999135

Hi all,

As I wasn't able to build OpenSTA due to lack of access to 
required third party tools, I binary patched the product. I'm 
not sure whether the license gives me the rights to distribute 
the patch, but I gather that it is for everybodys' best if I at 
least inform you of what I did.

I binary patched gwhttp.dll to make it work with SSL even 
after security patch MS04-004 on Internet Explorer 6.

The idea is to replace the ordinary "http://{" sequence 
with "http://-" so that IE doesn't destroy the URL when it is 
UrlEncoded. Note that this quick and dirty fix will make 
OpenSTA stop working with sites that begin with '-' (not very 
common though).

In gwhttp.dll I patched:
- Changed "http://{" to "http://-" (address ~00029380h)
- Changed "{" to "-" (address ~00029c40h)

I could write a long disclaimer here to inform you that I can't 
guarantee that the patch works, but I think you already know 
that. The patch surely seems to work for me.


----------------------------------------------------------------------

Comment By: jmrwnd (jmrwnd)
Date: 2004-09-21 09:23

Message:
Logged In: YES 
user_id=1125756

Has johnleij's fix been implemented anywhere, and is there 
any way to get this alternate version of the code?  I am 
unable to alter and build the source code myself and need a 
workaround for this problem ASAP.  I absolutely must test 
with IE6 or Netscape 6, but both escape the '{' character.

If all else fails, is there another similar tool that could record 
https traffic with these browsers that could run off of a 
laptop?  Any help is greatly appreciated.  Thanks.

----------------------------------------------------------------------

Comment By: John L (johnleij)
Date: 2004-03-16 09:28

Message:
Logged In: YES 
user_id=999135

A possible quick-fix could be to change the implementation of 
OpenSTA so that instead of using "http://{" to identify SSL, 
we could use "http://-". The latter isn't common on the 
internet and is not tampered with by MSIE.

A search through the source code gave me the idea to chage 
the value of constants CYRANO_SSL_URL and 
CYRANO_SSL_STR in HttpGateway\gwhttp\context.hpp to 
reflect this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=897856&group_id=10857

[OpenSTA-devel] Addition of Bandwidth Limiting Functionality

[Kamal, S. \(Cognizant\) <Swa...@co...>]

Hi All,

  We have identified and implemented a Bandwidth Limiting Functionality for=
 OpenSTA,the details of which are listed below.
Also we are interested in contributing this back to the OpenSTA project.=
 Kindly let us know the procedures for the same.


Details of the new Functionality:
*********************************

OpenSTA Bandwidth Limiting
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D

Functionality:
--------------
 The new functionality here is to Add "Bandwidth Limiting" feature to=
 OpenSTA. The OpenSTA commander will have additional provision to enable=
 bandwidth limiting and set the bandwidth to be simulated during testing.=
 Bandwidth limiting is the ability to simulate a lower bandwidth network=
 link over a higher bandwidth link.=0D

Bandwidth Limiting Algorithm:
----------------------------
 The Bandwidth Limiting algorithm used is a modified version of bandwidth=
 limiting algorithm used in "Trickle". More information about trickle and=
 the source code is available at <http://monkey.org/~marius/pages/?page=
=3Dtrickle>.
   The basic idea behind limiting bandwidth is the to limit the amount of=
 data read and written during each operation and introducing delays.

The Steps followed are as follows:
        1.If the total amount of data to be read or written is less than=
 the bandwidth to simulate, the read/write operation proceeds without any=
 change.
        2.If the length of data to be sent, or the length of data requested=
 in the read is greater than the bandwidth to be simulated, a delay is=
 calculated as
        delay =3D (length of data / simulated rate)*1000 (where delay is in=
 milliseconds)
        The corresponding length of data would be equal to simulated rate.=
 Thereby
        adjusted_length =3D simulated_rate
        3.If data is sent/received after each delay in this manner, the=
 traffic would become bursty. This burstiness may even result in over=
 shaping as network conditions are changing, and the scheduler could=
 allocate more I/O to the stream in question. In order to avoid burstiness,=
 a smoothing factor is introduced for the sleep time and transfer length.=
 If the calculated delay is greater than the time smoothing factor, the=
 delay is set equal to the time smoothing factor and the adjusted length is=
 calculated as
        delay =3D time_smoothing_factor
        adjusted length  =3D simulated rate * time_smoothing_factor /1000
        4.If the new adjusted length is too small i.e., almost equal to=
 zero then we set the adjusted length as the length smoothing factor and=
 calculate the new delay as
        adjusted length =3D lsmooth (where lsmooth is the length smoothing=
 factor)
        delay =3D adjusted length /rate

Implementing Bandwidth Limiting Algorithm in OpenSTA:
----------------------------------------------------
 As opposed to trickle, which uses synchronous sockets OpenSTA uses=
 Asynchronous sockets for running the tests. Hence the algorithm used in=
 trickle is modified to simulate Bandwidth Limiting for the Asynchronous=
 read and write in OpenSTA. For write the restricted length and delay time=
 is calculated using the algorithm and data, which is equivalent to the=
 adjusted length, is sent after each sleep (sleep time is equivalent to the=
 delay time calculated) till the whole data is sent.

For read, since the sockets are asynchronous, limiting can't happen by=
 simply performing a read after a sleep. So the restricted length is=
 calculated, and a read is scheduled as per the restricted length. After=
 completion of each read when a call back is received from the client we=
 introduce a sleep. The sleep time is proportional to the data received and=
 is calculated as:=0D
sleep_msec =3D (length)/simulated_rate*1000
(where, sleep_msec is the sleep time and simulated_rate is the restricted=
 read rate.)
But if the sleep time calculated, is too small then the sleep would not=
 make any effect. Such small sleep times accumulated for each socket. Once=
 the accumulated time is greater than 100 milliseconds, a sleep is=
 initiated.

Changes for the GUI:=0D
--------------------
An additional checkbox has been added to the openSTA commander, using which=
 a user can choose whether bandwidth limiting is enabled. On selecting this=
 checkbox the user is allowed to edit the "receive rate"(in kilo bits) and =
 "send rate"(in kilo bits) for Bandwidth limiting.=0D
      Once the user input is captured the data has to be passed from the=
 Test Manager to Test Executer. Since these are two independent exe(s)=
 there has to be a mechanism to pass parameters between the two. OpenSTA=
 uses CORBA for this. This mechanism in OpenSTA is used to pass the=
 bandwidth limiting parameters. Also the entire bandwidth limiting=
 functionality is captured in the singleton class CBwtThrottle.


Thanks,
Swathy

This e-mail and any files transmitted with it are for the sole use of the=
 intended recipient(s) and may contain confidential and privileged=
 information.
If you are not the intended recipient, please contact the sender by reply=
 e-mail and destroy all copies of the original message.=0D
Any unauthorised review, use, disclosure, dissemination, forwarding,=
 printing or copying of this email or any action taken in reliance on this=
 e-mail is strictly=0D
prohibited and may be unlawful.

  Visit us at http://www.cognizant.com

[OpenSTA-devel] Re: (no subject)

[Daniel S. <da...@op...>]

Ludovic Delachaux wrote:
> I'm looking for the pdf documentation provided by CYRANO.
> 
> Can anyone send me please a copy of it ?

A few points:
 - You are on the wrong mailing list - this has nothing to do with
   the "development" of OpenSTA and everything to do with its use,
   and therefore should be on the OpenSTA-users mailing list.
 - If anyone could provide you such items then legally they shouldn't
   really because of their licensing terms.
 - These documents are now so out of date that there usefulness is
   very questionable.

Hope this helps
/dan
-- 
Daniel Sutcliffe <Da...@Op...>
  OpenSTA part-time caretaker - http://OpenSTA.org/

[OpenSTA-devel] (no subject)

[<lud...@fr...>]

Hello,

I'm looking for the pdf documentation provided by CYRANO.

Can anyone send me please a copy of it ?

Regards.

Ludovic.