Thread: [OpenSTA-devel] Codemax
Brought to you by:
dansut
Menu
▾
▴
opensta-devel
|
From: Federico T. <ft...@fi...> - 2006-01-09 18:40:46
|
Hi: I'm Fede from Uruguay. I will try to make some improvement to OpenSTA. And for that I require Codemax (according to which I read in building prerequisits page). Could anyone help me in getting Codemax? I have use OpenSTA in a Load Testing proyect, and after that I have some ideas to do... bye pd: sorry for my english |
|
From: Daniel S. <da...@Op...> - 2006-01-09 20:34:20
|
Federico Toledo wrote: > I'm Fede from Uruguay. I will try to make some improvement to > OpenSTA. And for that I require Codemax (according to which I read > in building prerequisits page). Could anyone help me in getting > Codemax? I've made sure that Fede can get hold of CodeMax - all I ask in return for this (and future) help, is that you make sure to "give back" to the OpenSTA project and community ;-) > I have use OpenSTA in a Load Testing project, and after that I have > some ideas to do... Please share your ideas with us before you go ahead implementing anything. This is important for a number of reasons: - Someone else may have already done what you intend, or something similar, that isn't yet in the publically accessible source archives or CVS. - Someone may have the same desires as you, and be willing to help you achieve your goals quicker. - It would be great to get any improvements or bug fixes you make back into the published source as quick as possible - and that can only really happen if you work within the goals and standards of this project from the start, communicating with the other project developers is the best way to achieve this. Hope this helps, welcome aboard. Cheers /dan -- Daniel Sutcliffe <Da...@Op...> OpenSTA part-time caretaker - http://OpenSTA.org/ |
|
From: Federico T. <ft...@fi...> - 2006-01-10 12:57:46
|
Ok. thanks for the fast answer. When I use OpenSTA and run a TaskGroup with validations in the scripts (using tests case sentence) if a validation fails I didn't know why fails. I know that LoadRunner has a facility to know what the server answered. This tool save the pages that didn't pass the test case. So we can view the page returned by the server and then we can debug the er= ror. Resuming: i want to save the pages returned with error by the server to know why fails. this is the first idea. Then I want to investigate how to work with SOA and WebService, but this will be in the future. I have implement a couple of tools in Java to do some activities that can be done automaticaly. For example, a tool to refresh the script's parameters files (because after runing a test there are data that is no more valid, so we have to repeat the sql and export the result to the script's parameters files to can run again) I would like to integrate this funcionality to OpenSTA. Tell me what do you think. I have to take a time to do all this 'cause i have to get Visual Studio. I have a question: Could I use dotnet? ok, thks bye FedeFede |
|
From: Olaf K. <ok...@ab...> - 2006-01-10 15:14:13
|
Federico Toledo wrote: > I know that LoadRunner has a facility to know what the server answered. > This tool save the pages that didn't pass the test case. > So we can view the page returned by the server and then we can debug th= e error. >=20 > Resuming: i want to save the pages returned with error by the server > to know why fails. I'd like to request a slightly different feature (if I understand you correctly): Saving a page should be explicitly triggered by the script whenever the script would like to save a response - not on HTTP errors in general: I believe that load tests can easily and correctly test that state 404 is returned by the application. On the other hand if the page is returned with http-state 200 but contains unexpected content (e.g. an application error message) the script should be able to save this unexpected content. > I have to take a time to do all this 'cause i have to get Visual Studio= . > I have a question: Could I use dotnet? Not having developed any program component for OpenSTA my opinion is probably inferior to others, but I'd prefer not adding the huge dependence on dot net (or worse a particular dot net version) to implement "just an additional feature". This'd be different once lots of features would require dot net (or be easier to add using dot net) or at least if it's planned to move into that direction rather soon. The current stable release definitely should not add the dependency on dot ne= t. My 2 cents (=80uro-cent, of course :-) Olaf |
|
From: Daniel S. <da...@Op...> - 2006-01-10 20:23:37
|
Olaf Kock wrote:
> I'd like to request a slightly different feature (if I understand
> you correctly): Saving a page should be explicitly triggered by
> the script whenever the script would like to save a response - not
> on HTTP errors in general: I believe that load tests can easily
> and correctly test that state 404 is returned by the application.
> On the other hand if the page is returned with http-state 200 but
> contains unexpected content (e.g. an application error message) the
> script should be able to save this unexpected content.
Your description makes me think of one worthwhile change that could
probably be made fairly easily and is unlikely to upset any existing
users:
If after doing a GET/POST you try to pull something from the BODY
content returned using:
LOAD RESPONSE_INFO BODY ON ConId INTO VarName, WITH DomIdentifier
And the DomIdentifier does not match the structure of the content
returned, then perhaps it would be nicer (rather than falling on its
face with "unresolved variable" as it does now) if the whole BODY
could be logged somewhere or placed in a variable.
Of Course you can do something like this now by saying:
LOAD RESPONSE_INFO BODY ON ConId INTO VarName, &
WITH DomIdentifier, RETURNING STATUS RetStat
IF (RetStat < 0) THEN
LOAD RESPONSE_INFO BODY ON ConId INTO VarName
LOG VarName
ENDIF
but perhaps something more elegant and automatic would be much more
useful.
Just thinking out loud,
/dan
--
Daniel Sutcliffe <Da...@Op...>
OpenSTA part-time caretaker - http://OpenSTA.org/
|
|
From: Daniel S. <da...@Op...> - 2006-01-10 20:00:56
|
Federico Toledo wrote: > When I use OpenSTA and run a TaskGroup with validations in the > scripts (using tests case sentence) if a validation fails I didn't > know why fails. Presumably you have done some sort of conditional before your: FAIL TEST-CASE Why not just LOG the error (or BODY contents) before doing the fail. Maybe I'm missing something but hopefully you can explain further. > I know that LoadRunner has a facility to know what the server > answered. This tool save the pages that didn't pass the test case. > So we can view the page returned by the server and then we can > debug the error. How are the "test cases" defined? Do the pages get written out each to their own file, or in a log? > Resuming: I want to save the pages returned with error by the > server to know why fails. If you just want to do it by HTTP Code then this would be relatively easily done in SCL - except if you want them writing out to seperate files, or if they are longer than 64K... > Then I want to investigate how to work with SOA and WebService, but > this will be in the future. I think you'll find lots of people interested in this type of work and ideas that make testing this type of use of HTTP easier or more complete will be most welcomed. > I have implement a couple of tools in Java to do some activities > that can be done automaticaly. > For example, a tool to refresh the script's parameters files > (because after runing a test there are data that is no more valid, > so we have to repeat the SQL and export the result to the script's > parameters files to can run again) > I would like to integrate this funcionality to OpenSTA. The ability to run "tools" that can return your test system to its baseline standard either at the end or prior to starting a test, is a problem that many people have solved in many different ways. Ideas of ways to make these tasks generically easier to automate within OpenSTAs toolset would be welcomed. > I have to take a time to do all this 'cause I have to get Visual > Studio. I have a question: Could I use dotnet? To the best of my knowledge, no you can not use .Net or any Visual Studio version newer than 6 to build OpenSTA. The projects long term goal is to move away from requiring ANY development environment that you have to pay for to build OpenSTAs tools. Hope this helps, /dan -- Daniel Sutcliffe <Da...@Op...> OpenSTA part-time caretaker - http://OpenSTA.org/ |
|
From: Federico T. <ft...@fi...> - 2006-01-11 18:52:56
|
Ok, thanks for your comments (all) > I'd like to request a slightly different feature (if I understand you > correctly): Saving a page should be explicitly triggered by the script > whenever the script would like to save a response - not on HTTP errors > in general: I believe that load tests can easily and correctly test that > state 404 is returned by the application. On the other hand if the page > is returned with http-state 200 but contains unexpected content (e.g. an > application error message) the script should be able to save this > unexpected content. > Presumably you have done some sort of conditional before your: > FAIL TEST-CASE > Why not just LOG the error (or BODY contents) before doing the fail. > How are the "test cases" defined? > Do the pages get written out each to their own file, or in a log? > If you just want to do it by HTTP Code then this would be relatively > easily done in SCL - except if you want them writing out to seperate > files, or if they are longer than 64K... I will explain a little more. In some cases, in a execution I see that the server return aplication's errors and so the test case fail. If I want to know why it fails, what try to say me the aplication, i can't. For example, to do this to one VU I use BView, because I can view the pages returned by the aplication. It's allright. But if I run the script with two VU and there are problems, because the aplication fails with concurrents users (for example too), I'd like to view the pages with error to detect the error. The easiest form to do this is that OpenSTA save this pages automatically. I hope that you understand me. > Your description makes me think of one worthwhile change that could > probably be made fairly easily and is unlikely to upset any existing > users: > > If after doing a GET/POST you try to pull something from the BODY > content returned using: > LOAD RESPONSE_INFO BODY ON ConId INTO VarName, WITH DomIdentifier > > And the DomIdentifier does not match the structure of the content > returned, then perhaps it would be nicer (rather than falling on its > face with "unresolved variable" as it does now) if the whole BODY > could be logged somewhere or placed in a variable. > > Of Course you can do something like this now by saying: > LOAD RESPONSE_INFO BODY ON ConId INTO VarName, & > WITH DomIdentifier, RETURNING STATUS RetStat > IF (RetStat < 0) THEN > LOAD RESPONSE_INFO BODY ON ConId INTO VarName > LOG VarName > ENDIF > but perhaps something more elegant and automatic would be much more > useful. Ok, thats a good idea, that could helpme with other problems, but it isn't what I want. > > Then I want to investigate how to work with SOA and WebService, but > > this will be in the future. > > I think you'll find lots of people interested in this type of work > and ideas that make testing this type of use of HTTP easier or more > complete will be most welcomed. Ok, thats true, but for now I dont know a lot about so I'll try on this in the feature. > > I have implement a couple of tools in Java to do some activities > > that can be done automaticaly. > > For example, a tool to refresh the script's parameters files > > (because after runing a test there are data that is no more valid, > > so we have to repeat the SQL and export the result to the script's > > parameters files to can run again) > > I would like to integrate this funcionality to OpenSTA. > > The ability to run "tools" that can return your test system to its > baseline standard either at the end or prior to starting a test, is a > problem that many people have solved in many different ways. Ideas > of ways to make these tasks generically easier to automate within > OpenSTAs toolset would be welcomed. I supouse that you like me parametrice your scripts, and get your values from file. This files you have to generate with some data from DataBase with the corresponding SQL. I have 10 .fvr with parameters. After each execution I have to refresh the contenent of this 10 archives. My tool gets the sql from a file (acording how you configure a XML), and execute this and save the result in the corresponding .fvr. Before i made the tool, I take 10 minutes to do this job, and now I take 1 second to press the buton. I made this and anothers tools in java, but could be better that we can integrate all of this to openSTA. thks to all greetings FedeFede |
|
From: Daniel S. <da...@Op...> - 2006-01-12 02:52:56
|
Federico Toledo wrote: > I will explain a little more. > In some cases, in a execution I see that the server return > aplication's errors and so the test case fail. OK, so this is the key question: these "application errors", what signifies them? Because, as far as I am concerned there is no standard way of reporting Web application errors - HTTP error codes certainly normally have little relation to Web application errors. > If I want to know why it fails, what try to say me the aplication, > I can't. For example, to do this to one VU I use BView, because I > can view the pages returned by the aplication. It's alright. But > if I run the script with two VU and there are problems, because the > aplication fails with concurrents users (for example too), I'd like > to view the pages with error to detect the error. The easiest form > to do this is that OpenSTA save this pages automatically. I hope > that you understand me. I think I understand what your intentions are, I just can't understand how you intend to spot the error automatically. > > Of Course you can do something like this now by saying: > > LOAD RESPONSE_INFO BODY ON ConId INTO VarName, & > > WITH DomIdentifier, RETURNING STATUS RetStat > > IF (RetStat < 0) THEN > > LOAD RESPONSE_INFO BODY ON ConId INTO VarName > > LOG VarName > > ENDIF > > but perhaps something more elegant and automatic would be much > > more useful. > > Ok, that's a good idea, that could help me with other problems, but > it isn't what I want. If you want to do something that it is not caused by contents of the returned HTTP Reponse body; say you want to make your condition on the HTTP return code then you just need a ",RETURNING CODE HttpCode" on your SCL HTTP request command, then change the conditional based on this return. If you did base it on HTTP error code then for many codes the returned BODY would not be at all helpful. Does that help you more? If not, give us more of a clue of what the condition you would base your "failure log" on. > I supouse that you like me parameterize your scripts, and get your > values from file. > This files you have to generate with some data from DataBase with > the corresponding SQL. > I have 10 .fvr with parameters. After each execution I have to > refresh the contenent of this 10 archives. My tool gets the SQL > from a file (acording how you configure a XML), and execute this > and save the result in the corresponding .fvr. > Before I made the tool, I take 10 minutes to do this job, and now I > take 1 second to press the buton. > I made this and anothers tools in java, but could be better that we > can integrate all of this to OpenSTA. The Modeler can already use an ODBC data source to populate an FVR file - is this the area you would like to extend? Or would you want to integrate into the Commander, or just be another seperate tool? Cheers /dan -- Daniel Sutcliffe <Da...@Op...> OpenSTA part-time caretaker - http://OpenSTA.org/ |
|
From: Federico T. <ft...@fi...> - 2006-01-12 18:58:47
|
> > OK, so this is the key question: these "application errors", what > signifies them? Because, as far as I am concerned there is no > standard way of reporting Web application errors - HTTP error codes > certainly normally have little relation to Web application errors. > In my case, for example, i want to receive a page with the message "Facturaci=F3n completa" so I use a Load response info ... and compare the string. But if the load response info fails because not match with the page i can't know what happened. I could save a log...but i want to save the HTML page in the case that fail the load response info ... or fail the test case, and then, after of a execution, i could see the pages returned in each error. This facilitates the concurrence test more than the stress test. Ok? Understand my intention? > > I supouse that you like me parameterize your scripts, and get your > > values from file. > > This files you have to generate with some data from DataBase with > > the corresponding SQL. > > I have 10 .fvr with parameters. After each execution I have to > > refresh the contenent of this 10 archives. My tool gets the SQL > > from a file (acording how you configure a XML), and execute this > > and save the result in the corresponding .fvr. > > Before I made the tool, I take 10 minutes to do this job, and now I > > take 1 second to press the buton. > > I made this and anothers tools in java, but could be better that we > > can integrate all of this to OpenSTA. > > The Modeler can already use an ODBC data source to populate an FVR > file - is this the area you would like to extend? Or would you want > to integrate into the Commander, or just be another seperate tool? ok, but the functionality that you say in the Modeler populate an FVR at the moment that we create the variable. After an execution we can not update the contenent of a FVR quickly and easily. Now i use my tool as a separate tool, but could be good that all be together in the same toolset, perhaps as a part of the Commander. saludos! FedeFede |
|
From: Daniel S. <da...@Op...> - 2006-01-13 02:34:25
|
Daniel Sutcliffe wrote: > > OK, so this is the key question: these "application errors", what > > signifies them? Because, as far as I am concerned there is no > > standard way of reporting Web application errors - HTTP error > > codes certainly normally have little relation to Web application > > errors. Federico Toledo wrote: > In my case, for example, I want to receive a page with the message > "Facturaci=F3n completa" so I use a Load response info ... and compare > the string. That is the normal/good way to validate a Web transaction... However, often doing this causes problems because specific Web applications may be very dynamic in the HTML they create and trying to match to a DOM identifier will fail even though the transaction was successful. The alternative is to not use a DOM identifier, to dump the whole body, and then to use ~LOCATE, ~EXTRACT, etc. to pull out your text. The problem with this can be that you are limited to the first 64K of content... > But if the load response info fails because not match with the page > I can't know what happened. I could save a log... This is the code fragment I provided earlier. > but I want to save the HTML page in the case that fail the load > response info ... or fail the test case, and then, after of a > execution, I could see the pages returned in each error. The real problem you face with writing out a complete file after a=20 script action is that of doing file IO from SCL. The problems that exist in this area are mostly related to the potential distributed nature of Script execution. > This facilitates the concurrence test more than the stress test. > Ok? I think it is acknowledged that even in stress/load testing you need to know at what point things start to fail ... > Understand my intention? I totally understand your intention, and think I always have. I'm still not sure of what you would like to implement though. > > The Modeler can already use an ODBC data source to populate an > > FVR file - is this the area you would like to extend? Or would > > you want to integrate into the Commander, or just be another > > seperate tool? >=20 > ok, but the functionality that you say in the Modeler populate an > FVR at the moment that we create the variable. After an execution > we can not update the contenent of a FVR quickly and easily. OK, I think the "right" initial way forward is to expand the Modeler's existing facilities to make it possible to update the FVR contents at any time in the most flexible manner. Warning: there are bugs associated with the whole FVR updating and usage area. Check the bug database. > Now I use my tool as a separate tool, but could be good that all be > together in the same toolset, perhaps as a part of the Commander. An important part of any features you provide in this area should be a command line interface. Many people run their tests in "batch mode" where they are not interested in the GUI at all - this type of usage is a key element of OpenSTAs future goals. Cheers /dan --=20 Daniel Sutcliffe <Da...@Op...> OpenSTA part-time caretaker - http://OpenSTA.org/ |
|
From: Federico T. <ft...@fi...> - 2006-01-13 12:38:35
|
> That is the normal/good way to validate a Web transaction... > > However, often doing this causes problems because specific Web > applications may be very dynamic in the HTML they create and trying > to match to a DOM identifier will fail even though the transaction > was successful. > > The alternative is to not use a DOM identifier, to dump the whole > body, and then to use ~LOCATE, ~EXTRACT, etc. to pull out your text. > The problem with this can be that you are limited to the first 64K of > content... > Perfect, an other good issue, but it is not what I want. Let me say that the system that I tested was made with autogenerate code. It has a lot of HTTP trafic and parameters. Some times, I made chages to the original script to parametrize, but this didn't work. Or perhaps it works fine to one VU but not with 2 or more VU. Using BView we can debug a lot of the script running alone. I want to a facilitie to debug a script that has problems to run with two or more VU. I want to see the pages, complete html pages, that the server return with error (aplication errors), that dont pass the test case. > > > The Modeler can already use an ODBC data source to populate an > > > FVR file - is this the area you would like to extend? Or would > > > you want to integrate into the Commander, or just be another > > > seperate tool? > >=3D20 > > ok, but the functionality that you say in the Modeler populate an > > FVR at the moment that we create the variable. After an execution > > we can not update the contenent of a FVR quickly and easily. > > OK, I think the "right" initial way forward is to expand the > Modeler's existing facilities to make it possible to update the FVR > contents at any time in the most flexible manner. > I'd like to expand the Commander with this facilitie, because I want to update all the variables existing in all the scripts contained in a Test. Previusly we could set a couple of parameters, and then just with a click update all the FVRs correspondig. > Warning: there are bugs associated with the whole FVR updating and > usage area. Check the bug database. I know the bugs, and I had to use the workarrounds > > > Now I use my tool as a separate tool, but could be good that all be > > together in the same toolset, perhaps as a part of the Commander. > > An important part of any features you provide in this area should be > a command line interface. Many people run their tests in "batch > mode" where they are not interested in the GUI at all - this type > of usage is a key element of OpenSTAs future goals. > Ok, thanks saludos! fedeFede |
|
From: Daniel S. <da...@Op...> - 2006-01-16 03:57:05
|
Federico Toledo wrote: > I want to see the pages, complete html pages, that the server > return with error (aplication errors), that don't pass the test > case. I understand that you want to save out the complete HTML files on application errors, and you understand that spotting application errors is something you will need to hand code for the SCL code for. To try and say this absolutely clearly: you want to be able to write downloaded HTTP reponses (Web pages) to individual files. To do this you need to be able to write data from SCL to a file ... What you then need to understand is that this basic capability is supposed to be part of SCL but has always been broken, bug# 730311: http://sourceforge.net/tracker/index.php?func=detail&aid=730311&group_id=10857&atid=110857 The reason this has never been fixed is that apparently it isn't easy, I understand this has lots to do with the distributed nature of OpenSTAs replay capabilities. I guess what I'm trying to say is, that to achieve your goals, what you really need to do is to fix the above bug... ;-) > > > > The Modeler can already use an ODBC data source to populate > > > > an FVR file - is this the area you would like to extend? Or > > > > would you want to integrate into the Commander, or just be > > > > another seperate tool? > > > > > > ok, but the functionality that you say in the Modeler populate > > > an FVR at the moment that we create the variable. After an > > > execution we can not update the contenent of a FVR quickly and > > > easily. > > > > OK, I think the "right" initial way forward is to expand the > > Modeler's existing facilities to make it possible to update the > > FVR contents at any time in the most flexible manner. > > I'd like to expand the Commander with this facility, because I want > to update all the variables existing in all the scripts contained > in a Test. Previusly we could set a couple of parameters, and then > just with a click update all the FVRs corresponding. I agree that the correct long term goal is to be able to manage your "Test Data" from within the Commander. However, there are currently no features within Commander for dealing with FVR files, whereas the Modeler already has the start of features for doing this ... I suggest that improvements to the Modeler's current features would be a great first step and more importantly something that could go into the "stable" 1.4 series for all to use in the near future. Further features could be well thought through and put into a much improved Commander in the 1.5 developer release series. It's your time and effort though, I can only encourage you to use it in ways that will help the OpenSTA community for the best in the shortest time. Cheers /dan -- Daniel Sutcliffe <Da...@Op...> OpenSTA part-time caretaker - http://OpenSTA.org/ |
Thanks for helping keep SourceForge clean.
X