[OpenSTA-devel] [ opensta-Bugs-1013510 ] Result files may become corrupted due to bad test closedow

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Bugs item #1013510, was opened at 2004-08-21 15:44
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1013510&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Results Analysis
Group: Fatal Error
Status: Open
Resolution: Accepted
Priority: 6
Private: No
Submitted By: Daniel Sutcliffe (dansut)
Assigned to: Daniel Sutcliffe (dansut)
>Summary: Result files may become corrupted due to bad test closedown

Initial Comment:
A large result set from a test may cause issues when
trying to view graphs (the results analysis phase). 
When this happens the results graphs are empty or an
"Out of memory" can be shown.  This issue seems to
occur more with the graphs that are "v Elapsed Time"
but not exclusively so.  All reports have been with
large result sets.

The issues also may not occur the first time you view
your results but once it has happened it will not go
away on its own.  Deleting the HttpStats.cache and
Combined_HTTPDataFile.dat will cause your results to be
analysed again and may help avoid the problem in your
specific instance.

This problem seems to happen more if your test did not
finish cleanly, either through another problem or
through user aborting during the run.  Monitoring
results whilst the test is running also seems to
increase the chance of this happening.

A sure sign that you have a problem is the TestPlugin
process using up large amounts of memory or CPU.  If an
"Out of memory" condition is reported it will come from
this TestPlugin process.

Please add any more useful information to this bug
report.  We are specifically interested in a simple
consistent way of reproducing this error and/or small
resultsets which are known to cause the error.

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2007-08-22 19:33

Message:
Logged In: YES 
user_id=19748
Originator: YES

Jerome's workaround patch, which abandons result file parsing if
corruption
found, has been committed into CVS to be included in the 1.4.4 general
release.

HttpResultsDLL.dll: 1.4.4.2
OpenSTA: 1.4.4.11

This bug will not be closed or marked fixed though - it will serve as a 
marker in the hope we can fix/improve the source of the file corruption
in the future.

----------------------------------------------------------------------

Comment By: Jerome Delamarche (trickyjer)
Date: 2006-12-12 03:17

Message:
Logged In: YES 
user_id=681140
Originator: NO

I released on small tool (ostarepair.exe) that can be used in batch mode
to fix corrupted result files. A patch has been provided to fix the data
automatically in the Commander.

ostarepair.exe is available at http://www.trickytools.com/php/opensta.php

----------------------------------------------------------------------

Comment By: Jerome Delamarche (trickyjer)
Date: 2006-10-04 17:18

Message:
Logged In: YES 
user_id=681140

The "out of memory" message can be generated by a 
corrupted data file when the running test has 
been "killed" instead of being "stopped".

This can be easily fixed:
- either with a batch command soon available on my site 
http://www.trickytools.com)
- either by fixing the bug in OpenSTA itself. But to do 
so, I need some information: is there some documentation 
about OpenSTA design ?

Jerome

----------------------------------------------------------------------

Comment By: Danny Faught (faught)
Date: 2006-04-28 19:02

Message:
Logged In: YES 
user_id=576825

I can easily reproduce this error in v. 1.4.3.20 by
corrupting the contents of HttpStats.cache.

1. I captured a single web page hit, and built a test with
the resulting script and a cpu utilization collector for the
local host.
2. I ran the test.
3. In the results directory, I chopped the last byte off of
the HttpStats.cache file.  It doesn't matter if the
Commander is running when I do this.
4. In the Results tab for the test, as soon as I try to
expand the tree for the test run, I get a non-fatal "Out of
memory" error dialog from TestPlugin.  I don't know if it's
actually out of memory - the error is coming very quickly,
so it may be just a wild integer value asking for an
unreasonably large single block of memory.

Whether the HttpStats.cache file is really getting corrupted
by something is another matter, but this proves that the
tool is not robust against unexpected contents in the test
results.

I also have a reproducible crash that seems related to this
where I didn't intentionally corrupt the HttpStats.cache
file.  I've had three different real-world test runs that
produced results directories that give me a fatal crash
every time I try to expand the results in the tree. 
Removing HttpStats.cache fixes the problem.  I can try to
scrub the file sufficiently for my client to allow me to
send it in if that would help.

----------------------------------------------------------------------

Comment By: chris mead (chrisrmead)
Date: 2005-08-25 12:29

Message:
Logged In: YES 
user_id=1335128

For me, there is no clear correlation between large result sets 
and the crash.  I get the crash repeatedly with a test that only 
has 1060 rows in the Data List, but I get no crash with a test 
that has 50,856 rows, for instance.  

I have a ZIP to attach, but I don't see how to do it yet, maybe 
a comment is required first?

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2005-06-09 11:51

Message:
Logged In: YES 
user_id=19748

>From deleted bug#1216593 by javier_dba:
Out of memory error appears while trying to show
"HTTP Responses v Elapsed time".
Test was done successfully ramping 400 vusers along 1
hour.
Attached a zip file ResultsAnalysisCrash.zip containing:
- picture with the error.
- test file
- script file
- output from dir command of Repository folder
- zipped Scripts folder from Repository
- zipped ObjectCode folder from Repository
- zipped ArchMgr folder from Repository
- zipped Profiles folder from Repository

Ten computers generating load, sharing the repository, one
commanding the load. Used OpenSTA 1.4.2 and 1.4.3. Both
versions have the same behaviour "out of memory".

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2004-12-13 07:48

Message:
Logged In: YES 
user_id=19748

The results analysis phase has multiple problems -
especially when
dealing with large result sets.  Instead of filing multiple
bug reports
please add any useful and concrete information here that may
help
us get to the bottom of this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=1013510&group_id=10857

[OpenSTA-devel] [ opensta-Bugs-1013510 ] Result files may become corrupted due to bad test closedow

[OpenSTA-devel] [ opensta-Bugs-1013510 ] Result files may become corrupted due to bad test closedown