Re: [OpenSTA-users] Performance testing with Open STA

[Bernie V. <Ber...@iP...>]

> So OpenSTA timing out could be an indication of server application
> overloading.  And if the scripts are recorded and modeled correctly,
> that could well be the point that Chris is looking for to stop putting
> more load when doing the throughput test.
>
> --Michael

Hi Michael,

I agree as long as the default timeout is appropriate given how long the 
transaction runs for a single user. I am testing an application right now 
that does complex delivery scheduling for a package delivery service where 
some transactions take 3 minutes when no one else is using the system! Given 
the work that is being done, it is completely reasonable. Forget usability 
factors, it is what it is and the users are thankful the task is automated. 
If I used a default timeout of 1 minute, an OpenSTA error would not indicate 
a server problem. This is an extreme example, but I don't have any idea what 
Chris is modeling, so I'm pointing it out.

Next, I don't know how close Chris' scripts emulate "reality". Determining 
that the system supports 10 VUs tells me absolutely nothing about it's 
capacity unless I know how a VUs work compares to "real life" users. IF a VU 
is a reasonable approximation of a real user and the system under test 
begins to timeout at 10 VUs, then it is appropriate to say that the system 
has a severe bottleneck at 10 users. I suspect in this case, performance at 
9 users was also bad. To determine a limit meaningful for capacity planning, 
you need to have emulate virtual users in a realistic way and judge capacity 
limits by looking at how response times, throughput, and server resources 
vary as load changes.

-B