Jim Kring wrote:
> One of the main issues at the moment is that when I run the testers I
> get slightly different results (primarily for the top entries). I
> have been running the tester on LV 7.0 and Windows XP. I have reduced
> the problems by disabling my network adapter and shutting down
> anti-virus and other services. However, there are still some problems.
Unfortunately I believe that you have found the limitation of the
measurement system and perhaps the top entries are equal in
performance? Since we are using the built in MS timer we must run the
code N times and average to get the actual time per iteration. N must
be chosen so that it is large enough such that the 1 msec resolution
(and overhead of reading the MS) timer becomes insignificant as compared
to the actual time per iteration. But as N gets larger, the total time
increases which gives a greater chance that the CPU may get hijacked for
another Windows task. You have minimized this by shutting down services
but there will always be something. (Don't move the mouse during testing)
If you have a DAQ card available, you could use one of the
counter/timers to get a 20MHz timestamp which solves the resolution
issue, but there is still the issue of not knowing how much of the time
for one iteration is due to reading the timer. The attached VI shows
the variability.
Christian introduced the concept of picking the fastest run and not
averaging which I can agree with as long as memory leaks and other
issues that could cause worsening performance over time are caught by
another test. If so, you could use the test harness as it is to get the
VI's that are performing equally and then we could pick the one test
that was most representative of the task and run it many times with the
hardware timer. The winner would be the one with the single fastest time?
Or since the top performers may be equal in performance, you could use
one of the other criteria as the tie break
> Would it help if I made copies of all the entries available to those
> of you who want to work on the test harness? I could password protect
> them all, first.
I've been using different versions of my own VI's so I don't need copies
for development purposes.
|
