Menu
▾
▴
Re: [sensor-dev] Results of very simple performance test + reported Jamon values are false
|
From: Jeroen B. <jbo...@xe...> - 2006-03-24 12:06:18
|
Hi Age,
Okay, I've installed the svn Eclipse plugin and checked out Sensor and
achieved to get it running in Eclipse. :-)
I run the jUnit test in Eclipse of SimplePerformanceTest. The second
time I tested it, I got:
Total time (parent timer): 937 ms.
Number of hits (test timer): 1000000.
Total time (test timer) : 125 ms.
Average time (test timer) : 0 ms.
So, this is even better than JAMon!
1.067.000 calls/s.
Not bad!
How much time did it take on your laptop?
If it is so much slower for you, I would consider switching to another OS!
Jeroen.
Age Mooy wrote:
>I ran the Jamon performance test (twice) as it was specified on the
>Jamon page. I took the two "Full Factory" results and looked at the
>same number you looked at (the "it took" number. But on my Inspiron
>9300 powerhouse laptop it consistently took a full 15000 (!) ms !
>That's 10 times as long as your test... so something is really weird
>here.
>
>This could theoreticaly be a hard peformance difference between linux
>(me) and windows (tou) but it's very hard to believe.
>
>But we can test this. I checked in a special unit test called
>SimplePerformanceTest last night. It's excluded from the normal Maven
>unit test list but you can run it seperately with Maven or with
>eclipse. I ran it with eclipse and got the same performance as the
>Jamon 2 tests on my machine. It prints some simple lines to System.out
>after running the 100000 iterations.
>
>Could you please run the SimplePerformance test on your machine and
>see what you get ?
>
>Age
>
>
>On 3/20/06, Jeroen Borgers <jbo...@xe...> wrote:
>
>
>>Hi Age,
>>
>>Sounds great, but I am not convinced.
>>
>>How did you test the JAMon performance?
>>The JAMon page refers to com.jamonapi.TestClassPerformance, it also uses
>>a monitor around the test with the loop (of 1.000.000 iterations),
>>the last code block:
>> timingMon.start();
>> test.factoryMonitor();
>> log(timingMon.stop()); :
>>
>>Gives me the next output:
>>----
>>Full Factory TimingMonitor()- uses cached version so doesn't create
>>child monitors
>> Monitor mon=MonitorFactory.start('pages.admin');
>> mon.stop();
>>JAMon Label=pages.admin, Units=ms.: (Hits=2000000.0, Avg=7.0E-5,
>>Total=140.0, Min=0.0, Max=16.0, Active=0.0, Avg Active=1.0, Max
>>Active=1.0, First Access=Mon Mar 20 10:28:02 CET 2006, Last Access=Mon
>>Mar 20 10:28:05 CET 2006)
>>It took 1594 ms.
>>----
>>The JAMon label Avg=7.0E-5 is bogus because of the clock resolution as
>>you point out. However, the time all 1.000.000 iterations took is: 1594 ms.
>>This means 627.000 calls per second on my 5160 laptop with jdk 1.4.2.
>>
>>So, I would say the numbers on the JAMon page are correct (405.000
>>calls/s for jdk1.4). Agree?
>>
>>Ciao,
>>Jeroen.
>>
>>PS. I haven't had time to time Sensor yet.
>>
>>Age Mooy wrote:
>>
>>
>>
>>>Hi
>>>
>>>I did some extremely rudimentary performance overhead tests using the
>>>following code (snippet):
>>>
>>>============================
>>>
>>>parentTimer.start();
>>>
>>>for (int i = 0; i < 1000000; i++) { // or 500000
>>> timer.start();
>>> timer.stop();
>>>}
>>>
>>>parentTimer.stop();
>>>
>>>=============================
>>>
>>>I consistently got performance of around 68000 start/stop operations
>>>per second (the average time for 500000 iterations was 7.4 seconds and
>>>the average time for 1000000 iterations was 14.6 seconds)
>>>
>>>I measured the total time of the parent timer since the time
>>>resolution of the VM/OS is not accurate enough in the 0 - 10 ms range.
>>>The total time of the timer that was being started and stopped in the
>>>loop consistently reports values about 3 times as fast, which would
>>>lead to performance of about 200000 iterations per second.
>>>
>>>Funnily enough that last number is prety close to the performance
>>>listed on the Jamon 2.0 page so I ran the Jamon 2.0 performance tests
>>>and got almost exactly the same performance as Sensor. Sensor was
>>>slightly faster on a 1.4 jdk.
>>>
>>>This means that the performance listed on the Jamon 2.0 website is not
>>>correct since they did not take the VM/OS timing resolution into
>>>account.
>>>
>>>Conclusion: Sensor does not seem to have any performance problems
>>>relative to Jamon and performs similar or better than Jamon 2.0.
>>>
>>>Age
>>>N�HS^�隊X���'���u��<�ڂ�.���y�"� �*m�x%jx.j���^�קvƩ�X�jب�ȧ�� m�ݚ���v&��קv�^�+����j�Z���{az���^��h���n���)�{h���� �ا��+h�(m�����Z�� jY�w��ǥrg�y$���Oxḝn�m j��^��'����z������x%��Rz{(�ׯzZ)z�b��,���y�+��m����+-��.�ǟ�� ���+-��b�ا~��z{(�ׯzZ)er==
>>>
>>>
>>>
>>--
>>drs. Jeroen Borgers
>>Senior Consultant, SCJP, SCEA
>>Xebia IT Architects BV, Utrechtseweg 49, 1213 TL Hilversum, The Netherlands, www.xebia.com
>>office: +31(0)35-5381921, mobile: +31(0)6-30128951, jbo...@xe...
>>
>>
>>
>>
>>
--
drs. Jeroen Borgers
Senior Consultant, SCJP, SCEA
Xebia IT Architects BV, Utrechtseweg 49, 1213 TL Hilversum, The Netherlands, www.xebia.com
office: +31(0)35-5381921, mobile: +31(0)6-30128951, jbo...@xe...
|
