Re: [Physicsbench-general] game physics benchmarks (ISCA paper)

[Thomas Y. <to...@CS...>]

Hi Kenneth,

   You are asking good questions, so I am forwarding this thread to the 
physicsbench mailing list. Feel free to sign up so future Q/A can be 
searched by others.

On Thu, 9 Aug 2007, Kenneth Hoste wrote:

> Hi Thomas,
>
> On 09 Aug 2007, at 04:00, Thomas Yeh wrote:
>
>> Hi Kenneth,
>>
>>  ODE is a physics simulation library. All benchmarks uses this library
>> for physics simulation. An analogy would be when multiple games uses the 
>> same game engine. Would these be considered separate apps or one app with 
>> different inputs? It depends on your preference.
>
> Ok, I see. So they do count as seperate benchmarks I guess.
>
>> Well, it depends on what you're trying to accomplish. It may not be very 
>> effective to try to evaluate all domains that happen to have benchmark 
>> suites if there is no underlying theme.
>
> We are comparing benchmarks from different domains to see if there are any 
> common behaviors. If a benchmark suite for say bioinformatics has very 
> similar dynamic behavior (in terms of stressing the processor, not in terms 
> of the type of work it performs) to other benchmarks, then there's no need to 
> incorporate that suite into a processor design cycle. Adding it would only 
> add to simulation time, without gaining additional insights into the 
> shortcomings of some design.

Ok. I see.
Some additional domain specific benchmarks would be netbench and biobench.

>
> Some additional questions on PhysicsBench:
>
> - The inputs seem rather small; the largest benchmark yields 4*10^9 dynamic 
> instructions, which is rather low (the largest PSEC CPU2006 benchmark yields 
> 9000*10^9 instructions).  I tried chaning the number of frames, by specifying 
> "./benchmark 0.01 1 100" for example, but that resulted in Segmentation 
> faults for some of the benchmarks. Have you tried working with larger inputs? 
> Which other parameters could I change to increase the workload?

Can you post the instruction count that you are seeing for 
different benchmarks?

The released version is a serial version that was created for the public 
release to be platform independent. So the specific stats may vary 
slightly from the ISCA paper. Also, you should track stats at a per frame 
boundary (afters skipping the first 3 warm-up frames).

As far as increasing the workload, the problem size for these benchmarks 
are actually setup appropriately based on the info in the paper.
PhysicsBench can not be compared to SPEC, in terms of dynamic 
instructions, due to its real-time nature.

>
> - Are the runs deterministic, or is there some form of randomness involved?
>

ODE by default may have some randomness, but the released version of 
PhysicsBench should be deterministic.

> greetings,
>
> Kenneth
>
>> 
>> Tom
>> 
>> On Wed, 8 Aug 2007, Kenneth Hoste wrote:
>> 
>>> Hi Tom,
>>> 
>>> I've been looking into your PhysicsBench benchmark suite, and I have some 
>>> questions.
>>> 
>>> It seems to me that the benchmarks you describe in the ISCA paper (per, 
>>> rag, ..., mix) are actually inputs for one particular 'program', i.e. ODE. 
>>> Is that correct? Can ODE be viewed as one single application, with each of 
>>> the benchmarks desribed as inputs to that application?
>>> 
>>> Are you aware of any other domain-specific benchmark suites? I know about 
>>> BioPerf (bioinformatics), BioMetricsWorkload (biometrics) and MediaBench 
>>> II (multimedia workloads), but maybe you know some others too? I'd like to 
>>> expand my set of benchmarks as much as possible for future work...
>>> 
>>> greetings,
>>> 
>>> Kenneth
>>> 
>>> On 01 Aug 2007, at 05:21, Thomas Yeh wrote:
>>> 
>>>> Hi Kenneth,
>>>> 
>>>> Thanks for your interest. I have released a version of PhysicsBench at 
>>>> https://sourceforge.net/projects/physicsbench/. Feel free to check it 
>>>> out.
>>>> Please refer to our ISCA 2007 paper if the benchmarks will be used for 
>>>> published work.
>>>> Tom Yeh
>>>> On Fri, 29 Jun 2007, Kenneth Hoste wrote:
>>>>> Hello,
>>>>> Are there any plan to make the physics benchmarks decribed in this years 
>>>>> ISCA paper freely available for the community?
>>>>> I'm pretty familiar with available benchmarks, and I'm sure there's no 
>>>>> decent (free) benchmark suite out there concerning game workloads...
>>>>> greetings,
>>>>> Kenneth
>>>>> -- 
>>>>> Computer Science is no more about computers than astronomy is about 
>>>>> telescopes. (E. W. Dijkstra)
>>>>> Kenneth Hoste
>>>>> ELIS - Ghent University
>>>>> email: ken...@el...
>>>>> blog: http://www.elis.ugent.be/~kehoste/blog
>>>>> website: http://www.elis.ugent.be/~kehoste
>>> 
>>> -- 
>>> 
>>> Computer Science is no more about computers than astronomy is about 
>>> telescopes. (E. W. Dijkstra)
>>> 
>>> Kenneth Hoste
>>> ELIS - Ghent University
>>> email: ken...@el...
>>> blog: http://www.elis.ugent.be/~kehoste/blog
>>> website: http://www.elis.ugent.be/~kehoste
>
> -- 
>
> Computer Science is no more about computers than astronomy is about 
> telescopes. (E. W. Dijkstra)
>
> Kenneth Hoste
> ELIS - Ghent University
> email: ken...@el...
> blog: http://www.elis.ugent.be/~kehoste/blog
> website: http://www.elis.ugent.be/~kehoste