Re: [FreeWRL-develop] Testing > file compare

[doug s. <hig...@ho...>]

>
>>> 3. Compare - perl script that compares Record vs Playback snapshots for a given test file
>>> -- there might be several snapshots for a given test file run, named by the frame#
>>> -- calls the platform file compare function
>>> --- in ms windows its fc file1 file2
>> .
>> The perl I have on my windows XP has compare(f1,f2)
>> So this works with text and binary files:
>> #!/usr/bin/env perl
>> use File::Compare;
>> if (compare("file1.rgb","file2.rgb") == 0) {
>> print "They're equal\n";
>> }else{
>> print "They're different\n";
>> }
>> -Doug
>> more...
>> -IIRC I downloaded the Activestate perl from http://www.perl.org/get.html
> .
> 2 screen snapshots of the same scene 1.wrl that _should_ be equal are different, and fc shows the differences spread out. (I hid the statusbar with its frames-per-second readout, by mousing over it, which brings up the menu bar). Hypotheses for the diff:
> H0: something about my video board - a few bad pixels and I have the window in a different place
> H1: floating point computations round differently depending how many frames have been rendered
> H2: something in the .bmp header
> If it's a small difference like 1/256 (1 bit out of a pixel's color value) perhaps a special compare function will be needed.
.
I downloaded GraphicsMagick from http://www.graphicsmagick.org (a forked & advanced project of ImageMagick) and ran the image compare function http://www.graphicsmagick.org/compare.html as follows on my 2 screen shots:
.
gm compare -metric mse freewrl_snapshot_A.bmp freewrl_snapshot_C.bmp
pause
gm compare freewrl_snapshot_A.bmp freewrl_snapshot_C.bmp -file diff.bmp -highlight-color purple -highlight-style Assign
gm convert diff.bmp win:
pause
.
The first compare shows a statistical summary:
Image Difference (MeanSquaredError):
           Normalized    Absolute
          ============  ==========
     Red: 0.0000000000        0.0
   Green: 0.0000000000        0.0
    Blue: 0.0000010175        0.0
   Total: 0.0000003392        0.0
.
The second compare generates an image with any different pixels colored purple. Here's the diff image converted to png and compressed - you'll see several purple pixels around the ! and checkmark icons on the menu bar:
http://dug9.users.sourceforge.net/web3d/diff.png
.
I had played with those two menu options during one of the runs. Hypothesis: it renders a bit different between original and post-activation on the menu bar.
.
Summary: looks like the main graphics window renders the same -no random floating point rounding differences in opengl- and differences are due to different user activity during one run, as seen by GraphicsMagic compare on uncompressed snapshots.
.
Conclusion: we should be able to do software testing based on snapshot differences.
-Doug