Re: [FreeWRL-develop] Testing update

[doug s. <hig...@ho...>]

>> That's why I use the graphicsMagick mean square error measure.
> Ah, btw, I noticed that when running the test in the console is best if
> I you're not interacting physically with it as it seems to generate
> different images here and there.

I noticed that too, on linux, and wondered if the mouse cursor steals a bit, and it maybe it shows up as the low bit. If so, then can you move the mouse cursor off before hitting the 'x' snapshot? The graphicsmagick mse measure shows 0.0 if its just a few dozen low order bits.

>>
>> 64 BIT VS 32 BIT, INTEL VS MOTOROLA
> I mentioned before I was having trouble opening the .bmp files being
> generated. Not even graphicsmagic would do the trick.
> But honestly I was not really interested in opening them, the point was
> just to prove that one could generate the fixtures and test them for
> comparison even if restricted to the same machine they were being
> generated in.

If you -or developers who use your method- switch to the graphicsmagick mse you'll need a proper image file format with a header (to give gm the height x width etc).

>
> As a side note, my headerless bmp files differ between console snapshots
> and remote session snapshots.

Interesting. I wonder what the difference is.

> Doug, you were right when you said in reply to John that "This type of
> testing won't say if something is right or wrong, only if something
> changed."
> This is just black box testing and it's far away from any form of unit
> testing.... but that said, I do think it's better than nothing.
> My suggestion for an approach going forward is the following:
>
> To pick the most iconic .wrl/x3d file in terms of testing features (
> and committed or kept somewhere for reference.
> ...
> Then it would be a matter of running the test script that would go about
> picking each .fwplay in playback mode, generating the snapshots and in
> the end compare them.

OK sounds generally good, although I don't have any comprehensive/iconic scenes prepared - mostly hundreds of small test files (which I need to spend days to sort through to find good, orthogonal ones).
And we do make improvements. Different developers will have their own special test scenes. So developers need to have a convenient way to locally re-generate fixtures for their special/private scenes.