Menu
▾
▴
freewrl-develop
|
From: doug s. <hig...@ho...> - 2013-05-29 20:57:59
|
>>>> >>>> I'd like FREEWRL_STEREO_RENDERING to be defined in my linux build. I can hack the .c code to define it, and I see sidebyside and anaglyph rendering work well, toggled by the statsbarHud options page. I know little about the linux build system. I did do ./configure --with-statusbar=hud and that worked. >>>> Q. How do I do define FREEWRL_STEREO_RENDERING during the linux build process? >>>> >>>> >>>> Apparently, FREEWRL_STEREO_RENDERING, is only defined in these files: >>>> >>>> projectfiles_vc10/config.h >>>> 273:#define FREEWRL_STEREO_RENDERING 1 >>>> >>>> projectfiles_vc9/config.h >>>> 273:#define FREEWRL_STEREO_RENDERING 1 >>>> >>>> Which leads me to believe that whoever implemented stereo never >>>> implemented the definition of the variable for the linux build system. >>>> To achieve that, one needs to create that variable. >>>> >>>> something like this in configure.ac should do the trick. >>>> >>>> ------ Cut here----- >>>> AC_MY_DEFINE_SWITCH([stereo],[enable stereo >>>> (default=off)],off,FREEWRL_STEREO_RENDERING) >>>> AM_CONDITIONAL([FREEWRL_STEREO_RENDERING],[test "$sw_stereo" != no]) >>>> --------------- >>>> >>>> After these lines have been added to configure.ac, run autoreconf and >>>> afterwards ./configure --help should give you an option which starts >>>> with "--enable-stereo". >>> Yes - works, and checked in. Thanks Paulo. >>> -Doug >> >> >> Curious -- is there any particular need for this to ever be off? Or are >> we using compile-time #ifdefs to do something that should probably be >> configured at runtime? . OK I made it build stereo by default, and took out the configure.ac option. . >> > I'm OK if it's built in / no ifdefs or .configure needed. Shall I? > -Doug > more.. > It's always built in win32. I think during development there was some distaste for it being in the code, or perhaps creating liabilities if someone's eyes going crosseyed as a result. > The shutterglasses option just wiggles around harmlessly. The sidebyside needs a screen bigger than your eyebase to be usable, but does no harm on a small screen. The Blackberry playbook Qnx tablet implementation does side-by-side nicely. The analgyph should work no matter what the screen size, although John was saying he found it fiddly on Android. I don't know if there are keyboard commands for all 3 modes, but if the .configure --with-statsubar=hud is used, there's an options button/panel that allows you to toggle / choose stereo mode and set any parameters, and toggle it off when done. So no command line option is needed. I think there are 2 more if()s it has to check per loop if you're not using it and its compiled in. > I was honoring the historical #ifdefs by other developers who found the whole stereovision thing distasteful and wanted no part of it. I've seen that reaction before when I gave a seminar on a virtual tour I did of an historical townsite, given to a museum association at their annual meeting. I passed out anaglyph glasses with the DVD and papers. After the class, all the papers and DVDs were taken, but a few anaglyph glasses were left behind. I suspect that people who can't see stereo find it distasteful or humiliating. Which I respect. > But perhaps the code now -in it's much cleaner, proven state, with fiducials and eyebase control- perhaps its acceptable to all for default inclusion. In that case, yes we can build it in by default. > Summary: it doesn't hurt to always compile it in. Then if you don't want to use it, don't. |
|
From: doug s. <hig...@ho...> - 2013-05-29 21:28:48
|
If I -R record an .fwplay file on win32, and -F or -P playback on linux, it doesn't work quite the same, for example linux freewrl doesn't quit on the win32 q keystroke. win32 keystrokes: 14 58111.046000 "X,2,x,1,X,3," "" 24 58121.687000 "Q,2,q,1,Q,3," "" linux keystrokes: 24 1369782152.529543 "x,2,x,3," "" 29 1369782155.037159 "q,2,q,3," "" . In win32, the 2 means raw down, and 3 means raw up, and 1 means shift-applied down+up complete keystroke. In linux 2 means shift-applied down, and 3 means shift-applied up. We don't get a 1. . It seems like a good goal to generate .fwplay files on one platform, and then send them to others to automatically replay them on other platforms. That way the work of manually running the program and manually moving the mouse around and clicking, and typing characters, is only done once. After that it's automatic. . so somehow need to get the two into a normalized form before recording/playing back . -Doug . A few other issues: a) dtime - in win32 the dtime variable can be float with 6 decimal places. - in linux, it looks like time will get rounded. So the time needs to be read-back after sprintf and applied to dtime if doing -R -F together. Maybe that was the cause of the small diff I saw before - slightly different time delta and resulting animation progress. b) statusbar - ./configure --with-statusbar=hud vs no statusbar - might need different /recording set |
|
From: doug s. <hig...@ho...> - 2013-05-30 00:16:55
|
>for example linux freewrl doesn't quit on the win32 q keystroke. > win32 keystrokes: > 14 58111.046000 "X,2,x,1,X,3," "" > 24 58121.687000 "Q,2,q,1,Q,3," "" > linux keystrokes: > 24 1369782152.529543 "x,2,x,3," "" > 29 1369782155.037159 "q,2,q,3," "" > . . I modified linux to give also a 1 ie when it gets a 3, it spits out a 1 first. Then made it so linux and win32 both act on the 1 key (the 2, and 3 are used for a Node called KeySensor) . > . > A few other issues: > a) dtime > - in win32 the dtime variable can be float with 6 decimal places. > - in linux, it looks like time will get rounded. So the time needs to be > read-back after sprintf and applied to dtime if doing -R -F together. Maybe that was > the cause of the small diff I saw before - slightly different time delta > and resulting animation progress. dtime DONE/FIXED . > b) statusbar > - ./configure --with-statusbar=hud vs no statusbar - might need different /recording set |
|
From: doug s. <hig...@ho...> - 2013-05-31 23:32:39
|
> >>for example linux freewrl doesn't quit on the win32 q keystroke. >> win32 keystrokes: >> 14 58111.046000 "X,2,x,1,X,3," "" >> 24 58121.687000 "Q,2,q,1,Q,3," "" >> linux keystrokes: >> 24 1369782152.529543 "x,2,x,3," "" >> 29 1369782155.037159 "q,2,q,3," "" >> . > . > I modified linux to give also a 1 ie when it gets a 3, it spits out a 1 first. > Then made it so linux and win32 both act on the 1 key (the 2, and 3 are used for a Node called KeySensor) > . I changed the .fwplay recording of keystrokes to integers (from char) so I can transmit control characters, And I moved the platform2web3dActionKey(key) mapping function earlier, and specialized it for LINUX and WIN32 for a bit finer granularity control to accommodate platform differences better -such as the DEL key which on win32 needs to be explicitly inserted into the transformed character stream for actions in StringSensor node, but on linux is a routine character- and so the special keys recorded in the .fwplay file can be played back on a different platform with the same meaning. . 10 30662.796000 "32,12,72,2,72,1," "" 11 30663.296000 "72,3,32,13," "" 12 30663.796000 "79,2,111,1,79,3," "" 13 30664.296000 "87,2,119,1,87,3," "" 14 30664.796000 "68,2,100,1,68,3,89,2,121,1," "" 15 30665.296000 "89,3," "" 16 30665.796000 "" "" 17 30666.296000 "" "" 18 30666.796000 "" "" 19 30667.296000 "" "" 20 30667.796000 "65535,12,65535,1,65535,13," "" The 65535 is 0xFFFF and stands for DEL, and the 12 means raw-action-key-down, 13 up (whereas regular raw keys get a 2 down, 3 up, and 1= translated (modifiers like shift applied) down+up keystroke. . Win32 can now use the numpad in fly mode, as can any system that identifies its numpad key scancodes and does the transform to NUM0-NUM9,NUMDEC which I've defined as an extension along with the web3d actionKey codes for KeySensor nodes http://www.web3d.org/files/specifications/19775-1/V3.2/Part01/components/keyboard.html#Nodereference . I was able to test on Ubuntu linux and win32 XP. I don't own a Mac, and mobile devices are untested, but I tried very hard to leave things working the same assuming an app calls fwl_do_keyPress(char). The modified linux -using xlib handle_XEvents in Mainloop.c- and win32 call a new function fwl_do_rawKeyPress(int key, int type) directly, to achieve the better control and platform-specific platform2web3d lookup. A few things to check if you rebuild OSX: 1. Fly mode 2. KeySensor and StringSensor - here's some tests -use the *All.x3d except in the StringSensor set allowDeletion='true' to test the DEL key. http://dug9.users.sourceforge.net/web3d/tests/KeySensor/KeySensorTests.zip -Doug |
|
From: doug s. <hig...@ho...> - 2013-06-01 19:57:08
|
made the following change: For testing only, and only when there's a KeySensor or StringSensor, freewrl will exit with ESC key. -Doug more... When doing automated testing on a scene with a KeySensor node, we need a .fwplay recordable way to exit freewrl, and the 'q' is swallowed by the KeySensor or StringSensor node. 'q' is a key often needed when typing strings for a stringsensor, so I'm reluctant to unconditionally quit on 'q'. ESC (escape, 27) key is less likely to be needed in a StringSensor node -although a KeySensor might need it. So only for automated testing with Key/StringSensor we can exit on ESC char 27. Q. should ESC be the normal way to exit? |
|
From: doug s. <hig...@ho...> - 2013-06-01 20:55:50
|
Testing> -L --logfile <file> With a non-graphical scene, like a KeySensor or StringSensor test, there are a few approaches to generating a test fixture: 1. with statusbarHud there's a ! panel, showing some console printout on the opengl window, so a snapshot of that would show a bit. 2. logfile. Freewrl has a -L --logfile <filename> option, generally saving what comes out on the console. Two logfiles could be compared: A) generate .fwplay freewrl KeySensor/KeySensorTestAll.x3d -R -N test4 generates recording/test4.fwplay: window_wxh = 600, 400 scenefile = KeySensor/KeySensorTestAll.x3d 1 52026.812000 "" "" 2 52027.312000 "" "6,0,398,153,0;" 3 52027.828000 "" "6,0,390,157,0;" ... . B) generate fixture (before your source code refactoring begins) freewrl KeySensor/KeySensorTestAll.x3d -F -N test4 -L fixture/test4.txt generates fixture/test4.txt: . resize_GL showed window updated window - leaving createwindow opengl version=3.3.0 1 52026.812000 "" "" initBrowserType isFreewrl=true isFlux=false isCortona=false shortname=FreeWRL longname=FreeWRL VRML/X3D Browser 2 52027.312000 "" "6,0,398,153,0;" 3 52027.828000 "" "6,0,390,157,0;" 4 52028.328000 "" "6,0,379,204,0;" 5 52028.828000 "" "" ... C) generate a playback after you've refactored some code freewrl KeySEnsor/KeySEnsorTestAll.x3d -P -N test4 -L playback/test4.txt generates playback/test4.txt: . resize_GL showed window updated window - leaving createwindow opengl version=3.3.0 1 52026.812000 "" "" initBrowserType isFreewrl=true isFlux=false isCortona=false shortname=FreeWRL longname=FreeWRL VRML/X3D Browser 2 52027.312000 "" "6,0,398,153,0;" 3 52027.828000 "" "6,0,390,157,0;" 4 52028.328000 "" "6,0,379,204,0;" 5 52028.828000 "" "" ... D) Then the fixture/test4.txt would be compared to the playback/test4.txt with perl compare. If equal, pass. If different fail. This will pass only if no date, time, file path or malloc pointer information shows up during the log sessions. -Doug . more.. Remember the goal with refactoring is to change the structure of the code but not its functionality - so before and after outputs should be identical to pass. So there's 4 steps to testing: A) generate automated test. In our case thats the .fwplay manually generated. B) generate a test fixture before refactoring code, by playing the automated test C) after refactoring, generate a playback file D) compare B and C output files. They should be the same if your refactoring was good. If a test fails, roll back your refactoring changes, and try again. |
|
From: doug s. <hig...@ho...> - 2013-06-02 16:52:09
|
> Testing> -L --logfile <file>
> 2. logfile. Freewrl has a -L --logfile <filename> option, generally saving what comes out on the console.>
.
I added a keyboard command in freewrl "`" toggleLogfile() to give a way to turn on/off logfile logging once running freewrl. This will help avoid a lot of system noise on startup, and bracket other verbose commands.
- "`" - its the back-single-quote I have on all my PC keyboards, on a key shared with the ~ tilde character, above the left TAB key
1. if you specify a -L --logfile <name> on the command line it uses <name> for the logfile
2. if you specify no -L and are not in testing fixture or playback mode, it writes to logfile.log in the current working directory
3. if you are in testing -F --fixture or -P --playback mode AND:
3a) you specified a -N --nametest <nametest> then it writes to /fixture/nametest.log or /playback/nametest.log OR
3b) you did not specify -N --nametest, then it writes to /fixture/1_wrl.log or /playback/1_wrl.log for 1.wrl
3c) you specify a -L <name> on the commandline, it uses that.
.
I tested on win32 and linux, and works.
But warning to other platforms: it has a slightly platform specific way to toggle off logging:
#ifdef _MSC_VER
freopen("CON","w",stdout);
#else
freopen("/dev/tty", "w", stdout);
#endif
and this hasn't been tested on all platforms.
-Doug
|
|
From: doug s. <hig...@ho...> - 2013-06-02 18:27:49
|
>
>> Testing> -L --logfile <file>
>> 2. logfile. Freewrl has a -L --logfile <filename> option, generally saving what comes out on the console.>
> .
> I added a keyboard command in freewrl "`" toggleLogfile() to give a way to turn on/off logfile logging once running freewrl. This will help avoid a lot of system noise on startup, and bracket other verbose commands.
> - "`" - its the back-single-quote I have on all my PC keyboards, on a key shared with the ~ tilde character, above the left TAB key
> 1. if you specify a -L --logfile <name> on the command line it uses <name> for the logfile
> 2. if you specify no -L and are not in testing fixture or playback mode, it writes to logfile.log in the current working directory
> 3. if you are in testing -F --fixture or -P --playback mode AND:
> 3a) you specified a -N --nametest <nametest> then it writes to /fixture/nametest.log or /playback/nametest.log OR
> 3b) you did not specify -N --nametest, then it writes to /fixture/1_wrl.log or /playback/1_wrl.log for 1.wrl
> 3c) you specify a -L <name> on the commandline, it uses that.
.
4) if it's like #3 except your test scene has a KeySensor or StringSensor, then as with ESC for exit, the grave accent "`" is caught, and used for toggling logging (since it's exactly these test scenes -with no testable opengl graphics image- that need logging)
.
> .
> I tested on win32 and linux, and works.
> But warning to other platforms: it has a slightly platform specific way to toggle off logging:
> #ifdef _MSC_VER
> freopen("CON","w",stdout);
> #else
> freopen("/dev/tty", "w", stdout);
> #endif
> and this hasn't been tested on all platforms.
|
|
From: doug s. <hig...@ho...> - 2013-06-02 23:24:15
|
>> . >> gm compare -metric mse fixture/1_wrl.0001.png playback/1_wrl.0001.png >> (shows a statistical table with 0.00000 difference) >> gm compare fixture/1_wrl.0001.png playback/1_wrl.0001.png -file diff.png -highlight-color purple -highlight-style Assign >> # win32: gm convert diff.bmp win: >> # linux: gm display diff.png >> gm display diff.png >> (shows a .png snapshot image with any different pixels highlighted in purple - no differences) >> . > . > in win32 I got a gm compare difference when I did > freewrl 1.wrl -R -F > freewrl 1.wrl -P > to generate the fixture while recording. I got a few pixels different. > . > When I did it in 3 steps: > freewrl 1.wrl -R > freewrl 1.wrl -F > freewrl 1.wrl -P > Then the fixture and playback images are identical. > . > Hypotheses: > H0: it has something to do with the cursor, which is over the window during -R > H1: random - need to do more tests to see if it happens again > . On linux, for -F --fixture and -P --playback testing modes only, I changed Snapshot.c to save to windows .bmp format which has a 58 byte header, and stores uncompressed -instead of .rgb which has no header, or .png which by default gets some compression. GraphicsMagick on linux will display it: gm display 1_wrl.0001.bmp There's no date stamp or filename stored in the .bmp header, so 2 test fixtures should not differ in the header, so perl compare should work as well as gm compare. And any differences won't be due to compression since there isn't any. -Doug |
|
From: Paulo E. C. <pau...@gm...> - 2013-06-06 10:22:13
|
On 01/06/13 21:55, doug sanden wrote: > identical to pass. So there's 4 steps to testing: > A) generate automated test. In our case thats the .fwplay manually generated. > B) generate a test fixture before refactoring code, by playing the automated test > C) after refactoring, generate a playback file > D) compare B and C output files. They should be the same if your refactoring was good. If a test fails, roll back your refactoring changes, and try again. Aren't we better off generating the testing upfront of any changes ? These could be stored as fixtures in tandem with the tests dir ? Also, they could be used as a sort of integration testing whenever any other change had been done to any other bit of the code ? It needs not only be for the potential refactoring job coming ahead ... This is all of course assuming I've understood everything which has recently been accomplished ... a) We can play any given .wrl file and generate fixtures out of them. b1) Those fixtures are replay-able in different architectures ( Win / Linux / OSX ) b2) The playing can also generate snapshots that can be used for comparison with previous snapshots ( to be validated ) Doug, Are my assertions correct ? Also, is it worth if we'd start condensing all this new testing functionality into some sort of text document to serve as documentation for future reference rather then being scattered through emails ? |
|
From: Paulo E. C. <pau...@gm...> - 2013-06-06 10:22:20
|
On 03/06/13 00:24, doug sanden wrote: > On linux, for -F --fixture and -P --playback testing modes only, I changed Snapshot.c to save to windows .bmp format which has a 58 byte header, and stores uncompressed -instead of .rgb which has no header, or .png which by default gets some compression. GraphicsMagick on linux will display it: > gm display 1_wrl.0001.bmp > There's no date stamp or filename stored in the .bmp header, so 2 test fixtures should not differ in the header, so perl compare should work as well as gm compare. And any differences won't be due to compression since there isn't any. I'll try and run some tests on this... |
|
From: doug s. <hig...@ho...> - 2013-06-06 13:15:32
|
>> identical to pass. So there's 4 steps to testing: >> A) generate automated test. In our case thats the .fwplay manually generated. >> B) generate a test fixture before refactoring code, by playing the automated test >> C) after refactoring, generate a playback file >> D) compare B and C output files. They should be the same if your refactoring was good. If a test fails, roll back your refactoring changes, and try again. > Aren't we better off generating the testing upfront of any changes ? Yes. A. Once per lifetime, and for all platforms, you generate list of tests, and walk through them generating the /recording/*.fwplay files, one per test. You .zip it and send it to other developers on other platforms B. At least once per platform with touchups after manually tested functional changes, or once before each major refactoring effort: replay the recordings with -F --fixture to /fixture files C. After each tiny refactoring change rerun with -P --playback to /playback D. and after each tiny refactoring change run (some yet to be developed) perl script that does comparisons between fixture and playback files and shows PASS or FAIL for each test > These could be stored as fixtures in tandem with the tests dir ? Yes. tests/recording, tests/fixtures tests/playback /recording the <testname>.fwplay files generated with -R --recording -N <testname> /fixtures the <testname>.000x.bmp image files and <testname>.log files generated with -F --fixture -N <testname> /playback same as /fixtures, except in a different directory Perl compare script would verify files in /fixture are the same as in /playback . > Also, they could be used as a sort of integration testing whenever any > other change had been done to any other bit of the code ? . In the following way: you may be trying to improve the functionality in one tiny area. You think/hope you aren't hurting something else. You run the tests, and the area where you are improving, the tests _should_ fail where you made noticable improvements. All other tests should pass -no changes-. If you see unrelated tests failing, then you know you have side effects. Lets say no side-effects, and some tests are failing due to noticable improvements. Then you would re-run -F --fixture on just those tests directly and positively impacted by the improvement. Then re-run the playback and comparisons, and everything should pass again. . > It needs not only be for the potential refactoring job coming ahead ... right - improvements will cause test failure, but if no side-effects just in a few tests you know should benefit. Then re-run -F on just those tests. > > This is all of course assuming I've understood everything which has > recently been accomplished ... > > a) We can play any given .wrl file and generate fixtures out of them. > b1) Those fixtures are replay-able in different architectures ( Win / > Linux / OSX ) Correct. I haven't tried OSX, and there's still lots of scenarios this doesn't cover in the wireless devices and desktop -such as the browser plugin, or SAI/EAI- but it's a very essential, core testing approach that will help developers improve the structure of the main code. > b2) The playing can also generate snapshots that can be used for > comparison with previous snapshots ( to be validated ) Correct. Perl compare can be used to compare the /fixture and /playback image and .log files > > Doug, Are my assertions correct ? Yes. > > Also, is it worth if we'd start condensing all this new testing > functionality into some sort of text document to serve as documentation > for future reference rather then being scattered through emails ? . Uhm - I think if you go freewrl --help you should get a brief reminder, except for the "`" and ESC which are internal commands I forgot to put help on - I'll work on that. I wonder where more detailed help would go. On a web page? Here's something for Users: http://freewrl.sourceforge.net/use.html but Testing is more for developers. Maybe a separate html page for testing? Also there's still some parts missing - like perl examples. |
|
From: doug s. <hig...@ho...> - 2013-06-06 13:31:54
|
> > On 03/06/13 00:24, doug sanden wrote: >> On linux, for -F --fixture and -P --playback testing modes only, I changed Snapshot.c to save to windows .bmp format which has a 58 byte header, and stores uncompressed -instead of .rgb which has no header, or .png which by default gets some compression. GraphicsMagick on linux will display it: >> gm display 1_wrl.0001.bmp >> There's no date stamp or filename stored in the .bmp header, so 2 test fixtures should not differ in the header, so perl compare should work as well as gm compare. And any differences won't be due to compression since there isn't any. > I'll try and run some tests on this... > OK. Let me know your experience and thoughts! -Doug more.. Here's a sample recording.zip - the kind that will be passed around between platforms. You would unzip it in the tests/ folder so you get tests/1.wrl tests/recording/*.fwplay http://dug.userssourceforge.net/web3d/recording.zip -there's a logging test that uses /tests/KeySensor/(...).x3d - for the tests/KeySensor folder unzip the following in /tests: http://dug9.users.sourceforge.net/web3d/tests/KeySensor/KeySensorTests.zip . Example for /recording/fly.fwplay navigate to the tests directory To generate /fixture/fly.bmp freewrl 1.wrl -F -N fly To generate /playback/fly.bmp freewrl 1.wrl -P -N fly . |
|
From: doug s. <hig...@ho...> - 2013-06-06 13:37:25
|
> http://dug.userssourceforge.net/web3d/recording.zip CORRECTION: http://dug9.users.sourceforge.net/web3d/recording.zip |
|
From: doug s. <hig...@ho...> - 2013-06-06 14:13:21
|
> Example for /recording/fly.fwplay > navigate to the tests directory > To generate /fixture/fly.bmp > freewrl 1.wrl -F -N fly > To generate /playback/fly.bmp > freewrl 1.wrl -P -N fly . Paulo, IDEA: instead of perl reading the .fwplay to get the scenefile, let freewrl read it and set its own scene file. I'll try it. -Doug more.. Notice you have to specify the .wrl file as well as the test file on the command line. The plan was that perl would read the first 2 lines of the .fwplay: window_wxh = 600, 400 scenefile = 1.wrl 1 55342.921000 "" "" ... and use the scenefile = value to set the scene file on the freewrl command line. One idea that just popped to mind 5 minutes ago: in freewrl if no scenefile is specified during -F / --fixture or -P / --playback , but a -N <nametest> is specified, then freewrl would set it's own scene file and width x height. Proposed like this: freewrl 1.wrl -R -N fly2 (generate the recording/fly2.fwplay) freewrl -F -N fly2 (read fly2.fwplay and set scene to 1.wrl) freewrl -P -N fly2 (read fly2.fwplay and set scene to 1.wrl) |
|
From: doug s. <hig...@ho...> - 2013-06-06 22:32:20
|
> >>> identical to pass. So there's 4 steps to testing: >>> A) generate automated test. In our case thats the .fwplay manually generated. >>> B) generate a test fixture before refactoring code, by playing the automated test >>> C) after refactoring, generate a playback file >>> D) compare B and C output files. They should be the same if your refactoring was good. If a test fails, roll back your refactoring changes, and try again. > . I make a change to freewrl, so now for -F / --fixture and -P / --playback it reads the name of the scenefile (ie 1.wrl) from the .fwplay (so don't put scenefile on command line for fixture and play, when using -N <nametest> option): . A) generate .fwplay freewrl 1.wrl -R -N fly2 (then hit "`" to toggle logging on, then off, hit 'x' to snapshot, hit 'q' to quit) creates: /recording/fly2.fwplay /freewrl_tmp/freewrl.snap.0001.bmp /freewrl_tmp/logfile.log . B) generate test fixture freewrl -F -N fly2 creates: /fixture/fly2.0001.bmp /fixture/fly2.log . C) generate playback freewrl -P -N fly2 creates: /playback/fly2.0001.bmp /playback/fly2.log . D) run a perl script to compare fixture and playback files . -Doug |
|
From: Paulo E. C. <pau...@gm...> - 2013-06-12 15:21:39
|
On 06/06/13 23:32, doug sanden wrote: > A) generate .fwplay > freewrl 1.wrl -R -N fly2 > (then hit "`" to toggle logging on, then off, hit 'x' to snapshot, hit 'q' to quit) > creates: > /recording/fly2.fwplay > /freewrl_tmp/freewrl.snap.0001.bmp > /freewrl_tmp/logfile.log > . The recording seems to be working well but.... "`" - I can't see any logging whatsoever as my keyboard is PT and triggering a "`" involves a combination of Shift+and another key. So perhaps I'm not triggering the logging correctly. Hitting "x" for a snapshot generates a png file and not a bmp. The movements whilst recording are slow, I noticed your note in the code regarding "reduce empty meaningless records" but I wonder if it is feasible for anyone to generate recordings with that level of slowness? If I interpreted correctly your intention, I wonder If it wouldn't be better to exclude those "empty meaningless records" at flush time or similar and still keep the fluidity of a normal app. > B) generate test fixture > freewrl -F -N fly2 > creates: > /fixture/fly2.0001.bmp > /fixture/fly2.log > . This does seem to play the recording well but ... - The bmp file created is unreadable in my system, it looks like a file with a bunch of data and no header.... - I don't see any logfile being created. > C) generate playback > freewrl -P -N fly2 > creates: > /playback/fly2.0001.bmp > /playback/fly2.log > . Same thing here unreadable bmp and no logfile. > D) run a perl script to compare fixture and playback files I think this bit will be rather straightforward to achieve once everything is oiled enough. Following on the initial code I have on github for the testing bit my plan is to have a script that drives FreeWRL and generates playbacks to then compare to the fixtures. |
|
From: doug s. <hig...@ho...> - 2013-06-12 16:37:56
|
>> A) generate .fwplay
>> freewrl 1.wrl -R -N fly2
>> (then hit "`" to toggle logging on, then off, hit 'x' to snapshot, hit 'q' to quit)
>> creates:
>> /recording/fly2.fwplay
>> /freewrl_tmp/freewrl.snap.0001.bmp
>> /freewrl_tmp/logfile.log
>> .
> The recording seems to be working well but....
> "`" - I can't see any logging whatsoever as my keyboard is PT and
> triggering a "`" involves a combination of Shift+and another key.
> So perhaps I'm not triggering the logging correctly.
Paulo,
Thanks again for recommending automated testing. We'll see if this 'input capture' method I chose works.
Sorry that was a bad character to pick - it just worked well on my keyboards. But I thought it should still work with 'modifiers' like shift, since it's the translated character I trigger on.
Having said that, the logfile.log would be empty -nothing in it.
Hypotheses:
H0: on your system closing an empty file doesn't save it?
to test that, open freewrl with 1.wrl scene and no other command line parameters, put the ` character. On the console/terminal you should see it say something about logging.
Then hit \ It should write something to the logging file. Do ` again. On the console it should say something about stopping logging. Then in /freewrl_tmp it should show logging.log.
H1: the ` character is a different numerical value
To test: pick another character you like and change it in Mainloop.c L.3355
case '`': { toggleLogfile(); break; }
If another key works that you prefer, and isn't being used for something else in that function, then email me back and I'll change it in CVS.
> Hitting "x" for a snapshot generates a png file and not a bmp.
.png is correct for -R and ordinary freewrl snapshots on linux. (example above was win32) (the .bmp on linux is only for -F, -P modes)
> The movements whilst recording are slow, I noticed your note in the code
> regarding "reduce empty meaningless records" but I wonder if it is
> feasible for anyone to generate recordings with that level of slowness?
> If I interpreted correctly your intention, I wonder If it wouldn't be
> better to exclude those "empty meaningless records" at flush time or
> similar and still keep the fluidity of a normal app.
Problem: there are no completely empty records. The timestamp keeps advancing. The fly navigation mode and various interpolators keep doing work with only another frame and timestamp. To make the input-capture method of testing work reliably I'm about 97.3% sure it needs to replay all the timestamps/frames the Record method had. (a small amount of doubt - perhaps some hacking on fly mode is all it would need)
Having said that, I just picked .5 seconds arbitrarily. The easiest change would be to make this number smaller ie .1 seconds -that would give you 10 fps, and 5x as many .fwplay records, although many of those records would be 'short'. It would also take longer to replay, but if you have a long perl automated test suite, you could sip a coffee or read email while it replays.
Test: try changing that .5 to .1 and do a -R (recording) and see if .1 is OK for you, if so let me know.
.
>> B) generate test fixture
>> freewrl -F -N fly2
>> creates:
>> /fixture/fly2.0001.bmp
>> /fixture/fly2.log
>> .
> This does seem to play the recording well but ...
> - The bmp file created is unreadable in my system, it looks like a file
> with a bunch of data and no header....
Q. .bmp is a Microsoft image file format. do you have GraphicsMagick installed? If so then at command level:
gm display fly2.0001.bmp
should show it - I've tried it on Ubuntu.
GraphicsMagick is a re-write of an older ImageMagic(k?) utility - and there's also a perl interface (perlMagick) for calling graphicsMagick functions directly (bypassing the commandline syntax).
http://www.graphicsmagick.org/
http://www.graphicsmagick.org/perl.html
(some linux gurus hate anything microsoft. And I stewed for a few days before conceding it was the easiest and maybe even the best image format choice for this testing job - no compression, small generic header, I had the C code working for win32, and -I thought- displayable on all platforms with graphicsmagick)
If you try graphicsMagick and it doesn't display, email me off list and attach the .bmp -I'll check if it displays on win32.
(perhaps I didn't do the header right for 64bit. the header is 58 bites, starting with 2 byte "BM", then a struct of 54 bytes. I'm assuming the 4 byte integers aren't padded out to 8 bytes in the struct in 64 bit. If they are then I need to change the way I malloc and write the header so it works better on 64 bit. Are you compiling for 64 bit?)
> - I don't see any logfile being created.
>> C) generate playback
>> freewrl -P -N fly2
>> creates:
>> /playback/fly2.0001.bmp
>> /playback/fly2.log
>> .
> Same thing here unreadable bmp and no logfile.
>> D) run a perl script to compare fixture and playback files
> I think this bit will be rather straightforward to achieve once
> everything is oiled enough.
> Following on the initial code I have on github for the testing bit my
> plan is to have a script that drives FreeWRL and generates playbacks to
> then compare to the fixtures.
Bravo - I have about 1 day of perl, about all I remember is she-bang = #!.
I have a tests/ folder that has a lot of tests besides the 1-52, and in sub-folders. So before generating .fwplays, I need a script to find all the .wrl and .x3d in current and subfolders, and generate a script that will run freewrl -R on each of them in sequence, so I can spend a couple of intense 'guided' hours generating .fwplays that I can then .zip and send to others, along with the expanded tests folder. How do I find all the .wrl and .x3d using perl?
-Doug
|
|
From: doug s. <hig...@ho...> - 2013-07-14 21:10:15
|
I put the command line option summary on http://freewrl.sourceforge.net/use.html page for now, but I think it needs a separate page. I've found the GraphicsMagick command: gm compare -metric mse file1 file2 very helpful, as about 20% of screen shots differ by a few pixels and up to 10% of color value on each of those pixels, more so with stretched texture maps. The mse (mean square error) returns 0.0 for these small differences, and .1 or greater for 'truly' different files. For example if I move something even slightly in a scene, mse will be .1. I find the whole system useful, and time saving, but a bit verbose, and needing lots of folders, utilities and supporting files: perl graphicsmagick tlist.txt - a list of test scene files ie 1.wrl 2.wrl .. KeySensor/KeySensorTestAll.x3d .. gen_test_scripts.pl - a perl script that reads the list of test scenes, and generates 4 shell scripts / batch files: - record_script.bat ---- interactively runs through tests so you generate .fwplay recordings - fixture_script.bat ---- automatically generates fixture .bmp/.png and/or .log files in /fixture for all tests - playback_script.bat ---- automatically generates playback .bmp/.png and/or .log files in /playback for all tests - compare_script.bat ---- calls a compare script on each test, and echoes PASS ~PASS or FAIL for each test compare.pl - complex compare script. It tries simple perl file compare. If that fails, it tries the graphicsmagick compare. if that fails the test FAILs. Else it PASSes. I just wish it was a bit less clumsy, with a simple push-button gui. In case someone wants to play with it, here's my two .pl scripts (open/MIT) and /recording/*.fwplay for tests 1-52.wrl http://dug9.users.sourceforge.net/web3d/testing/testing1.zip -Doug |
|
From: John A. S. <iva...@gm...> - 2013-07-14 21:49:29
|
Doug - very interesting that you got this working, especially as direct comparisons (Bit by bit) do not work, as you know. Don Brutzman - hope you get a chance to look at what Doug did - great for conformance tests between browsers. JohnS. |
|
From: doug s. <hig...@ho...> - 2013-07-15 20:52:52
|
> > Doug - very interesting that you got this working, especially as direct > comparisons (Bit by bit) do not work, as you know. > > Don Brutzman - hope you get a chance to look at what Doug did - great > for conformance tests between browsers. > This type of testing won't say if something is right or wrong, only if something changed. The fixtures need to be regenerated on each platform, and perhaps the recordings as well if the opengl-statusbar vs. gui-statusbar offsets the mouse too much or the window size can't be matched. -Doug more... > I just wish it was a bit less clumsy, with a simple push-button gui. I reworked it into 5 perl scripts: run_record.plrun_fixture.plrun_playback.plrun_compare.pl compare.pl http://dug9.users.sourceforge.net/web3d/testing/recording2.zip http://dug9.users.sourceforge.net/web3d/testing/testing2.zip And you put a list of (new / additional) scenes you want to test into a file with scenefilename testname on each record: tlist.txt 1.wrl t1 .. 52.x3d t104 tlist.txt is only read by run_record.pl The run_fixture, run_playback, and run_compare scan the /recording folder for the list of .fwplay to test. For 104 tests (1-52.wrl + 1-52.x3d) run_playback.pl - takes 145 seconds to run run_compare.pl - takes 35 seconds to run Total time: 3 minutes. This would be run after each small change in code. Output looks like: .. t77 PASS t78 t79 PASS t9 PASS t90 PASS t91 PASS t92 PASS t93 PASS t94 PASS t95 PASS t96 PASS t97 PASS t98 PASS t99 FAILIf something FAILS you roll back your last change and try another way.see also REFACTORING: http://en.wikipedia.org/wiki/Code_refactoring http://refactoring.com/catalog/index.html So what's missing? a) some way to echo the name of the scene file when a test fails b) some way to generate / add / remove / update / delete tests c) chaining of playback and compare d) compare summary stats |
|
From: Paulo E. C. <pau...@gm...> - 2013-07-16 14:23:09
|
Hi, I can confirm that the fixtures generated seem different per machine.... Even when testing in two similar architectures "Linux" the fixtures seemed to differ. I think this might be down to a combination of library versions, graphic cards etc ... Further, I commented the code that was writing the header file of the bmp file and that seemed to generate more stable results across runs. At least I don't have any bit differences between runs. I used a different approach though ... First I commented that time reduction hack so that I could record things smoothly. https://github.com/pecastro/freewrl/commit/4926a80062179dd1223296537eb45bf04b9a17eb Using recording mode, I manually created one master .fwplay whilst moving in the scenegraph back and forth rotating etc and in between each move taking some manual snapshots. This master file was created using the freewrl/tests/2.wrl https://github.com/pecastro/freewrl/commit/044850835cc7d56e4f6c2b52360485134620e4d3 Then I used this .fwplay file to test all of the .wrl/x3d files. I iterate over the list of .wrl files to be tested and for each one I amend the .fwplay file, and substitute the scenefile for the current file being tested. I then run freewrl in playback mode for that specific .wrl file. I kept all the headerless .bmp files generated and committed them to a local branch. Then I built freewrl from scratch and ran exactly the same procedure of running the script in playback mode and in the end it was a matter of asking my source control system if any of those .bmp files had changed. I never really used the Playback option. The code hacky has it is can be seen at https://github.com/pecastro/freewrl/tree/testing-freewrl Cheers, PECastro |
|
From: doug s. <hig...@ho...> - 2013-07-16 15:40:03
|
Paulo, > > I can confirm that the fixtures generated seem different per machine.... > Even when testing in two similar architectures "Linux" the fixtures > seemed to differ. > I think this might be down to a combination of library versions, graphic > cards etc ... I have bit differences in 20% of comparisons on a single win32 platform, but they are scattered across the image -not header related- and larger color spreads on textured objects. That's why I use the graphicsMagick mean square error measure. > Further, I commented the code that was writing the header file of the > bmp file and that seemed to generate more stable results across runs. > At least I don't have any bit differences between runs. The header issue is puzzling to me. I should have zeroed the header, and put only useful stable numbers in it. There are a few gotchas that I didn't look into: struct alignment -if a compiler pads a struct to an even 8 bytes then I write out the struct as a binary blob I'll get padding, which is bad. And intel vs motorola byte ordering - I'm not sure how to fix that. 64 BIT VS 32 BIT, INTEL VS MOTOROLA The padding can be fixed by writing each element of the struct out separately. And do I have the right data types to avoid 64bit ints? One way to tell is to try the graphicsmagick or gimp readers on each platform and see if they can read my header generated on various platforms, assuming gimp and graphicsmagick have the right idea. (Right now I have only 32bit hardware and compilers, and opengl 2 only on one machine) > > I used a different approach though ... > > First I commented that time reduction hack so that I could record things > smoothly. > https://github.com/pecastro/freewrl/commit/4926a80062179dd1223296537eb45bf04b9a17eb > > Using recording mode, I manually created one master .fwplay whilst > moving in the scenegraph back and forth rotating etc and in between each > move taking some manual snapshots. > This master file was created using the freewrl/tests/2.wrl > https://github.com/pecastro/freewrl/commit/044850835cc7d56e4f6c2b52360485134620e4d3 > > Then I used this .fwplay file to test all of the .wrl/x3d files. > I iterate over the list of .wrl files to be tested and for each one I > amend the .fwplay file, and substitute the scenefile for the current > file being tested. > I then run freewrl in playback mode for that specific .wrl file. > I kept all the headerless .bmp files generated and committed them to a > local branch. > > Then I built freewrl from scratch and ran exactly the same procedure of > running the script in playback mode and in the end it was a matter of > asking my source control system if any of those .bmp files had changed. Great idea. So summary: this method is good for checking geometry rendering changes in detail, but doesn't allow for scene-specific mouse or keyboard input -except see below^- such as clicking something. What it's good for: a) inspecting scene rendering in finer detail b) using only one fwplay, with high-fidelity avatar movements, to view each scene using the same avatar movements for each scene ^It might also be very helpful when you have one giant test scene with everything in it -or everything that's of interest for your development changes- and you use just that one scene for testing. In that case you can do clicking and keyboarding for that scene. For example we have 52 tests, and you could put them all side-by-side in a single scene file, then navigate the avatar between them during a test run. Q. if you have 100 test scenes, how long does it take to run the test on all of them? (the degraded-frame-rate, one-fwplay-per-small-scene takes 3 minutes to playback and compare 104 scenes on a 32 bit pentium) > > I never really used the Playback option. -P --playback is just -F --fixture, with a different output folder name. Comparing current fixture to source-control fixture is equivalent to comparing playback to fixture. If someone improves the code -and verifies manually that the code is correct- they should re-generate the fixtures or in your case check-in the current fixtures to source control. > > The code hacky has it is can be seen at > https://github.com/pecastro/freewrl/tree/testing-freewrl Thanks for the link. And the ideas. I wonder if the two methods can be combined somehow, perhaps as options / parameters, so developers can conveniently choose. -Doug |
|
From: doug s. <hig...@ho...> - 2013-07-16 16:41:45
|
>> Further, I commented the code that was writing the header file of the >> bmp file and that seemed to generate more stable results across runs. >> At least I don't have any bit differences between runs. > > The header issue is puzzling to me. I should have zeroed the header, and put only useful stable numbers in it. There are a few gotchas that I didn't look into: struct alignment -if a compiler pads a struct to an even 8 bytes then I write out the struct as a binary blob I'll get padding, which is bad. And intel vs motorola byte ordering - I'm not sure how to fix that. > > 64 BIT VS 32 BIT, INTEL VS MOTOROLA > When I look at other code, they have a technique for writing binary ints in a standard way: charbuf[0] = (my32bitint & 0x000000ff)>> 0x00; charbuf[1] = (my32bitint & 0x0000ff00)>> 0x08; charbuf[2] = (my32bitint & 0x00ff0000)>> 0x10; charbuf[3] = (my32bitint & 0xff000000)>> 0x18; which looks like it would put it in least to most significant digit order -little endian as wintel (windows + intel) is natively, birthplace of the .bmp file format- in the character buffer whatever the endianness of the platform, and converting to char* variable by variable would avoid 64 bit struct padding/alignment. I didn't do that. I'll put it on my to-do list. -Doug |
|
From: doug s. <hig...@ho...> - 2013-07-16 22:08:36
|
>> The header issue is puzzling to me. I should have zeroed the header, and put only useful stable numbers in it. There are a few gotchas that I didn't look into: struct alignment -if a compiler pads a struct to an even 8 bytes then I write out the struct as a binary blob I'll get padding, which is bad. And intel vs motorola byte ordering - I'm not sure how to fix that. >> >> 64 BIT VS 32 BIT, INTEL VS MOTOROLA >> > > > When I look at other code, they have a technique for writing binary ints in a standard way: > charbuf[0] = (my32bitint & 0x000000ff)>> 0x00; > charbuf[1] = (my32bitint & 0x0000ff00)>> 0x08; > charbuf[2] = (my32bitint & 0x00ff0000)>> 0x10; > charbuf[3] = (my32bitint & 0xff000000)>> 0x18; > which looks like it would put it in least to most significant digit order -little endian as wintel (windows + intel) is natively, birthplace of the .bmp file format- in the character buffer whatever the endianness of the platform, and converting to char* variable by variable would avoid 64 bit struct padding/alignment. > > I didn't do that. I'll put it on my to-do list. DONE and checked-in. The .bmp header should write properly on 64bit and big-endian computers. -D |
