Re: [FreeWRL-develop] Codebase port to c++?

[doug s. <hig...@ho...>]

 
Paulo,
 
> Your trunk should ideally always be in a buildable condition.
 
The win32 version builds from .sln /.vcproj build files.
I'll reserve comment on the linux build except to say recent activities on the win32 build could have broken something. On win32 I just build what's in freewrl/freex3d/src /lib and  /bin and /libeai.
 
> I would like to help a bit more if it was easier for me to help.
Yes and we are mostly volunteers now:
- so addressing your concerns will help us all
- we are constrained to what volunteers are interested in and willing to do
As a volunteer I do what I feel like doing, when I feel like doing it. Usually I don't have many hours or days in a row, so I prefer to set bite-sized goals and tasks, or break big ones into bite size. I'm often driven by curiosity or have a fuzzy idea and want to see if I can pull it off as a learning challenge.

> And you try to find some understanding to how it all fits together and 
> there's none or it is so scattered that it is easy to loose track of it.
 
It follows www.web3d.org specs, and in general vrml browsers work like game engines:
parse input files
fetch image resources
Loop:
- get input from mouse, keyboard
- get time delta since last frame
- run scripts based on input and time
- update geometry based on scripts, time and input
- render geometry to frame
end loop
 
 
> - Documentation briefly ( optionally as comprehensive has possible ) 
> describing the contents of each directory.
I'm not familiar with all the directories myself.
And some directories no longer adhere to their original intent - they are what I'd call fuzzy categories, not precise. To get precise I build and run and trace the code. 
If we do a big refactoring, we can perhaps refactor into better category directories in the process.
 
Here's the ones I know:
freewrl/freex3d/codegen - perl files to generate GeneratedCode.c and Structs.h node and field plumbing - only run perl after an infrequent change to the node or field structs, check in the GeneratedCode.c and Structs.h.
freewrl/freex3d/projectfiles_vc9 and _vc10 - win32 MSVC .proj and .sln build files
freewrl/freex3d/src - cross platform source code files for:
/libeai - in web3d.org specs there's something called External Authoring Interface EAI that allows scripting via port communications
/java - for SAI or Scene Authoring Interface writtin in java
/bin - main.c console app - calls lib
/libtess - used to tesselate (triangulate into mesh) truetype font polygon outlines, called by /lib
/lib - libfreewrl library called by main.c and other gui apps depending on platform
 
 
within /lib - the directories are fuzzy categories, not strict, and I have only fuzzy ideas about what the cateogry was originally meant for - so check the code to see specifically.
 
Q. If you had to decide between better documentation or better code and directory structure which would you choose?
 
The 'agile' / eXtreme programming school of thought skips the documentation and goes for good structure. A problem with detailed documentation: it can get stale, and keeping it up slows down structural refactoring. And assume for a moment know one knows exactly what a function is supposed to be doing. Then no one can properly document it. That's pretty close to where we are with freewrl.
 
On the other hand, before being able to wisely refactor code, you would need to have some fuzzy idea what it is supposed to do, perhaps through guessing or debugging sessions, and looking where and how it is used, at least enough to form an hypothesis about what it's doing. I'm used to this. 
 
The Agile school sets up lots of tests -automated, so they can run them quickly between changes- and if they break tests when refactoring, they roll back. When I do testing there are 50 or so .wrl/.x3d test files I run. 
http://dug9.users.sourceforge.net/web3d/tests/screenlinks.html 
Plus depending on what I'm tinkering with, I'll set up special .wrl/.x3d test files, and I've accumulated some samples people have sent in.
I think there are different names for different levels of tests. A Unit test is supposed to be for a small bite size module, class or even a function, I think. Functional tests are more from the outside looking in, as a user would see it. I think the 50 wrl tests would be called functional tests.
A problem with unit tests: if you're going to refactor your modules and functions, then the unit tests would need to change a lot in the process to keep up. But functional tests as seen from the outside should be stable. So if I was setting up automated tests before major refactoring I'd want functional tests, especially for freewrl which is mostly working well and free of bugs, as seen from the outside, but is "hell's kitchen" on the inside. Or perhaps unit tests could be generated from perl.
 
Having said that, some contextual and overview documentation would likely be stable and helpful to everyone. After a major refactoring, the code might be stable enough to make more refined granularity of documentation useable.
 
> - Some contextual explanation of the purpose of the codegen/ folder and 
> how to run it.
>   - Why it exists
>   - What it generates and why
>   - How it runs... it seems you're suppose to pipe something into perl 
> -MVRMLC ? Is that written somewhere ?
VRML has 2 main entity types: nodes and fields. There's a lot of repetitive plumbing for each of those types, and for each specific node and field type. When changing the system occassionally -very infrequently- if a generic field struct or node struct needs to be changed, instead of changing it repetitvely in hundreds of node or field structs, it's changed on one line in a codegen/ perl file, and the perl is re-run to generate all the repetitive plumbing code.
To re-run the perl, in windows I put it in a .bat file in /codegen, install perl somwhere, then run the .bat, which looks like this:
E:/Perl/bin/perl.exe VRMLC.pm
pause
That's it. It does it's own running of the other perl .pm files.
It outputs 
freex3d/lib/scenegraph/generatedCode.c and 
freex3d/lib/vrml/structs.h and
freex3d/libeai/GeneratedCode.c and GeneratedHeaders.c
So these output files should never be editied directly (although I have for a throw-away proof-of-concept test before editing the perl)

>   - How do you validate that it's producing the right thing ?
>     - How could one test the result of this codegen process ?
first, change them very carefully so you never break them.
second, try building and running freewrl on functional tests. If you break something, rollback your changes. Third use a diff tool to see if your changes make sense in each file.
 
> - Consider moving to at least SVN preferably Git|Mercurial
>   - CVS is old, cumbersome and hard to enable collaboration of multiple 
> people, basically everything is hard with CVS...
Agreed. Next time we change repositories. 
Q. which is easier for newbies to learn? SVN or Git? I've never used git but it seems more complicated, with local and branches and main repositories. Which do you recommend for us? Why do you use Git? Is there a Git client for all desktop platforms: osx,linux,win32? Are all git tools compatible? Does sourceforge support Git, or is Git a commercial effort in competition with sourceforge - would we be forced to move?
 
>- Roadmap
>   - Where is the project heading
>   - How much of the X3d/VRML7 spec is the project interested in 
> covering/implementing
 
There is no written or dictated roadmap, other than what volunteers/contributors want to do with it. Usually there is concensus to making it a better performing and conforming web3d.org browser and library component, or perhaps going beyond specifications to something even better, depending on the imagination and initiative of the individual volunteer contributor.
 
For example I ported freewrl to win32. Freewrl had shutterglasses type stereovision when I joined. I added side-by-side and anaglyph stereovision. I made a menu in opengl, so no platform specific GUI technology is needed, and that works on win32 and I got a varient of it working in a BB/QNX version. In win32 I made a separate launcher program, that sets up command line parameters, picks internet explorer plugin by mime type, tests a URL, and does a GoogleEarth snapshot including generating x3d file. 
 
My own current roadmap: I'm re-working parsing so it works better for nested protos.
Others are working on javascript upgrades, shader program rendering and native SAI.
 
It sounds like you are a code-structure kind of guru, so some of your wishlist and complaints could be rolled into self-initiated projects on that theme.
 
> I have extensive experience integrating different components and writing 
> Modern Perl code.
 
A funny story I heard: freewrl was originally developed in perl.
It ran too slow, and was converted to C.
But somewhere in all of its organization and structs are the seeds of its history.

> My C/C++ skills are a bit rusty but I can get along fine.
> I don't have much time in my hands but every so often I do enjoy a 
> detour into playing with something like FreeWRL.
> If every such time I could make a positive contribution albeit small I 
> supposed that everyone would win ...
 
 
> Hoping this is seen as positive feedback...
I agree -code hard to read and understand, hard to dip into it casually, and makes me feel ill in many places- so no personal offence taken. 
 
-Doug