Re: Ideas on the future of Quesa.

[Dair G. <da...@re...>]

Jose' Cruanyes wrote:

>>> that's just using inlines or reworking the internals to make the=20
>>> Quesa classes real C++ classes with inheritance and the like?
>>
>> When working on texture sharing code recently (not yet checked in) I=20
>> had to deal with dynamic arrays, and wished that I could just use=20
>> std::vector.
>
>not that I'm contrary, the reasons for the decision taken 5 years ago=20
>(not general support for STL in compilers) had been superseded,

This is definitely true - when we started (1999), STL implementations
still had plenty of annoying differences between platforms.

That's still true for some cases, but all of the common container stuff
(vector, list, map, etc) is now pretty portable (apart from
vendor-specific extensions obviously :-).


This mail has been a long time coming, but Peter's mail gave me the
nudge I needed to make a decision.

So, as I'm sure everyone has seen, for most of this year I've not really
had any time free for Quesa. Things are just too busy at work, and we
don't have any projects there that use Quesa.

It's always easier to work on this kind of project if you have a product
that actually uses it, but without that motivation you're fighting a
losing battle against a)projects that bring in your salary and b)your
non-work life.


So, I think the time has come for me to move on from Quesa - and to
leave it in the capable hands of the other developers.

I'm glad that people have found the project useful, and to be honest if
you'd asked me before we started if this project would still be going 5
years later I'd have said our chances weren't high.

But here we are, 5 years on, with shipping apps built on top of it: so I
think that has to be a success. :-)


Where we go from here is up to you guys - if you want to replace the
internals with C++ objects, go for it.

The kind of things I think are useful are:

- Bring the general side up to the final state of QD3D. Which bits are
important depends on what you use in your apps, but I think the two most
significant things are probably the remaining IO bits and visibility
culling.

- Bring the rendering side up to the current state of GL/D3D. That means
texture combines, render-to-texture, using VBOs for geometry data,
vertex and pixel shaders, etc.

Unfortunately the nature of GL means that a lot of those are exposed
through extensions that may/may not be present; but setting a base
supported OS and having some feature querying system to check what's
available would let features turn on/off as they're present.

- Don't lose abstraction unless you really need it. I'm quite pleased
with the way the internals turned out, in that we kept a very OOish
style internally (where each .c file exposes some opaque types and
functions to work on those types; so geometry code doesn't see any
details of cameras, etc).

It's sometimes necessary to break that for efficiency, but that really
is the exceptional case rather than the default. Given that computers
always get faster, and code always gets more complex, you can almost
never have too much abstraction. ;-)

- Keep the plug-in renderer idea. This is one of the nicest things about
QD3D/Quesa IMO, the idea that the organisation side of things is
completely separate from the rendering side.

The overhead of doing this is pretty minimal (call through a function
pointer), but it means you have a clean separation between Quesa and its
renderers.

- Think about changing data structures. The public QD3D types have
several limitations; vectors/colours are triples rather than quads (so
not so amenable for AltiVec processing), and the structures don't have
size fields in them.

One of the problems with the QD3D API is that, like a C++ API, it's not
good for backwards compatibility. Not saying that MS has all the
answers, but the Win32 style of having a size field at the start of each
structure at least gives you the option of adding new fields without
breaking older apps (although nowadays the Carbon approach of opaque
types and generic accessor functions is probably better).

- Look at GL/D3D. The way data (texture and geometry) is submitted to
cards has changed quite a bit since the RAVE time frame, and the key to
getting good performance nowadays is to minimise the interaction the CPU
has with the data and submit data in as large a batch as possible.

Ideally you just want to have the data in memory somewhere, give the
card a pointer to it, and let it handle the transfer itself (with a VBO
style lock/unlock to synchronise access).

- Look at some other scene graphs. We kept ourselves focused on just
doing QD3D initially, which I think was a good plan (as it meant things
didn't go off in a hundred different directions), but there are other
libraries out there which have good ideas.

Wouldn't hurt to take a look at some of them, as exposing something like
shading languages will need to be done in a similar way (no point in
inventing our own shading language, but we should have something that
lets you plug in either ARB or D3D shaders).

- Keep the history. Even if you move on to using subversion at some
point, I would recommend keeping as much of the CVS history as possible.

We don't go all the way back to 1999, which is something I always
regretted, but I think it's useful to be able always get back to older
versions. If some planned development doesn't work out, that means you
can then rewind N months and easily start again.

- Keep the licence. Switching from the LGPL to BSD solved a lot of
problems for me at work, and allowed me to use the math sections of
Quesa in our D3D->GL translation library. The GPL licences wouldn't
have.

- Think about a CoreFoundation approach. Something that's worked well
for Apple is to have a CoreFoundation library which provides basic
primitives (strings, sets, etc) and then have other libraries built on
top.

Something similar could help popularise the library and make it easier
for people to dip into. E.g., regular requests on the mac-opengl and
mac-games-dev lists are requests for file format readers and 3D math
code. So:

- 3D math library. All the standard operations, intersections, etc; you
could see this growing into more general purpose fields like collision
detection, visibility tests, or physics.
 =20
- 3D file format parser (harness that would register N plug-ins for
doing IO, the plug-ins parse their data and return it back via callbacks
with some type info.

Means that people could link to the file format library, and get called
back as objects are parsed out of the file; the object description
allows them to build their own data structure out of it. Quesa would
also link to the file format library, and have a set of callbacks that
build Quesa objects out of the data.

I know we moved the Viewer library into Quesa so we could have a single
shared library for users, but think of these more like the plug-in
renderers.

I.e., a well defined interface between them and the rest of Quesa, and
build them (and perhaps even the plug-in renderers) as static libraries
that then get linked into the Quesa shared library.

=46or this to work the interface would need to be 100% independent; i.e.,
you would need to be able to say "I just want to use the math library"
and know exactly what headers/.lib files were needed for your app (and
the math library couldn't call anything in Quesa; it would have to sit
under everything - perhaps need a CF-style library to hold truly common
bits below these utility libraries).

- Split up the source files. Some of them (math, camera, etc) are
getting too large and could benefit from what happened to E3Geometry.c.

I.e., have an E3Camera.c for the common stuff; then put the individual
camera types into their own E3CameraFoo.c files. If you're thinking
about C++ this would be a good first step, as it means you then have one
C++ object per source file and keep everything well separated.

- Don't go crazy. :-) One of the problems with C++ IMO is that it's very
easy to create systems that are just unmaintainable. Everyone feels
differently about what's good and what's bad about C++, but I've seen
plenty of projects that just got out of control when they started taking
a "C++ purist" view of things rather than the pragmatic view.

The STL is definitely a useful source for all the standard data types
(lists, vectors, hash tables, etc), but I would be wary of going
overboard with stuff like operator overloading.

One of the major problems on the games we see on the games we port is
that CW/gcc do a terrible job of keeping simple C++ objects like a
vector tuple in registers - so overloading "+" to give you a vector
addition for "x + y" will often generate truly dire code compared to
adding the fields yourself.

Similarly for exceptions - I've yet to see a project where they helped,
but have seen several where they were the beginning of the end. :-)


But I'm sure you guys have your own ideas, so feel free to take the
project in whatever way makes sense for you.

I've had a lot of fun working on Quesa, and definitely learned a lot
about what works and what doesn't (both 3D and otherwise).

I'll stay on the list for a day or so, but will be unsubscribing later
this week - my email will be dair AT refnum.com for the forseeable
future, so feel free to drop me a line if you want my 2 pence on
anything. :-)

But for now, have fun - and good luck!


-dair
___________________________________________________
mailto:dair+refnum.com       http://www.refnum.com/