[opendemo-devel] performance and other issues

[Conor D. <co...@ma...>]

Hello all,

It's been a long time since I've done anything about opendemo (many
apologies, I really need to take a time management class)! Anyway, I have
about a week and half with little else to do, so I can work on opendemo for
a while.

Congrats to hmage for fixing the event time bug; I don't think I would have
found that one for a long time. However, it looks like we are using
CS_LEVEL_START when writing to xml file, and since it is defined in
bg_public.h, in theory a mod author may change/remove it and screw
everything up. I've tried to only use stuff in q_shared.h to make the file
format as flexible as possible between all the various mods. Can we move
this code into the cgame module? I think we will have to modify cgame
eventually, so why not start now?

About performance:
- the od_filebuf.c routines use a linear buffer, which now I can see is a
really stupid thing to do. It shouldn't be too hard to convert it to use a
growable circular buffer which should solve the fbReadChar problems
- I'd say the problem in odpStartElement is in the <snapshot ...> element,
with:
   *od.demo.currentState = *od.demo.deltaState;
  This copies a lot of data (half a meg) every snapshot, which is not good.
We can save some time by looping through each player and entity, and only
copy the ones which are active.

Another option is to remove the whole 16 snapshot states and force delta'ing
only from the previous snapshot (since it is recording on the server, we
don't have to worry about dropped snapshots and whatnot). This would
completely kill any chance of converting real (client-side) demos into the
xml format (I'm just guessing that they use 16 states, since q2 does that),
and would hinder recording over a network (server-to-server), but would
solve the whole copying issue. Or we could leave the 16 snapshot history in
the file format, but have odplay only use one state (the odDemo_t structure
is almost 10 megs, the 16 states take up almost all of it).

I'm looking into using zlib for compression; it doesn't look too bad if we
can emulate ANSI file i/o and dynamic memory management, which we have
already done mostly. Speed will probably be an issue (using zlib's file i/o
routines, which use the emulated file i/o routines, which use quake3's trap
file i/o routines).

Can anyone forsee using od_file*.c, od_mem*.c, or od_xml_read.c outside of a
q3 mod? I tried to make them generic so we could use them in
demo-manipulating utilities, but we could just as well use a real xml parser
(expat, libxml, java stuff) for that. od_fileio is only needed for
od_filebuf, which is only needed by od_xml_read, so if we don't need
od_xml_read for other programs, we can move them to the q3 game directory
and do away with the abstract interfaces. od_mem*.c may be useful just to
have growable character arrays.

I don't think random-access play can be done well using a qvm, since AFAIK
we would need real random file i/o and dynamic memory access for that. We
can do it, but it looks like we would have to re-open the file every time we
go backward and re-read all the snapshots until we get to the desired frame.

-Conor