Re: XML was RE: [GD-General] RE: A portable preferences library

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Gareth and Jorrit are going each too far in their own direction. I believe
that the reality lies in between. Putting everything into platform-specific
down-to-the-bit-optimized binary data is killing a fly with a cannon, for most
purposes. The only place where the load peformance matters so much is if you
want to stream your data during gameplay, and if you need it quickly, so you
have very little CPU to spend. Other than that, the only thing that you should
care about is that you are limited by HDD throughput, not by CPU speed. As
long as your CPU is ahead of the HDD, you are fine.

What I do is to have a binary format that is universal and that can load any
kind of objects. It can swap the bytes for endianness, etc. but it knows in
advance what layout looks like (in endianness neutral format), and doesn't
have to parse ASCII, let alone generic XML syntax. This is not giving any
generality away, while still allowing you to keep you loading speed under
control. I don't know of any platform where the CPU is so slow that it cannot
perform endianness swapping and such small adjustments while loading. But
going towards XML might be just a bit of an overkill. I say _might_, not _is_.
... because I asked the original question about loading speed plainly because
I am interested in whether XML can be as efficient as a binary format. I
belive that it is not very likely, but not impossiible, and I will accept any
well profiled arguments. While this:

>In case of CS making such a faster loader
>really is a HUGE amount of work. We have LOTS of different data to save

is not something like that. I understand that CS team is not working on a
budget, and it is great that they do stuff for fun and for others (though back
when I was doing coding for my own fun, I wanted the results to be perfect as
well, but thats a different topic). But priorities of CS team _have nothing to
do_ with XML qualities. If I recall, it was the performance of XML parsers
that was discussed, not the idea of whether a loader should be fast or not. If
I say that my loader should be fast, then I guess it should. I was just hoping
that someone can show me some exact numbers.

On a side note, this comment that "it is a HUGE ammount of work", is based on
a wrong assumption. Their problem is that the design was wrong. If they went
for generic binary format from the start, the work would be near zero. But I
digress.

Can you Jorrit please get some more correct profiling data. It doesn't matter
that your app does preprocessing in-game, it is your choice. But how fast is
the parser itself?

Thanks,
Alen

----- Original Message -----
From: "brian sharon" <pud...@po...>
To: <gam...@li...>
Sent: Tuesday, December 16, 2003 16:16
Subject: Re: XML was RE: [GD-General] RE: A portable preferences library

> Would you please give it a rest?  In case you haven't noticed there's
> no "shop" involved for you to return his game to, as he's giving his
> work away freely.  Jorrit is not doing this as a full-time job.  If
> load times are not his first priority I can hardly blame him, it's not
> exactly the "fun" part of the job.
>
> Please visit http://crystal.sf.net to get clued in.
>
> --brian
>
> On Tuesday, December 16, 2003, at 07:59  AM, Gareth Lewin wrote:
>
> > Sorry, but you are just wrong.
> >
> >>> Lets not ignore the fact that loading bits of the level is
> >> way easier with a
> >>> binary specific format that can be dumped directly into data
> >> structures.
> >>>
> >>>
> >> Yes but that's not portable. CS has to run on all kinds of operating
> >> systems and cpu's. We can't
> >> dump binary structures like that. The endianess wouldn't be
> >> correct and
> >> padding can differ
> >> too.
> >>
> >
> > There is no reason whatsoever for final proccessed data to be
> > portable. You
> > have your data in xml. You preprocess it into binary format for a
> > specific
> > build/platform. And you use that. Your load times go from 16seconds to
> > 2 or
> > 3 seconds.
> >
> > And the amount of effort required to implement binary saving off of
> > structures isn't anywhere near comparable with the effort required to
> > tweak
> > the speed of a graphics engine. Not to mention that 16-20 second load
> > times
> > would have me returning the game to the shop.
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: IBM Linux Tutorials.
> > Become an expert in LINUX or just sharpen your skills.  Sign up for
> > IBM's
> > Free Linux Tutorials.  Learn everything from the bash shell to sys
> > admin.
> > Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
> > _______________________________________________
> > Gamedevlists-general mailing list
> > Gam...@li...
> > https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> > Archives:
> > http://sourceforge.net/mailarchive/forum.php?forum_id=557
> >
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IBM Linux Tutorials.
> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557
>