Re: [Algorithms] Data base choices
Brought to you by:
vexxed72
From: Jamie F. <ja...@qu...> - 2007-04-05 13:04:50
|
Tom Plunket wrote: > Jamie Fowlston wrote: > >>> I may not be relating very well to the OP's point (I certainly don't >>> know anything about Boost serialization :) I'm simply objecting to >>> "serialization = bad", which seemed to be the main point of your post >>> (when condensed to two words!) >>> >>> If you're a big fan of "exhale/inhale style binary serialization" then >>> that's fine. >> No, I'm not, and I hope it's clear that the system I've described is >> nothing like that :) I think we're all agreed there is a distinction >> between "exhale/inhale" (or naive) serialization and the more sane and >> structured systems we've described. > > I don't see the distinction, frankly. Although my thoughts on the topic > initially were that "serialization" equated to "loading and storing > stuff from persistent media" and didn't necessarily imply that reads > from storage were done field-wise. My understanding of serialization is much like yours, and I think some of the confusion has come from others understanding serialization as an inhale/exhale thing, which is pretty much just blatting the memory the object occupies to disk (fixing pointers up on the way). Whilst the latter is a type of serialization, it's not one I'd choose to use any more (having done so many years ago). > Everything I've read here, it really sounds like everyone ultimately > ends up with the same thing, and I don't see much in the way of actual > distinction except for the actual persistence mechanism (structured > database vs. flat files). Did anybody argue for a flat file? > I do wish to throw out that I've authored several systems that allow > most objects to be forward and backward compatible, but those all relied > on the data being text; i.e. the schema was baked into the data. I > could see such a mechanism working with binary blobs, too, though. It can indeed work for binary blobs. Our system uses either text or binary depending upon which backend you choose. > Looking back through the thread, though, is "inhale/exhale" meant to > imply every member of every structure is serialized in order, whereas > "not inhale/exhale" means only the data that is required to reconstitute > the object is stored? My understanding is that an inhale/exhale system has the data in the file being as close to the runtime memory layout as possible. "Not inhale/exhale" covers everything else; which would usually mean you'd only store the data that you know you'd need to reconstruct the object. > I'm firmly in the old-school camp, where you do a single big read and > your whole world is "up". Then you hit your save game data, one read > into one memory location and wham it's restored. I understand that this > is hostile to versioning, although in my experience on consoles this has > never been a problem. I recognize though that the online presence of > consoles now will be changing that, however. At the same time, I still > feel the idea has merit; just stream a big data block off of disc and be > running. The near-zero CPU load streaming of terrain, for instance, > made for some really large levels with no "loading please wait" prompts > on-screen. The main advantage of this system is engineering simplicity, I would suggest. > The more CPU you need to use after loading your data, the > less likely you'll be able to implement a seamless streaming solution. A key part of our seamless streaming solution is a background work queue; one of its purposes is to prepare recently loaded data for use. This can include decompressing JPEGs, PNGs, or anything else which a plugin (could be ours, could be a custom one) chooses to do when it's loaded. > This point may ultimately be irrelevant to the discussion taking place, > however, if folks don't care about streaming. :) We care deeply about streaming :) Jamie |