RE: [Algorithms] Disk Thrashing

[Tom F. <to...@mu...>]

It sounds good.

> Another strategy I am considering using is that for every single
> texture I store two versions, one at 1/2 resolution, which 
> takes up 1/4 as
> much disk space.

A bit like a mipmap then :-) Seriously, I'm a big fan of "just too late"
texturing, where you always have tiny (16x16) mipmaps resident, and you
dynamically load the other textures as you need them. you can use prediction
to try to load them early, but you always need to be able to cope with the
worst-case scenario - that your prediction completely fails, and you have
nothing loaded.

But you always have the 16x16 levels loaded, so you can use them. At least
they're roughly the right colour :-) And meanwhile your texture loader is
loading the higher mipmap levels in the background (from the 32x32 upwards).
IMHO, it is far better to have a blurry texture for a few frames than to
stall the rendering while you load the proper high-res versions, especially
as with a bit of luck the player may never notice the blurry texture - in a
lot of cases it will actually be hidden, it's just that the PVS system was
being a bit conservative.

A warning - I implemented all this for StarTopia, and it worked really well
on some cards, and bloody terribly on others. The problem was that on the
troublesome cards, the instant you uploaded a texture level, the whole
rendering pipe came to a grinding halt, and then took ages to get started
again. So this lovely "background loading" of textures wasn't at all
unobtrusive. On those cards, it was far quicker to just stall, load all the
levels, and do the whole upload in one go. On decent cards & consoles, you
shouldn't have this trouble.

Tom Forsyth - Muckyfoot bloke.

What's he up to now (and can I have a go)?
http://www.eidosinteractive.com/downloads/search.html?gmid=86

> -----Original Message-----
> From: Ratcliff, John [mailto:jra...@so...]
> Sent: 31 October 2001 15:07
> To: Game Dev Algorithms (E-mail)
> Subject: [Algorithms] Disk Thrashing
> 
> 
> In this email I want to ask a question related to the texture 
> management
> issues recently discussed.  The problem I want to resolve is 
> disk thrashing
> for art assets.  For many of us, gone are the days of 
> 'loading level please
> wait'.  Our 'levels', are just too friggen huge.  We are 
> getting close to
> shipping games with a 1 or 2 gigabyte disk footprint.  
> Especially if you are
> working on a massively multiplayer game that requires an 
> extremely huge
> world with massive amounts of content.
> 
> So, the question is this.  Let's say you have a very efficient texture
> manager that keeps you from ever thrashing video memory.  
> That's great, you
> get high frame rate, no problem.  However, where previous discussions
> referenced 2mb and 4mb of texture memory available, we are 
> talking about 20
> and 40mb of texture memory per-frame.  These nice 64mb video 
> cards gotta be
> good for something, and uploading incredible looking high-resolution
> mipmapped textures including cubemaps, accounts for a lot of content.
> Again, we are not talking about a texture management problem 
> or frame rate
> problem.  There's plenty of video memory available and 
> rendering is very
> fast.  The issue I want to address is the disk hit associated 
> with accessing
> hundreds of megs of art assets as a user interacts with the 
> environment.
> 
> Let's say, for example, you are walking through a portalized interior
> environment.  Where any given portal region does not consume an excess
> amount of video memory and renders very fast.  But, as you 
> walk through the
> building, you may cause 100mb of unique textures to need to 
> be loaded from
> disk, and while your user has this kick as 64mb video card, 
> he's running
> WindowsXP on a 128mb machine.
> 
> Here's what I am contemplating doing to solve this problem 
> and I wanted to
> propose it to the list to find out if anyone else had tried something
> similar.
> 
> My strategy is to create a disk-cache thread that is pretty 
> much always
> reading from disk as a background task.  It reads compressed 
> data in, so it
> can fit as much into the cache as possible.  The game engine will send
> 'hints' to the disk cache manager of textures and other 
> assets it thinks it
> may need in the near future.  For example, if you are in a portalized
> building, likely you will need textures of adjoining portal 
> spaces pretty
> soon.  So while you are wandering around one room, the disk 
> cache manager is
> busy reading all of the textures for the adjoining rooms in 
> the background
> and storing them (still compressed) in it's cache.
> 
> When you actually need a data asset your hope is that most of 
> the time it
> will be found in the disk cache already loaded, thus 
> minimizing the disk
> hit.  Another strategy I am considering using is that for every single
> texture I store two versions, one at 1/2 resolution, which 
> takes up 1/4 as
> much disk space.  When disk hits are becoming too painful I 
> can defer to the
> lower resolution versions of the textures, essentially a form 
> of mipmapping
> purely to minimize disk access times.
> 
> Does any of this sound reasonable/feasible?
> 
> John
> 
> _______________________________________________
> GDAlgorithms-list mailing list
> GDA...@li...
> https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list
>