RE: [Algorithms] Disk Thrashing
Brought to you by:
vexxed72
From: Gyurchev, Y. P <yor...@bg...> - 2001-10-31 16:35:25
|
Recently I've tried to implement something like that. A bit different though. The app tries to figure during the current frame what resolution the concrete texture is needed (meaning the highest of all references to this texture this frame). And gives a hint to the manager what resolution is expected for this texture (for the next frame). The manager then runs a separate thread to load the texture in higher resolution and when it is ready it "swaps" internally the pointers - the new texture is used and the old one is freed. (slot scheme is used to fit the video mem) This results in several frames with lower resolution but ensure smooth disc performance (or at least control... of how many loading threads can be started). Switching to lower res comes at no loading cost as the lower res is already in the mipmap. Eventually arises the case when more high res textures are needed to be loaded than high res slots are available. Some priority can be used to lower the res of the "lowest" priority high res texture and load the new one at the available slot. Or LRU? The problem is really to decide what res do we need. Jan Svarovsky's mipmap factor (http://www.svarovsky.freeserve.co.uk/ExtremeD/) can be used but I think for static geometry some recalculation is better. I've tested with simple linear distance calculation and it worked fine. > Our stuff here loads lowest mip levels first, so there'll (almost) always be > something to see. Agree. Though it might be the case that there are lots of those textures (and the API or driver allocates space per each texture at least some of them) and excessive number of small textures could not fit the mem. Any comments? Yordan > -----Original Message----- > From: Ratcliff, John [mailto:jra...@so...] > Sent: Wednesday, October 31, 2001 9:07 AM > To: Game Dev Algorithms (E-mail) > Subject: [Algorithms] Disk Thrashing > > > In this email I want to ask a question related to the texture > management > issues recently discussed. The problem I want to resolve is > disk thrashing > for art assets. For many of us, gone are the days of > 'loading level please > wait'. Our 'levels', are just too friggen huge. We are > getting close to > shipping games with a 1 or 2 gigabyte disk footprint. > Especially if you are > working on a massively multiplayer game that requires an > extremely huge > world with massive amounts of content. > > So, the question is this. Let's say you have a very efficient texture > manager that keeps you from ever thrashing video memory. > That's great, you > get high frame rate, no problem. However, where previous discussions > referenced 2mb and 4mb of texture memory available, we are > talking about 20 > and 40mb of texture memory per-frame. These nice 64mb video > cards gotta be > good for something, and uploading incredible looking high-resolution > mipmapped textures including cubemaps, accounts for a lot of content. > Again, we are not talking about a texture management problem > or frame rate > problem. There's plenty of video memory available and > rendering is very > fast. The issue I want to address is the disk hit associated > with accessing > hundreds of megs of art assets as a user interacts with the > environment. > > Let's say, for example, you are walking through a portalized interior > environment. Where any given portal region does not consume an excess > amount of video memory and renders very fast. But, as you > walk through the > building, you may cause 100mb of unique textures to need to > be loaded from > disk, and while your user has this kick as 64mb video card, > he's running > WindowsXP on a 128mb machine. > > Here's what I am contemplating doing to solve this problem > and I wanted to > propose it to the list to find out if anyone else had tried something > similar. > > My strategy is to create a disk-cache thread that is pretty > much always > reading from disk as a background task. It reads compressed > data in, so it > can fit as much into the cache as possible. The game engine will send > 'hints' to the disk cache manager of textures and other > assets it thinks it > may need in the near future. For example, if you are in a portalized > building, likely you will need textures of adjoining portal > spaces pretty > soon. So while you are wandering around one room, the disk > cache manager is > busy reading all of the textures for the adjoining rooms in > the background > and storing them (still compressed) in it's cache. > > When you actually need a data asset your hope is that most of > the time it > will be found in the disk cache already loaded, thus > minimizing the disk > hit. Another strategy I am considering using is that for every single > texture I store two versions, one at 1/2 resolution, which > takes up 1/4 as > much disk space. When disk hits are becoming too painful I > can defer to the > lower resolution versions of the textures, essentially a form > of mipmapping > purely to minimize disk access times. > > Does any of this sound reasonable/feasible? > > John > > _______________________________________________ > GDAlgorithms-list mailing list > GDA...@li... > https://lists.sourceforge.net/lists/listinfo/gdalgorithms-list > |