[Dev-C++] Quick flat-file searching and I/O

[David M. <ci...@ya...>]

Application is used to store world data for a game. We wish
to use mySQL (or some other SQL database) as well asbe able
to store data in flat-files. My biggest problem with
flat-files is what would be the best way to store and
search such a system of files (a reccomendation, though not
a requirement is that the files remain in plain text. The
current format is the SMAUG world data format (I can send
an example file if you want).

Rooms, objects and mobiles are each indexed by an integer
value between 0 and 4 billion (2^32 to be exact). A room
and a object or mobile can have the same vnum but 2 rooms
cannot have the same vnum (same applies to objects and
mobiles). In other words the index for rooms / objects /
mobiles are separate.

Currently the files are sorted and (poorly) indexed. The
indexing essentially sets a limit to the contents of a file
(i.e. xxx.are contains all rooms/objects/mobiles whose
vnums are between x and y, with y-x = 1000)

The current system involves loading everything into memory
and accessed from there (thus removing the slowdown).
Problem is with only ~6000 rooms we are using 12Mb of RAM.
Even with virtual memory we are going to start running out
of memory as the world gets bigger (for those of you who
are about to curse me out for such an inefficient design,
please note it is not mine).

So far the best solution is to use the room exits. When you
enter a room all the rooms connected to it and all the
associated objects and mobiles are loaded. That way when
the player actually request to go to the next room it is
already loaded. Problem is there are 11 possible exits (up,
down and in the 8-point cardinal directions as well as
through a portal). Unfortunately this means I have a very
low chache hit-miss ratio (1:11) is not good odds. Also
immortals (as well as some players) have skills that can
take them to a random room in the world (there by bypassing
my caching system).

My question is does anyone know either an algorithm or a
library that will allow me to use flat-files fast enough
that I don't have to load up everything into memory. I do
not require it to handle 4 billion rooms/mobs/objects but
be able to handle about 1 million (if they want to go
higher they upgrade to mySQL)

=====
Signed 
David Mcken

Life Sucks
Live with it

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com