Re: [srvx-devel] next generation database approach

[R. B. S. <de...@vt...>]

I'm not familiar with BDB or the alternatives.. I suppose I'll have to read
up a little bit on them before I can contribute to this debate any more.

As for alternatives.. my only suggestion right now is magic midgets that
can fly and are good at relay races  : /

-def

> There are definite advantages to human readable, text based databases;
> but at the same time, it seems the goal for the srvx rewrite is to
> assure scalability to tens of thousands of users. It's debatable whether
> switching databases will be necessary at all -- do Dalnet, Undernet, or
> Quakenet have services that require souped up databases? It may be that
> we're trying to fix something that isn't broken. Of course, the fact
> that the big three networks with services are using derivatives of IRC
> Services or a similar home-brewed solution complete with primitive
> binary database doesn't mean their approach will work for us. Perhaps
> talking to some Dalnet or other big network administrators will prove
> informative.
> 
> The most compelling reason to switch databases as GamesNET grows may
> turn out to be fault tolerance, not performance. We should consider
> fault tolerance within one installation of srvx along with allowing for
> multiple, redundant srvx setups. For example, if the server's power
> cycles, srvx's database should be internally consistent and have
> retained its data up to the moment the power went out. It should also be
> possible for srvx to replicate its databases to slaves, which are ready
> to take over as the primary services should they disappear. Storage will
> be the most difficult part about allowing multiple redundant srvx
> setups, and our choice of database will be important in making it
> possible. It should be relatively easy for a backup set of services to
> realize when they need to kick in and do so.
> 
> If we choose an out of process relational database, it should not only
> pass the ACID test, but allow replication. As far as I know, MySQL and
> PostgreSQL have incomplete or experimental replication support. I assume
> the oodb approach would lend itself to database journalling and
> replication fairly easily with the dirty flags, though we would need to
> implement it ourselves; replication would just involve sending out
> journal entries as they are written to the slaves. Fault tolerance at
> just one level could suffice, though implementing one would be a step
> towards completing the other.
> 
> Issues with allowing other processes to directly access the data
> concurrently along with srvx are, as Entrope pointed out, locking and
> other synchronization issues, and enforcing rules to keep data
> consistent. This will likely not only requires isolated transactions,
> but the duplication of srvx's logic. For example, rules governing which
> characters a handle may or may not contain would be written in srvx,
> then written again for the web interface, etc. Updating this logic would
> be a nightmare. It would probably be best to enforce a familiar tiered
> database design; e.g. web interfaces and other external processes
> needing access go through srvx, which talks to the database. This design
> allows srvx to enforce rules to maintain the integrity of the database,
> and like Entrope said, makes having an out of process database mostly
> pointless. We must enforce the separation of storage and "business
> logic."
> 
> Looking for obvious candidates to cull, I believe we should eliminate
> (4), since it offers us only the advantages of offloading storage and
> comes with the double whammy disadvantages of having to implement
> locking and IPC. Choice (2) seems to lack any advantages whatsoever, so
> that's right out too. (3) and (6) seem to violate tiered design
> principles.
> 
> This leaves us with the choice of multiple saxdbs plus IPC, oodb on
> Berkeley DB plus IPC, and oodb on proprietary DB plus IPC. saxdb would
> require an overhaul to make fault tolerant, but that isn't yet a
> requirement; most likely saxdb is out. If this is the short list, I hope
> we've overlooked a marvellous approach, because I'm not sure that I like
> the choices.
> 
> # Zoot
> 
> 
> _______________________________________________
> srvx-devel mailing list
> srv...@li...
> https://lists.sourceforge.net/lists/listinfo/srvx-devel