[Quickfix-developers] Berkeley DB MessageStore

[Caleb E. <cal...@gm...>]

Attached please find my implementation of a Berkeley DB-backed
QuickFIX MessageStore class called "BDBMsgStore" (with associated
"BDBMSgStoreFactory")

The BDBMsgStore can be configured to use system memory (DB_SYSTEM_MEM)
for the shared memory regions (default is mmap'ed files) and/or
transactional semantics (DB_AUTO_COMMIT) for all writes.

This class depends on the following configuration settings to operate:

DBStorePath (string, global): directory where the database files will
be written.  Will be created if it does not exist.
DBUseShm (boolean, global): if enabled, the Berkeley DB environment
will be configured with the DB_SYSTEM_MEM flag
DBShmKey (integer, global): if DBUseShm is enabled, this should also
be set.  This is the key that will be used to create the shared memory
region.
DBAutoCommit (boolean, per connection): if enabled, turns on
transactional semantics for all database writes on a connection.  Not
enabled by default.

Additionally, Berkeley DB will read a file called DB_CONFIG if it
exists in the environment home directory (DBStorePath).  You can tune
things at a very fine-grained level with this file.  Please refer to
the Berkeley DB documentation for details.

I've done a bunch of testing on this code, and it seems to work well
and is quite fast.  Without transactions enabled, I'm able to push
approximately 7,000-8.000 small NewOrderSingle requests per second
through this store on a 2-way Xeon 3.2 GHz writing to (I believe) a
5400 RPM IDE drive.  This is actually on par with the MemoryStore,
which gives be about 8,500 msgs/second on the same hardware.  With
transactions enabled, this number drops to around 200 msgs/sec.

Please let me know if you have any questions with this implementation
(or find bugs in it).  I would be happy to see it included in future
releases of QuickFIX, and am releasing it under the terms of the
QuickFIX software license, version 1.0

-- 
Caleb Epstein
cal...@gm...