Hi All - I'm trying to resolve an occasional but debilitating DB_RUNRECOVERY error. We have two threads accessing the bdb concurrently. the threads runs fine up to around 80k entries, but then at a random entry poop out with a DB_RUNRECOVERY error.
much obliged for any assistance.
thread A: for(i = 0; i < 100000; i++)
creates a new key/value hash each time
writes new log entries to the db.
sleep(random ms upto 20ms)
updated synch variable WritingIsDone = true
thread B. while Not WritingIsDone
open a ReadUncommitted cursor,
gets all log keys
For each log key found
check value hash
delete the key
sleep(random upto 200ms)
Here's how the db is created:
// create and open the BDB
_DbEnv = new BerkeleyDb.Env(EnvCreateFlags.None);
const Env.OpenFlags envOpenFlags =
//Env.OpenFlags.InitCDB | Env.OpenFlags.InitMPool | Env.OpenFlags.Create;
Env.OpenFlags.InitMPool | Env.OpenFlags.Create | Env.OpenFlags.SystemMem | Env.OpenFlags.InitLock;
_DbEnv.Open(BDBDir, envOpenFlags, 0);
_Database = _DbEnv.CreateDatabase(DbCreateFlags.None);
(DbBTree)_Database.Open(null, BDBDir + "\\" + BDB_FILE, null,
BerkeleyDb.DbType.BTree, Db.OpenFlags.ReadUncommitted| Db.OpenFlags.Create, 0);
Here's how the cursor is setup:
using (BerkeleyDb.DbBTreeCursor cursor = _DbTree.OpenCursor(null, DbBTreeCursor.CreateFlags.ReadUncommitted))
DbEntry key = DbEntry.Out(new byte);
DbEntry data = DbEntry.Out(new byte);
while (ReadStatus.Success == cursor.Get(ref key, ref data, DbBTreeCursor.GetMode.Next, DbBTreeCursor.ReadFlags.ReadUncommitted))
string keystr = new string(System.Text.Encoding.ASCII.GetChars(key.Buffer));
keystr = keystr.Trim('\0');
string datastr = new string(System.Text.Encoding.ASCII.GetChars(data.Buffer));
entries[keystr] = datastr.Trim('\0');
key = DbEntry.Out(new byte);
data = DbEntry.Out(new byte);
} //using closes the cursor
Also wanted to add that the same database DbBTree handle is passed to both threads.
Have you tried opening the database with the addition flag Db.OpenFlags.ThreadSafe?
I am not sure this will help, since I am already synchronizing on the db handle,
but it is worth a try.
Just an unrelated comment: you don't need to allocate a new buffer every time your loop iterates.
Just allocate the key and data buffers once, like
byte keyBuf = new byte;
byte dataBuf = new byte;
and then, in the loop, initialize the DbEntry instance on each iteration:
key = DbEntry.Out(keyBuf);
data = DbEntry.Out(dataBuf);
So I tried the Db.OpenFlags.ThreadSafe flag to no avail. It doesn't make sense why it fails after 70-80k entries. What general strategies do you recommend for preventing the database from getting corrupt? I tried syncing every 10000 entries, but it didn't change anything.
Thanks for the optimization, btw.
These issues are probably more related to how BDB is used than my .NET adapter.
If you like, you could e-mail me (karl at waclawek dot net) a small self-contained
application which generates the error, and I will have a look.
But you should also try the Berkeley DB support forums at Oracle:
They know better than anyone how BDB works.