In AgileWiki2 there were utilities for dumping and restoring, for the
whole database or only a single Cabinet. There was also a snap utility
which would dump only present state. This latter was used to help manage
the size of the database, as well as being very helpful when we wanted
to change the low-level datastructures. AgileWiki2 was built on
BerklyDB, which handled the transaction processing, log files and recovery.
In AgileWiki3 we are not using BerklyDB and all the files are at least
an order of magnitude smaller. And between conversion to Java and
writing our own database, we've seen a speed increase which sometimes
reaches 3 orders of magnitude. But then, this is a custom database
designed just for AgileWiki. However, both TKS (the AgileWiki3 database)
and BerklyDB are btrees and support transactions.
So we are creating our own log file in AgileWiki3. And the first utility
will be to rebuild the database from scratch using only a set of log files.
The log files consist of a series of transactions, but the content is
comprehensive. In most cases, the entire request event (XML document)
from the client is captured. As well, every database property change is
recorded. However, updates to the inverted indexes are not captured
separately. The trick here is that TKS properties with a name beginning
with "_" are inverted properties.
Hmm. Another thing to keep in mind is that only the doc property can
contain line feeds. This property is then given special handling in the
log file.
Oh and then there are property tables. Properties with names ending with
"-" have as their value a series of key/value pairs. And the order is
significant (think list, not set). Of course, inverted table properties
are supported as well, though in this case the values are not used in
the inverted table.
Bill
|
