Database Redesign

If you're tracking the DAD project, you'll find this to be extremely important news. Over the past six to nine months, a variety of tables were added and other database changes made that were not the most optimal. We are happy to announce today that a significant redesign has begun.

The most significant and important change is that the way that events are stored is being completely changed. All of the events will now be stored in a table called 'events' and the entire content of the events will be full text indexed through a normalization process.

In the short term this means that we will be providing a brand new aggregator (really the same old aggregator with lots of changes) and a conversion utility to suck all of your old data into the new format.

Immediately after that, all of the existing alerts and queries need to be rewritten to take advantage of the new format. You should find that everything is extremely speedy though writing the queries will require some contortions since it will require several joins.

Please watch for details. I can tell you that we've already got what we believe is a firm schema for the events and the aggregator is already (mostly) functioning. We need to test it out under load and make sure that the performance hits are not too terrible.