Menu

#2 split large databases

open
nobody
5
2005-06-29
2005-06-29
Th Kn
No

Problem:
when a database gets bigger the filesize rises.
Since "db->write" is the most time consuming operation
things slow down a lot when getting a higher document
count.

Possible Solution:
Split the database into several files.

New Problems are:
- what about the unids? (same unids in different files
may occur)
- how to determine (fast) which doc is in which file

Possible Solution: make a new class "joDocLib" which
contains a number of databases and maintain an index
(unid->db). The joDocLib itself could be a database
which contains documents with just a single Item (for
now) pointing to the right db. Thus no really new
technology is needed, just a "chained" db access.

Important: unid's probably have to be longer!

Current unid length = 3 (46656 docs max)

Discussion


Log in to post a comment.