#45 Improve Cache.pm memory footprint

[Bert Driehuis]

In a fix for a severe performance problem, Cache.pm was modified to slurp the entire .db into memory. This should be made configurable, as cases exist where the performance degradation by going to the .db every time is more acceptable than growing the collector by the size of the .db file.

In the longer run, a better solution is needed to improve .db file access without slurping the whole thing into memory. This may involve beter caching, or making use of knowledge about the underlying database to avoid unnecessary accesses. This in turn may require ditching Berkeley DB, which I hope to avoid because I rather happen to like it, but it may be the only way around the issue.

At the least, alternatives to DB_File.pm should be investigated.

[Bert Driehuis]

Logged In: YES
user_id=8475

Just as a followup, this turns out to be tougher than it
seems. Some small improvements were made for 1.0.3, but a
complete solution is still far out.

[Terje Bless]

Logged In: YES
user_id=8470

Configurability should be done now (1.0.4), right? Perhaps
close this bug and add a "Task" on improving the
architecture to be more efficient (better tradeoff between
disk IO and memory use)?

[ This message was brought to you by the Campaign to Keep
the Bug Trackers Clean (CKBTC). :-) ]