Multliple Block Managers / Disk Configuration Files

2013-11-14
2013-11-15
  • Hello,

    Are there any plans to have more than 1 block_manager object in future versions of STXXL? Apologies if this question was asked and answered already. I'm interested in such a feature. I understand that extensive re-factoring may be required to facilitate that.

     
  • Timo Bingmann
    Timo Bingmann
    2013-11-14

    You already answered the question yourself:
    "extensive re-factoring may be required to facilitate that".
    So someone has to do it, that needs these features.

    The other issues with separate block_managers is that this goes against the design of one of STXXL's main features: parallel disk support.
    Parallel disks and also parallel compuation, however, are really needed these days because these are the bottlenecks in most applications.

    However, we do see that many people want to "put containers into files". This feature is currently only supported by the stxxl:vector. Support for file-based containers should be increased in the future.

     
  • Thanks for the reply.

    I'd be happy to do it and contribute that back. I am not so keen on keeping a permanent fork of stxxl all to myself. ( If I have to do that I will do my best to link to stxxl proper for as much of the implementation as I reasonably can. ) I'm also interested in developing persistence for the b+tree with checkpointing. Is there any interest in that?

     
  • Timo Bingmann
    Timo Bingmann
    2013-11-15

    Those are big plans, a full-fledged B+tree database with locking, checkpoints, logging and so on is probably out of scope of STXXL. BerkeleyDB already provides all that.

    But, what STXXL could be very useful for is a slim B+ tree implementation, that is mostly used for read queries.

    I guess the first steps would be to document the B+ tree adequately. Most is not very well explained.

    Also I just looked, and saw that the tree nodes currently contain pair<key,BID> where BID contains a file* to the object containing the block. That of course does not work for an implementation requiring persistance. This is the main problem one would be facing for a single-file B+tree implementation.

    Kind Regards, Timo