Multliple Block Managers / Disk Configuration Files

2013-11-14
2013-11-15
  • Kuberan Naganathan

    Hello,

    Are there any plans to have more than 1 block_manager object in future versions of STXXL? Apologies if this question was asked and answered already. I'm interested in such a feature. I understand that extensive re-factoring may be required to facilitate that.

     
  • Timo Bingmann

    Timo Bingmann - 2013-11-14

    You already answered the question yourself:
    "extensive re-factoring may be required to facilitate that".
    So someone has to do it, that needs these features.

    The other issues with separate block_managers is that this goes against the design of one of STXXL's main features: parallel disk support.
    Parallel disks and also parallel compuation, however, are really needed these days because these are the bottlenecks in most applications.

    However, we do see that many people want to "put containers into files". This feature is currently only supported by the stxxl:vector. Support for file-based containers should be increased in the future.

     
  • Kuberan Naganathan

    Thanks for the reply.

    I'd be happy to do it and contribute that back. I am not so keen on keeping a permanent fork of stxxl all to myself. ( If I have to do that I will do my best to link to stxxl proper for as much of the implementation as I reasonably can. ) I'm also interested in developing persistence for the b+tree with checkpointing. Is there any interest in that?

     
  • Timo Bingmann

    Timo Bingmann - 2013-11-15

    Those are big plans, a full-fledged B+tree database with locking, checkpoints, logging and so on is probably out of scope of STXXL. BerkeleyDB already provides all that.

    But, what STXXL could be very useful for is a slim B+ tree implementation, that is mostly used for read queries.

    I guess the first steps would be to document the B+ tree adequately. Most is not very well explained.

    Also I just looked, and saw that the tree nodes currently contain pair<key,BID> where BID contains a file* to the object containing the block. That of course does not work for an implementation requiring persistance. This is the main problem one would be facing for a single-file B+tree implementation.

    Kind Regards, Timo

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks