Thanks for the great library. I ran into a scalability issue and would greatly some input regarding this.
I have a setup where I create k threads. Each thread creates a separate stxxl vector. Each vector is associate with a syscall file that is in turn tied to the same disk resident file.
I would think that the access to vector vi by thread ti should be independent of vector vj by thread tj. But underneath the hood stxxl serializes these access by some scoped_mutex_lock.
Please let me know if this is the expected behavior and if there is a way to have separate queue for each vector.
Which scoped_mutex_lock do you mean exactly?
If the vectors are stored on the same disk, the requests go to the same queue, which involves locking. But the overhead of a lock should be dominated by far by a block access (although this might not be the case for small blocks on SSDs).
To circumvent that, you could attach a vector to a specific file, and associate each of the files to a separate queue (by setting a different queue_id).