I managed to rewrite some code in STXXL, and now I'm supposed to run a number of test.
The test data is relatively small (vectors and matrices of size 12), and the amount of time I
need to obtain the results is prohibitively large. Is there a way to speed this process up, so
that I obtain the results in the time nearly equal to the one I obtained using plain STL.
I changed the .stxxl file, included one disk, disk=/home/mirza/fileOne,3,syscall
but I obtain this message:
External memory block allocation error: 2097152 bytes requested, 1048576 bytes free. Trying to extend the external memory space…
What way I might speed this process up? Note that this is to check whether the results are the same as those obtained using
plain STL.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
For testing, you could specify 'memory' instead of 'syscall'. Then, the data is not written to disk, but stays in the (limited-size) RAM. Documentation will be added…
For memory, multiple disks will not help, of course.
stxxl::vector has an overhead compared to std::vector, of course. And the smaller the data is, the more obvious it will be. You can try decreasing the overhead by making the block size smaller (if your working set is much smaller than one block).
Johannes
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I managed to rewrite some code in STXXL, and now I'm supposed to run a number of test.
The test data is relatively small (vectors and matrices of size 12), and the amount of time I
need to obtain the results is prohibitively large. Is there a way to speed this process up, so
that I obtain the results in the time nearly equal to the one I obtained using plain STL.
I changed the .stxxl file, included one disk, disk=/home/mirza/fileOne,3,syscall
but I obtain this message:
External memory block allocation error: 2097152 bytes requested, 1048576 bytes free. Trying to extend the external memory space…
What way I might speed this process up? Note that this is to check whether the results are the same as those obtained using
plain STL.
For testing, you could specify 'memory' instead of 'syscall'. Then, the data is not written to disk, but stays in the (limited-size) RAM. Documentation will be added…
The error message can be avoided by either increasing the "disk" space (3 -> ?) or decreasing the internal vector buffers (see http://algo2.iti.kit.edu/dementiev/stxxl/tags/1.2.1/structVECTOR__GENERATOR.html)
Johannes
Thanks. Is the running time halved if I add one more disk (not syscall, but memory, as I need it for testing)?
ps. The general question is : How to achieve nearly same execution time as if the code was written using plain STL?
For memory, multiple disks will not help, of course.
stxxl::vector has an overhead compared to std::vector, of course. And the smaller the data is, the more obvious it will be. You can try decreasing the overhead by making the block size smaller (if your working set is much smaller than one block).
Johannes