Ryan Krauss wrote:
> The PickleCol works great. It is a neat idea and it does exactly what I
> need in this situation. But the files are very large. SciPy has a
> function called shelve and using it to store the same data seems to
> result in files about 1/10th the size of using PickleCol and matrices.
> I think shelve uses some sort of compression or something. Is there
> away to use compression with PickleCol?
Is shelve in SciPy based on the shelve module in the standard Python
library? If so, it uses pickle to store its data, and there should be
little difference. But since that doesn't seem to be the case, perhaps
you could look for helper functions for SciPy's shelve, possibily in
combination with cStringIO, to store the data using a StringCol and
restore it when the data is retrieved.
--
Brian Beck
Adventurer of the First Order
|