[Pytables-users] Re: I agree on factorizing things

[Francesc A. <fa...@op...>]

A Dimecres 16 Juliol 2003 21:03, vareu escriure:
> Oh ok, I misunderstood what you were implementing lazily.
>
> Regarding memory, I wasn't clear.
>
> If you do an, hdf_load(file), you'll get something that acts like a
> python dictionary
> that will load off of the disk the data when needed. It's not like a
> pickle which loads everything off of the disk into memory.

Oh I see... Indeed, this would a nice improvement to make pytables even more
pythonic. I remember something similar has been done with the Objectify
module by David Mertz. I think this is worth to read:
http://www-106.ibm.com/developerworks/xml/library/xml-matters2/

This was in the context of XML, but most of the ideas applies to pytables
(both databases are hierarchical).

>
> john
>
>
> -----Original Message-----
> From: Francesc Alted [mailto:fa...@op...]
> Sent: Wednesday, July 16, 2003 2:49 PM
> To: Nielsen John
> Subject: Re: I agree on factorizing things
>
> A Dimecres 16 Juliol 2003 20:01, vareu escriure:
> > Oh, so I won't worry about implementing the lazy stuff then.
>
> Why? I'm just avoiding reading the *attributes* (I mean VERSION, TITLE
> and
> attribute stuff like that) of objects, but my current code *still*
> builds
> the complete object tree. So your idea remains *completely* valid and I
> have
> not worked on that. So feel free to continue working on that issue if
> you
> are inclined to do so.
>
> > I am layering the data structure conversion code on top of yours.
>
> It'll
>
> > just be a library to import if one so chooses.
> >
> > In theory it would work like:
> > import hdf_file
> >
> > hdf_file.save(file,data)
> > data=hdf_file_load(file)
> >
> > Of course, the data isn't all there in memory, the access to it will
>
> be
>
> > managed.
>
> Uh? This was not for Python (in-memory) objects?

-- 
Francesc Alted