Re: [Numpy-discussion] how-to "put" RAM-based numarray into memmap

[Sebastian H. <ha...@ms...>]

> On Mon, 2003-11-24 at 14:13, Sebastian Haase wrote:
> > > Sebastian Haase wrote:
> > > >
> > > > Hi,
> > > > Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard)
> > > > numarray.
> > > > Now I would like to "save" that onto harddrive (with a small header
up
> > > > front
> > >
> > > How about:
> > >
> > > f = open(filename, 'wb')
> > > f.write(MyHeader)
> > > A.tofile(f)
> > >
> > > To read it back in, you need to know where your header ends, by either
> > > parsing it or using one of the same size every time, then you can use
> > > fromfile() to create an array form it.
> >
> > The main reason for my question was just to find out if NUMARRAY
supports
> > it, and how ?
> > Also I have many "bookkeeping" functions already implemented for the
> > memmap'd case.
> > (That is, I have a class with member methods operting on a member
memmapped
> > array)
> > So if what I descibed is possible I could save myself form duplicating
lot's
> > of code.
> >
> > Essentially I was hoping for the most ellegant solution ;-)
> >
>
> memmap's Memmap class does support an insert() method for adding a new
> slice to the end of (or anywhere in) an existing map.  The new slice,
> however,  will exist as a block of memory allocated on the heap until
> the memmap is saved to disk.
>
> Thus, two scenarios present themselves: (1) you allocate the new slice
> ahead of time and create an array from it, avoiding data duplication (2)
> you create an array and later copy it into (a newly inserted slice of)
> the memmap, thereby duplicating your data on the heap.
>
> When you close the map,  slices on the heap are written to the map file.
>
> Todd
>

Thanks for your reply.
Is it possible to define a memmap slice and giving it a (preinitialized !)
memory buffer ?
I'm thinking: I have a RAM-based numarray, I just take the buffer (pointer)
and hand it over to the memmap-slice so that it can make the association
between the disk-space and the RAM-space.
I guess you are calling "heap" what I call RAM.  Is memmap using something
inherently different that heap? (I might be missing something...)
As I was trying to illustrate in my example, my ram-numarray might already
be using most of the available address space.

Thanks,
Sebastian