Re: [Pytables-users] error creating table with many columns

[Stefan K. <pon...@ya...>]

The domain here is analytic ( statistics ) data, so 1000s of variables
is within the normal usage.  They are currently using SAS datasets.  I
guess we can have multiple tables look like one, but I fret about a
performance hit of an extra lookup layer in the Python space.  It
doesn't look like that header size value is expandable in HDF5.

S

--- Francesc Altet <fa...@ca...> wrote:

> A Dimarts 04 Octubre 2005 21:14, Stefan Kuzminski va escriure:
> > I can create multiple tables in the same group that are 500
> columns, so
> > I suppose this is a limit in a single table's header?  The overall
> size
> > of the full file does not seem limited. ( I hope )
> 
> Yes, that's right. Header table data (i.e. where the information on
> the table structure is kept) has a limit of 16k, while actual data
> space has not limit (I've created tables with more than 100 GB of
> data
> with not problem).
> 
> > The data is heterogenous so muti-dimensional columns aren't
> applicable.
> >  I suppose we will have to split the big datasets apart into
> multiple
> > tables and iterate them in parallel.
> 
> Well, that's a possibility.
> 
> -- 
> >0,0<   Francesc Altet     http://www.carabos.com/
> V   V   Cárabos Coop. V.   Enjoy Data
>  "-"
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads,
> discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> Pytables-users mailing list
> Pyt...@li...
> https://lists.sourceforge.net/lists/listinfo/pytables-users
> 



		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com