Re: [Pytables-users] Re: Re: nctoh5 script

[Jeff W. <js...@fa...>]

Francesc Alted wrote:

>A Dimarts 02 Novembre 2004 23:56, Jeff Whitaker va escriure:
>  
>
>>Francesc:  There was no good reason for using Array instead of EArray 
>>for rank-1 variables, other than I wasn't sure EArrays were appropriate 
>>for variables that were not going to be appended to.
>>    
>>
>
>Yes, they are. The only reason for using an Array instead of an EArray is a
>matter of simplicity. For me, creating Array objects from the interactive
>console is far easier than EArray, but after the creation, both objects
>works very similar. However, EArray do support filters (apart of being
>extensible), so, for programs, and if compression is desirable, I do
>recommend using EArray objects for everything, even when you don't want to
>enlarge them.
>
>  
>
Francesc:  OK, thanks for the explanation.

>>If I replace
>>
>>                vardata.append(var[n:n+1])
>>
>>with
>>
>>            if dtype == 'c':
>>                chararr = numarray.strings.array(var[n].tolist())
>>                newshape = list(chararr.shape)
>>                newshape.insert(0,1)
>>                chararr.setshape(tuple(newshape))
>>                vardata.append(chararr)
>>            else:
>>                vardata.append(var[n:n+1])
>>
>>
>>it seems to work.  Note that I have to reshape the chararray to have an 
>>extra singleton dimension or pytables complains that the data being 
>>appended has the wrong shape.   This is also the reason I had to use 
>>var[n:n+1] instead of var[n] in the append.  Is there a better way to do 
>>this?
>>    
>>
>
>Well, this is a subtle problem, as append(array) expects array to be of the
>same shape as the atomic shape. When the array has a dimension less, I guess
>it would be safe to suppose that what the user wants is to add a single row
>in the extensible dimension. Frankly, I don't know if implementing this kind
>of behaviour would help the user to understand how append() works.
>  
>

If updating the EArray via slice assignment were supported in the future 
this would be a non-issue, since I guess one could just do

vardata[n] = var[n]

>By the way, I've solved in PyTables the problem when the object to append is
>a Numeric object with Char type ('c' typecode). I'm attaching my new version
>for your inspection (beware, this will run only with PyTables 0.9!).
>
>You surely have noted that my code may convert NetCDF files with enlargeable
>dimensions other that the first one. I don't know whether this is suported
>in NetCDF or not. I only know that Scientific Python does not seem to
>support that.
>
>  
>
Excellent!  Thanks for taking an interest in my little script.  You've 
taught me a lot about PyTables along the way.

BTW:  netCDF only allows the first dimension to be unlimited.  
Interestingly, the upcoming netCDF 4 will be a wrapper on top of the 
HDF5 API, and will support compression and multiple unlimited dimensions.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : Jef...@no...
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124