Re: [getdata-devel] getdata bug report: when using gzip encoding and bitfields

[D. V. W. <ge...@ke...>]

On Wed, Feb 19, 2014 at 01:58:44AM -0500, Joy Didier wrote:
>    Hi,
>    I think I've found a bug with getdata when reading gzip encoded channels
>    and using bitfields.
>    The attached dirfile was written using pygetdata and gzip encoding.
>    It has a 100 sample long int8 "raw_flag" channel that increments from 0 to
>    3, and then back to zero.
>    It also has a bitfield "flag0" corresponding to bit 0 of "raw_flag".
>    When trying to read the data, I see the following bugs:
>    * reading the data using pygetdata: the raw_flag channel reads properly,
>    but the bitfield channel reads incorrect values. The incorrect values are
>    different every time I read the data.
>    * reading the data using KST2: the raw_flag channel reads zeros, but the
>    bitfield channel reads proper values. If I delete the bitfield from the
>    format file, then KST2 reads the raw_flag channel properly.
>    If I gunzip the channel, and get rid of the ENCODING directive in the
>    format file, all the channels read properly through KST and pygetdata.
>    Thanks,
>    Joy

Thanks for the report, but I can't seem to reproduce the problem here with
0.8.5 and python (I haven't tried kst yet):

  $ ls -l test-dirfile
  total 8
  -rw------- 1 dvw dvw 263 Feb 19 17:09 format
  -rw------- 1 dvw dvw  35 Feb 19 17:09 raw_flag.gz  
  $ python
  Python 2.7.5 (default, May 29 2013, 03:26:28) 
  [GCC 4.8.0] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import pygetdata as gd
  >>> D = gd.dirfile('test-dirfile')
  >>> D.getdata('raw_flag')
  array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2,
         2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3,
         3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0, 0, 0], dtype=int8)
  >>> D.getdata('flag0')
  array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1,
         1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0, 0, 0], dtype=uint64)
  >>> gd.__version__
  (0, 8, 5, '')
  >>>

Can you read the data correctly with, say, dirfile2ascii, which uses the
C library directly?  What architecture are you running on?

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/