Re: [q-lang-users] Two proposed improvements to Q

[Albert G. <Dr....@t-...>]

John Cowan wrote:
>>> Following that, byte vectors (mutable byte strings) would also be
>>> a useful addition for dealing with large quantities of homogeneous
>>> data; these have to be done in C, but would be far more efficient
>>> than any alternative representation.
>> Yes, actually I've had something like this on my TODO list for a while,
>> in order to provide better support for numeric and signal processing
>> applications. 
> 
> The proposed R6RS Scheme bytevectors provide read-write access to
> any point in a bytevector as signed and unsigned {8,16,32}-bit values,
> single and double floats, and strings using a specified character code.

Ok, at long last I decided to give this a go. It's is in cvs now. Here's
the relevant blurb from qdoc.info:

Byte Strings as Mutable C Vectors
---------------------------------

As of Q 7.11, `clib' supports a number of additional operations which
allow you to treat byte strings as mutable C vectors of signed/unsigned
8/16/32 bit integers or single/double precision floating point numbers.
The following functions provide read/write access to the elements of
such C vectors. Note that the given index argument `I' is interpreted
relative to the corresponding element type. Thus, e.g., `get_int32 B I'
returns the `I'th 32 bit integer rather than the integer at byte offset
`I'.

   NOTE: Integer arguments must fit into machine integers, otherwise
these operations will fail. Integers passed for floating point
arguments will be coerced to floating point values automatically.

     public extern get_int8 B I, get_int16 B I, get_int32 B I;
     public extern get_uint8 B I, get_uint16 B I, get_uint32 B I;
     public extern get_float B I, get_double B I;

     public extern put_int8 B I X, put_int16 B I X, put_int32 B I X;
     public extern put_uint8 B I X, put_uint16 B I X, put_uint32 B I X;
     public extern put_float B I X, put_double B I X;

   Moreover, the following convenience functions are provided to convert
between byte strings and lists of integer/floating point elements.

     public extern int8_list B, int16_list B, int32_list B;
     public extern uint8_list B, uint16_list B, uint32_list B;
     public extern float_list B, double_list B;

     public extern int8_vect Xs, int16_vect Xs, int32_vect Xs;
     public extern uint8_vect Xs, uint16_vect Xs, uint32_vect Xs;
     public extern float_vect Xs, double_vect Xs;

---

And a few examples:

     ==> def B = uint32_vect [100..110]

     ==> B
     <<ByteStr>>

     ==> uint32_list B
     [100,101,102,103,104,105,106,107,108,109,110]

     ==> get_uint32 B 1
     101

     ==> put_uint32 B 1 0xffffffff
     ()

     ==> uint32_list B
     [100,4294967295,102,103,104,105,106,107,108,109,110]

     ==> take 12 $ int8_list B
     [100,0,0,0,-1,-1,-1,-1,102,0,0,0]

     ==> float_vect [1..10]
     <<ByteStr>>

     ==> float_list _
     [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0]

Please note that this is just the "bare bones" C interface, but building
higher-level Q APIs on top of that should be a piece of cake now.

John, I hope that this is what you had in mind. It still lacks the "...
and strings using a specified character code" part, though. Considering
that a byte string might actually contain character data in an arbitrary
encoding, it's not clear to me how that should be done, could you
elaborate please?

Eddie, do you think that this will be good enough for the qcalc stats
stuff and GSL interface we talked about a while ago? Maybe for
efficiency there should be an additional function in your forthcoming
CSV module to directly convert between numeric data in CSV format and C
vectors as byte strings?

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag