SourceForge has been redesigned. Learn more.

#66 Binary buffer => PDL coercion


I was recently discussing methods for taking a Perl scalar filled with some sort of binary data (which I'll call a buffer) and shoving it into a piddle. This works fine if the data is one of the seven registered PDL data types, but obviously trouble arises if the data is some other format. For example, data that is represented as an unsigned char (i.e. unsigned byte) has not corresponding type in PDL, which means a simple copy of the data will not work. What does work is something like this (thanks to zpmorgan's GStreamer work for making this clear):

my $piddle = pdl (unpack('C*',$buf->data))->reshape(@dims);

I wonder if it might be possible to turn this into a function, such as coerce or some such, which basically allows you to specify the Perl unpack type along with the PDL type, so you could say:

my $piddle = coerce("C" => float, $buffer, @dims);

and, obviously, the reverse operation would be helpful, too:

my $buffer = coerce(float => "C", $piddle);

Obviously this could be made faster than using pack and unpack, but I suspect that it's pretty fast as written.


  • David Mertens

    David Mertens - 2012-01-16

    Wait, obviously the reverse operation would be a PDL method:

    $buffer = $piddle->coerce("C");

    Duh. :-)

  • David Mertens

    David Mertens - 2012-01-16

    zpmorgan suggested calling the method/function "finagle", since coerce is likely to conflict with other (non-PDL) modules. Plus it's funny.

  • Chris Marshall

    Chris Marshall - 2012-01-16

    Or we could add scalar buffers as possible
    inputs/outputs to readflex/writeflex. I think
    generalizing the pdll data type handling
    to allow arbitrary types and including all
    the standard C types would be good to have.
    Maybe a thought for the low level rework
    for PDL-2.5?

  • Chris Marshall

    Chris Marshall - 2012-03-21

    PDL does have byte data type which is unsigned char.
    It does not have plain ol' char so the example appears
    to be backwards as the problem would be in the
    other direction.

  • Chris Marshall

    Chris Marshall - 2012-09-02

    It would be nice if this could be made to work "inplace" if the input data buffer was compatible with a native PDL type. Maybe we could have something like a generic underlying byte piddle with a runtime conversion to the desired type for processing.
    That would allow efficient use of memory while supporting PDL operations.

  • Chris Marshall

    Chris Marshall - 2013-01-02
    • priority: 5 --> 2
  • Chris Marshall

    Chris Marshall - 2013-01-02

    Lowering priority to reflect reality as part of Bugs and Feature Requests review for an upcoming PDL-2.4.12 release---whenever that happens. :-)


Log in to post a comment.