#43 add IO by records to PDL::IO::FlexRaw

[Chris Marshall]

Data files often contain sequences of images, frames, spectra, matrices,... which need to be slurped in
for PDL to process. At the moment, you have to calculate the number of records that are in the file
from the file size before you can issue a readflex command.

The enhancement would be to allow the last dimension size to be unspecified (maybe undef?) and then
readflex would read the file until EOF and determine the final dimension at read time itself. E.g.

$video = readflex('frames.raw', [ { Type=>'byte', NDims=>4, Dims=>[4,640,480,undef] } ]);

instead of

$size = -s 'frames.raw';
$nframes = $size / (4*640*480);
$video = readflex('frames.raw', [ { Type=>'byte', NDims=>4, Dims=>[4,640,480,$nframes] } ]);

[David Mertens]

In addition, it would be nice to have the functions 'glueflex' and 'gluefraw', which would append a 'frame' to the end of the binary file, adjusting the header appropriately. I've coded this functionality in some of my scripts by hand and it's very useful. It allows me to accumulate data slowly over the course of a long-running program, and if the system goes down or if I have to kill it for some reason, I still have partial results. I can edit the script to have it pick-up where it left off.

[Chris Marshall]

Good idea and a nice complement to the read-by-records capability.

[Chris Marshall]

The ability to handle strided IO would be nice here as well.
Perhaps the input specification might be done with slice,
nslice,...syntax.

[Chris Marshall]

Bumping priority since it looks like a number of PDL::IO::FlexRaw enhancements could be completed for the next PDL release.

[mohawk]

Tracked at https://github.com/PDLPorters/pdl/issues/322, fixed