#421 PDL::IO::FITS can't handle 64-bit integers (longlong, indx)
Writing a 64-bit longlong or indx piddle with wfits() succeeds, but produces a bogus FITS header "card" of the form:
'BITPIX '
~~~
resulting in `$foo = \%hdrhash = { ..., BITPIX=>undef, ... }` at PDL/IO/FITS.pm line 447.
wfits() on a 32-bit long gives us:
'BITPIX = 32 '
rsp. `{ ..., BITPIX=>32, ... }` at the same line.
The resulting files cannot be read successfully with `rfits()`:
!/usr/bin/perl
use PDL;
use PDL::IO::FITS;
my $p = zeroes(longlong,10); ##-- also indx()
$p->wfits('broken.fits');
my $q = rfits('broken.fits'); ##-- ERROR: rfits: strange BITPIX value in header - I give up!
The attached patch (vs PDL-2.016) fixes the immediate problem by treating all 64-bit integers as `longlong`, which doesn't preserve the indx-vs-longlong distinction, but at least does preserve the allowable data range, and should also work on systems with a 32-bit indx type, treating these as `long`. output of perldl -V:
perlDL shell v1.357
PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
'COPYING' in the PDL distribution. This is free software and you
are welcome to redistribute it under certain conditions, see
the same file for details.
Summary of my PDL configuration
VERSION: PDL v2.016 (supports bad values)
$%PDL::Config = {
'BADVAL_PER_PDL' => '0',
'WITH_PROJ' => '1',
'PDL_BUILD_DIR' => '/home/moocow/local/src/pdl/PDL-2.016',
'GD_DEFINE' => '',
'WITH_POSIX_THREADS' => '1',
'PROJ_LIBS' => undef,
'POSIX_THREADS_LIBS' => undef,
'GSL_LIBS' => undef,
'WITH_SLATEC' => '1',
'GD_LIBS' => undef,
'PDL_BUILD_VERSION' => '2.016',
'WITH_IO_BROWSER' => '0',
'WITH_GD' => '1',
'POGL_WINDOW_TYPE' => 'glut',
'PDL_CONFIG_VERSION' => '0.005',
'GSL_INC' => undef,
'BADVAL_USENAN' => '0',
'POGL_VERSION' => '0.6702',
'WITH_GSL' => '1',
'SKIP_KNOWN_PROBLEMS' => '0',
'PDLDOC_IGNORE_AUTOLOADER' => '0',
'GL_BUILD' => '1',
'PROJ_INC' => undef,
'HTML_DOCS' => '1',
'OPTIMIZE' => undef,
'HDF_INC' => undef,
'TEMPDIR' => '/tmp',
'WITH_MINUIT' => '1',
'WITH_HDF' => '1',
'USE_POGL' => '1',
'WITH_DEVEL_REPL' => '1',
'HDF_LIBS' => undef,
'MALLOCDBG' => {
'libs' => '',
'include' => ''
},
'FITS_LEGACY' => '1',
'WITH_3D' => '1',
'POSIX_THREADS_INC' => undef,
'HIDE_TRYLINK' => '1',
'MINUIT_LIB' => undef,
'GD_INC' => undef,
'WITH_BADVAL' => '1'
};
Summary of my perl5 (revision 5 version 20 subversion 2) configuration:
Platform:
osname=linux, osvers=3.2.0-4-amd64, archname=x86_64-linux-gnu-thread-multi
uname='linux x86-csail-01 3.2.0-4-amd64 #1 smp debian 3.2.68-1+deb7u1 x86_64 gnulinux '
config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.20 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.2 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.20.2 -des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -g',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
perllibs=-ldl -lm -lpthread -lc -lcrypt
libc=libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so.5.20
gnulibc_version='2.19'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'
~~~
apologies for formatting hiccups!
This happened because FITS.pm has its own hardwired BITPIX <=> type conversions, and they hadn’t been updated for 64 bit support.
I fixed the bug, but I’m having trouble with the current workflow (old dog, new tricks I guess).
I’m attaching the fixed FITS.pm.
Related
Bugs:
#421thanks for the speedy reply; I didn't see an attachment though -- is there something amiss with my patch? In particular, I'm not sure if $PDL_LL is guaranteed to be defined and 64 bits wide ... if not, some additional bookkeeping will likely be required.
marmosets,
Bryan
Sorry, although you did mention a patch the link didn't work for me so I just banged out a solution. The link works fine now...
I think I like your solution better, though. I hammered all outbound pdl_indx variables into long longs, while you took the time to find the smallest int that fit. I will put yours in with another commit once I get back to my desktop.
(Mobile)
Related
Bugs:
#421OK, I think I got the workflow right this time, so it should merge OK.
I included your fix with your PDL_INDX logic, but replaced the “-263, 263-1” code with actual 64-bit PDL values constructed with bit shifts, since (263==263-1) evaluates TRUE on 64 bit machines.
Related
Bugs:
#421excellent, thanks!