#343 longlong constructor and display lose digits due to implicit double precision conversions

[Chris Marshall]

From this report on the perldl list:

http://mailman.jach.hawaii.edu/pipermail/perldl/2013-October/008212.html

The problem is that in going between the actual longlong PDL object
data (which is correctly 64bit integers) and the conversion or display
there is a hidden translation through a double precision value which
only supports 52bits of integer precision.

This shows translation through double precision in longlong constructor:

pdl> p longlong(10555000100001145) - longlong(10555000100001144)
0

A work around the limitation showing the longlong works once created:

pdl> p longlong(1055500010000114)10 + 5 - longlong(1055500010000114)10 - 4
1

Also confusing because the printing of the values seems to go through
double precision as well:

pdl> p longlong(1055500010000114)*10 + 5
10555000100001144

pdl> p longlong(1055500010000114)*10 + 4
10555000100001144

We'll open a bug ticket to track this issue. It will be more important to
resolve now that PDL will be supporting 64bit indexing operations.

[Chris Marshall]

The problem occurs in two places affecting PDL.

First, the various get and set routines all assume that a double is sufficient to hold any PDL datatype. This is not true for longlong. Nor would it be true for enhanced data type support as planned for PDL3 since clearly arbitrary data types have no reason to be only 64bits long.

Second, the badvalue handling makes the same assumptions about the largest possible badvalue being for double with the exact same problem.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

#!/usr/bin/perl use strict; use warnings; use Inline qw/Pdlpp/; use PDL; my $longlong=zeroes(longlong,1); $longlong->inplace->check_pdl_longlong(); wcols('pdl_wcols: %lld',$longlong); printf(" perl: %lld\n",$longlong->at(0)); $longlong->check_pdl_longlong_write(); __DATA__ __Pdlpp__ pp_def('check_pdl_longlong', Pars => 'longlong [o] output(q)', Code => '$output(q=>0) = 10555000100001145; fprintf(stderr,"pp_assign: %lld\n",$output(q=>0));', ); pp_def('check_pdl_longlong_write', Pars => 'longlong input(q)', Code => 'fprintf(stderr," pp_write: %lld\n",$input(q=>0));', );

[Chris Marshall]

Based on a recent conversation at #pdl:
http://irclog.perlgeek.de/pdl/2014-07-24

The partial implementation of the PDL 64bit index support
is apparently a blocker for PDL to be included in the next
release of Debian Linux: Jessie with a freeze date
of 04-Nov-2014.

Here is a link to the Debian bug report:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=735623

The good news is that we have an offer from cbmuser to
help with some testing on the problematic hardware platforms.

[Chris Marshall]

This fix won't make the PDL-2.008 release timeframe

[Chris Marshall]

This fix will have to be deferred until after the PDL-2.008 release as the master and core-cleanup release on which this was based have diverged too far to resume. Will keep the LLDF branch as a reminder until the full fix is implemented.

[Chris Marshall]

Fixed in git master and will appear in PDL-2.014. Thanks to kmx for wrapping this one up.