#325 hang/segv after applying inplace transform

critical
closed-works-for-me
nobody
core (120)
9
2013-07-06
2013-03-13
Diab Jerius
No

% perldl -V
perlDL shell v1.354_001
PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
'COPYING' in the PDL distribution. This is free software and you
are welcome to redistribute it under certain conditions, see
the same file for details.

Summary of my PDL configuration

VERSION: PDL v2.4.11 (supports bad values)

$%PDL::Config = {
'BADVAL_PER_PDL' => '0',
'WITH_PROJ' => '1',
'PDL_CONFIG_VERSION' => '0.005',
'POSIX_THREADS_INC' => undef,
'FFTW_TYPE' => 'double',
'PDL_BUILD_DIR' => '/tmp/axafbin/mst/cpan/build/PDL-2.4.11-g1zr50',
'FFTW_LIBS' => undef,
'WITH_FFTW' => '1',
'GSL_LIBS' => undef,
'GL_BUILD' => '1',
'WITH_IO_BROWSER' => '0',
'PROJ_INC' => undef,
'WHERE_PLPLOT_INCLUDE' => undef,
'HTML_DOCS' => '1',
'SKIP_KNOWN_PROBLEMS' => '0',
'WHERE_PLPLOT_LIBS' => undef,
'WITH_3D' => '1',
'WITH_POSIX_THREADS' => '1',
'POGL_VERSION' => '0.65',
'FFTW_INC' => undef,
'HIDE_TRYLINK' => '1',
'HDF_INC' => undef,
'WITH_HDF' => '1',
'POGL_WINDOW_TYPE' => 'glut',
'WITH_GD' => '0',
'WITH_BADVAL' => '1',
'FITS_LEGACY' => '1',
'WITH_SLATEC' => '1',
'BADVAL_USENAN' => '1',
'WITH_DEVEL_REPL' => '1',
'TEMPDIR' => '/tmp',
'PROJ_LIBS' => undef,
'USE_POGL' => '1',
'PDL_BUILD_VERSION' => '2.4.11',
'GD_LIBS' => undef,
'GSL_INC' => undef,
'GD_INC' => undef,
'WITH_GSL' => '1',
'OPTIMIZE' => undef,
'PDLDOC_IGNORE_AUTOLOADER' => '0',
'HDF_LIBS' => undef,
'POSIX_THREADS_LIBS' => undef,
'MALLOCDBG' => {},
'WITH_MINUIT' => '1',
'WITH_PLPLOT' => '1',
'MINUIT_LIB' => undef
};
Summary of my perl5 (revision 5 version 12 subversion 2) configuration:

Platform:
osname=linux, osvers=2.6.32-5-amd64, archname=x86_64-linux_debian-6.0-thread-multi
uname='linux macabre 2.6.32-5-amd64 #1 smp thu mar 22 17:26:33 utc 2012 x86_64 gnulinux '
config_args='-de -Dprefix=/proj/axaf/ots/pkgs/perl-5.12/x86_64-linux_debian-6.0 -Uinstallusrbinperl -Dperladmin=none -Dstartperl=#!/usr/bin/env /proj/axaf/bin/perl -Dlocincpth=/proj/axaf/ots/include -Darchname=x86_64-linux_debian-6.0 -Dusethreads -Dloclibpth=/proj/axaf/ots/x86_64-linux_debian-6.0/lib -Dglibpth=/lib /usr/lib -Dldflags=-L/proj/axaf/ots/x86_64-linux_debian-6.0/lib -Wl,-rpath=/proj/axaf/ots/x86_64-linux_debian-6.0/lib -Duseshrplib'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/proj/axaf/ots/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/proj/axaf/ots/include'
ccversion='', gccversion='4.4.5', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='cc', ldflags ='-L/proj/axaf/ots/x86_64-linux_debian-6.0/lib -Wl,-rpath=/proj/axaf/ots/x86_64-linux_debian-6.0/lib -fstack-protector'
libpth=/proj/axaf/ots/x86_64-linux_debian-6.0/lib /lib /usr/lib
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.11.3.so, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.11.3'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/proj/axaf/ots/pkgs/perl-5.12/x86_64-linux_debian-6.0/lib/5.12.2/x86_64-linux_debian-6.0-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/proj/axaf/ots/x86_64-linux_debian-6.0/lib -fstack-protector'

The attached code snippet hangs when run.

% perl tpdl.pl
....wait....wait...
^C

When run under the debugger, it SEGV's.

% perl -d tpdl.pl

Loading DB routines from perl5db.pl version 1.33
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

main::(tpdl.pl:4): my $c = pdl( double, 49, 49 );

DB<1> c
Signal SEGV at tpdl.pl line 10.
Aborted
%

In more complex production code, it SEGV's when run.

I've rewritten the code to not apply the transform inplace (see attached tpdl1.pl). Note the output:

% perl tpdl1.pl
PDL: PDL::Ops::plus(a,b,c): Parameter 'b':
Mismatched implicit thread dimension 0: should be 2, is 3
at tpdl1.pl line 9.
%

Discussion

  • Diab Jerius
    Diab Jerius
    2013-03-13

    Oops. Hit the wrong button; and mangled the report. The second file (tpdl1.pl) follows. [Can't seem to attach another file]

    use PDL;
    use PDL::Transform;

    my $c = pdl( double, 49, 49 );
    my $t = t_linear( { scale => pdl( [ 1, 3 ] ), offset => pdl( [ 12, 8 ] ) } );

    $q = pdl( double, 2,2, 9.3 );

    $c = $c->apply($t) + $q;

    print $c;

     
  • Diab Jerius
    Diab Jerius
    2013-03-13

    Don't use inplace transform

     
    Attachments
  • Diab Jerius
    Diab Jerius
    2013-03-13

    Use inplace transform

     
    Attachments
  • Craig DeForest
    Craig DeForest
    2013-03-13

    User error. $c (and the output of apply) is a 2-PDL; $q is a 3-PDL.
    Try

    $q = pdl(double,2,9.3)

    or

    $q = pdl(double(2,2,9.3)->(*1))

     
  • Chris Marshall
    Chris Marshall
    2013-03-13

    Running PDL-2.004_995 I get the following in the pdl2 shell:

    pdl> use PDL;
    pdl> use PDL::Transform;
    pdl>
    pdl> my $c = pdl( double, 49, 49 );
    pdl> my $t = t_linear( { scale => pdl( [ 1, 3 ] ), offset => pdl( [ 12, 8 ] ) }
    > );
    pdl>
    pdl> $q = pdl( double, 2,2, 9.3 );
    pdl>
    pdl> $c = $c->apply($t) + $q;
    Runtime error: PDL: PDL::Ops::plus(a,b,c): Parameter 'b':
    Mismatched implicit thread dimension 0: should be 2, is 3
    at /cygdrive/f/perl/local/lib/perl5/cygwin-thread-multi-64int/PDL/Complex.pm line 1222.
    PDL::cp('PDL=SCALAR(0x821332f8)', 'PDL=SCALAR(0x82136eb8)', '') called at (eval 471) line 7

    which may be related to this problem. I know there
    were some fixes related to complex constructors in
    the latest PDL git/release candidates.

     
  • Diab Jerius
    Diab Jerius
    2013-03-13

    I'm more concerned with the SEGV / hang than the user error. The latter I can fix (usually, depends upon how hard I kick myself).

     
  • Chris Marshall
    Chris Marshall
    2013-03-13

    Do you still get the SEGV/hang with the correct code?

     
  • Chris Marshall
    Chris Marshall
    2013-03-13

    Also, how is the linear transform expected
    to be meaningful against a shape [2] piddle?

     
  • Diab Jerius
    Diab Jerius
    2013-03-13

    A) I corrected the subtle typo (2,2 => 2.2) and still get a SEGV. I can make it segv w/o the debugger if I do this:

    $q = pdl( double, [2.2, 9.3] );

    B) I tried with perl 5.16.2 & PDL 2.4.11 (essentially same config for the latter) with similar results

    C) Do not read meaning into this code. It is a boiled down mix of my test suite's fuzzed input and the production code. Of course, the errant typo was a bug in the test suite, but still...

     
  • Chris Marshall
    Chris Marshall
    2013-03-13

    Could you please reply with a correct code that reproduces
    the problem? As I mentioned already, applying a 2D
    transformation to a linear pdl (i.e., $c is a shape [2]) doesn't
    seem valid to me. At the least, you seem to be stretching
    the limits of DWIM boundary handling with an offset of [12,8].
    Even the scale of [1,3] doesn't help as 1*2 is 2 and 3*0 is 0.

     
  • Diab Jerius
    Diab Jerius
    2013-03-13

    Well, a SEGV is a SEGV. It just shouldn't happen, unless I'm intentionally poking at the innards, and I'm not.

    I'm not trying to be pedantic here, but the code I've provided is sufficient to cause a catastrophic runtime failure, and that's the point of the bug report.

    As for whether I'm using the apply routine correctly, here's the documentation:

    The apply method accepts a PDL whose 0th dimension is coordinate index (all other dimensions are threaded over).

    I'm applying a 2D transform to a single 2D vector. As there's only a single 2D vector, there's no threading needed, so no extra dimensions required, either. If you'd prefer, I can add the useless dummy dimension:

    use 5.10.0;

    use PDL;
    use PDL::Transform;

    my $c = pdl( double, [[49, 49]] );
    my $t = t_linear( { scale => pdl( [ 1, 3 ] ), offset => pdl( [ 12, 8 ] ) } );

    $c->inplace->apply( $t );
    $q = pdl( double, 2.2, 9.3 );

    $c += $q;

    say $c;

    and it still segvs.

    If I do not use inplace transformation, it works fine, with or without the dummy dimension:

    use 5.10.0;

    use PDL;
    use PDL::Transform;

    my $t = t_linear( { scale => pdl( [ 1, 3 ] ), offset => pdl( [ 12, 8 ] ) } );

    $q = pdl( double, 2.2, 9.3 );

    {
    my $c = pdl( double, [49, 49] );
    $c = $c->apply($t) + $q;
    say $c;
    }

    {
    my $c = pdl( double, [[49, 49]] );
    $c = $c->apply($t) + $q;
    say $c;
    }

     
  • Craig DeForest
    Craig DeForest
    2013-03-13

    Yep. apply() is being used correctly and I can reproduce the crash. Working on it now. Nice catch, Diab -- sorry if I came across as short earlier!

     
  • Diab Jerius
    Diab Jerius
    2013-03-13

    No problem! Thanks!

     
  • Craig DeForest
    Craig DeForest
    2013-03-13

    This is not a Transform problem at all, except that Transform stimulates a bug in the inplace code.

    I can reproduce the crash with:

    pdl> $a = pdl(1); $a->inplace; $a += 2;
    Segmentation fault

     
  • Craig DeForest
    Craig DeForest
    2013-03-13

    • priority: 5 --> 9
    • milestone: --> critical
    • labels: --> core
     
  • Chris Marshall
    Chris Marshall
    2013-03-13

    Sorry, I was reading 'apply' as 'map'.

     
  • Craig DeForest
    Craig DeForest
    2013-03-13

    I fixed this in Transform by clearing the inplace flag explicitly after apply() calls; but I will submit a followon Core bug about the inplace flag.

     
  • Craig DeForest
    Craig DeForest
    2013-03-13

    • status: open --> open-fixed
     
  • Chris Marshall
    Chris Marshall
    2013-07-06

    The examples don't SEGV for PDL-2.006 and the status was listed as open-fixed. Closing the ticket. Thanks for reporting the problem.

     
  • Chris Marshall
    Chris Marshall
    2013-07-06

    • status: open-fixed --> closed-works-for-me