#236 pdl() constructor crashes perl for mixed ref/piddle args

critical
closed-fixed
core (120)
9
2011-03-27
2010-06-05
No

$c[0][0]=pdl(0,4,2,1);
$c[1][0]=pdl(0,0,1,1);
$c[2][0]=pdl(0,0,0,1);
$c[0][1]=pdl(0,0,3,1);
$c[1][1]=pdl(0,0,2,1);
$c[2][1]=pdl(5,1,1,1);
$d = pdl(@c);

Causes a crash in PDL-2.4.6_008. A preliminary look shows
that the crash happens in the pdl_avref routine. This problem
was reported by Jim Magnuson on the perldl mailing list.

Thanks, Jim!

Discussion

1 2 > >> (Page 1 of 2)
  • Chris Marshall

    Chris Marshall - 2010-06-06
    • priority: 9 --> 5
     
  • Chris Marshall

    Chris Marshall - 2010-06-06

    It looks like a documentation bug, first. Then, one should
    not get a core dump from mixed arg inputs. Eventually,
    it would be good to handle the more general input case.

     
  • Chris Marshall

    Chris Marshall - 2010-08-01

    This is the highest priority outstanding PDL bug to
    be resolved before a PDL-2.4.7 release is made.

     
  • Chris Marshall

    Chris Marshall - 2010-08-01
    • milestone: --> critical
    • summary: pdl() constructor crashes for mixed array ref/piddle args --> pdl() constructor crashes perl for mixed ref/piddle args
    • priority: 5 --> 9
     
  • Chris Marshall

    Chris Marshall - 2010-08-01
    • assigned_to: nobody --> zowie
     
  • Chris Marshall

    Chris Marshall - 2010-08-10
    • priority: 9 --> 3
    • status: open --> open-postponed
     
  • Chris Marshall

    Chris Marshall - 2010-08-10

    The bug reported here was more extensive than originally
    thought. I've marked this bug postponed as the final bit
    of triage in preparation for our PDL-2.4.7 release. The
    priority has also been lowered accordingly to reflect the
    re-prioritization of this ticket w.r.t. the PDL-2.4.7 release.

     
  • Chris Marshall

    Chris Marshall - 2010-09-28

    With the latest git, the core.t no longer crashes perl.
    However, it does not pass all in t/core.t:

    not ok 25 - Piddlifing two piddles catenates them and pads to length
    # Failed test 'Piddlifing two piddles catenates them and pads to length'
    # at core.t line 118.
    not ok 27 - Can catenate mixed-dim piddles: hairy case
    # Failed test 'Can catenate mixed-dim piddles: hairy case'
    # at core.t line 127.
    not ok 28 - undefval works for padding
    # Failed test 'undefval works for padding'
    # at core.t line 133.

    If I replace the ok all() tests with ok all(tapprox())) equivalent
    then all tests pass execpt for the last. In the perldl shell, here
    are the values created by the constructor for the 3 failing tests:

    pdl> $a = pdl( zeroes(5), ones(3) );

    pdl> p $a

    [
    [ 0 0 0 0 0]
    [ 1 1 1 1.0981569e-300 1.821236e-296]
    ]

    pdl> $c = pdl [1], pdl[2,3,4], pdl[5];

    pdl:/cygdrive/f/chm/pdl/git/pdl/t> p $c

    [
    [
    [1 0 0]
    [0 0 0]
    ]
    [
    [ 2 3 4]
    [ 5 0 1.8321365e-296]
    ]
    ]

    pdl> do { local($PDL::undefval) = 99; $c = pdl [1], pdl[2,3,4], pdl[5]; }

    pdl> p $c

    [
    [
    [ 1 99 99]
    [99 99 99]
    ]
    [
    [ 2 3 4]
    [ 5 99 1.820088e-296]
    ]
    ]

    I'm not sure why the value for zero is not identically zero
    but clearly, the answer is correct withing floating point
    tolerances so it may be some additional conversion or
    type coercion that is taking place.

    In the third test, the last value is not being converted to
    the 99 undefval.

     
  • Chris Marshall

    Chris Marshall - 2010-10-05

    The CPAN Testers reports from the PDL-2.4.7_002 developers
    release show failures for subtests 25, 27-28 on all platforms....
    No crashes though, as far as I can tell.

     
  • Chris Marshall

    Chris Marshall - 2010-10-07

    Here is an odd datum: I just tried running the latest t/core.t
    using a PDL-2.4.7_001 install and it passes. Running the
    test with perl -Mblib t/core.t fails in the same way as other
    platforms. I don't see anything in the git log that could
    suggest why it worked..... I'll have to try it out on a clean
    install.

     
  • Chris Marshall

    Chris Marshall - 2010-11-07

    Fixed in git and CPAN releases 2.4.7_003 and higher. Marked this ticket pending to close automatically in 2 weeks unless further issues are noted. Thanks for the debug and fix, Craig.

     
  • Chris Marshall

    Chris Marshall - 2010-11-07
    • status: open-postponed --> pending-fixed
     
  • Derek Lamb

    Derek Lamb - 2010-11-12

    make test output before constructor bug fix

     
  • Derek Lamb

    Derek Lamb - 2010-11-12

    make test output after constructor bug fix

     
  • Derek Lamb

    Derek Lamb - 2010-11-12

    plplot test output after constructor bug fix

     
  • Derek Lamb

    Derek Lamb - 2010-11-12

    The git commit that fixed the constructor bug had some unintended consequences. In particular, t/plplot.t now fails with a 'perl: malloc(): memory corruption' error. The commands the the results of the various make test steps are:

    $ git checkout a3a95e43ff79d4ec4bd86f7e338f32ed1cce84b7
    $ make realclean; perl Makefile.PL; make;
    $ make test 2>&1 | tee ~/test.good.txt
    That's for before the constructor fix: core.t fails, but plplot.t is passes.

    $ git checkout c1e833bfc3da7e61589c973c077e8c82d6dfc36a
    $ make realclean; perl Makefile.PL; make;
    $ make test 2>&1 |tee ~/test.bad.txt
    That's for after the constructor fix: core.t passes, but plplot.t fails.

    The output of
    $ perl -Mblib t/plplot.t
    is attached as test.plplot.txt

     
  • Derek Lamb

    Derek Lamb - 2010-11-12
    • status: pending-fixed --> open-fixed
     
  • Chris Marshall

    Chris Marshall - 2010-11-12

    I hope someone who has PLplot+PDL can track
    this down. I've never gotten PLplot to build on
    cygwin or to work with PDL if built. On my list to
    try but it definitely doesn't just "build out of
    the box" at the moment.

    I did notice that there is another PLplot related
    ticket #3080505. Perhaps the two problems are
    related?

     
  • Chris Marshall

    Chris Marshall - 2010-11-14
    • status: open-fixed --> open
     
  • Chris Marshall

    Chris Marshall - 2010-11-14

    Upgrading a linux system to PDL-2.4.7_006 I saw the
    same failure in t/plplot.t as was reported here.

    Along the same line, the latest CPAN Testers report
    for 2.4.7_006 have a case for dragonfly BSD where
    two tests fail in t/fft.t:

    http://www.cpantesters.org/cpan/report/e19e9bac-efbb-11df-bb29-ad544afd17af

    I don't have any specific reason to associate this
    failure with the PDL malloc report here but it is
    equally mysterious.

     
  • Chris Marshall

    Chris Marshall - 2010-12-06
    • priority: 3 --> 9
     
  • Derek Lamb

    Derek Lamb - 2010-12-07

    Attached is a log of the entire build process, along with a small test and valgrind output of that test (near the end). This is using the current git (commit hash is output at the top). In particular, I've isolated it to two calls-- plgvpd() and plgvpw(), which are low-level commands in the plplot package. They return a 4-element perl list of values (in this case zeroes), but the constructor fails when trying to turn it into a piddle. The "@wc = map{$_+0} @world_coords;" is in there because Craig thought there might be an issue with magic, but that does not seem to fix the problem.

     
  • Chris Marshall

    Chris Marshall - 2010-12-08

    Frank Boers reported this problem on the perldl list and responded to me. Here is his response text and the attachment are added below:

    Dear Chris,
    thanks for your quick response.

    best regards
    Frank

    >> >>Could you try running it in a script to report
    >> >>a full traceback? (It works the system I have
    >> >>here, but that is 32bit OS with cygwin perl).
    Yes I attached the script <pdl_uniq_check> but I got only the
    <floating point error> out,
    the same as calling from PDL.

    >> >>Could you also check if there are any other
    >> >>copies of PDL on the system that could be
    >> >>pulling in an earlier PDL?
    Yes I installed perl 5.12.2 and PDL 2.4.7_007 from scratch, please have
    look to Makefile.out, make.out and test.out

    My system:
    Linux 2.6.31.12-0.2-desktop x86_64
    System: openSUSE 11.2 (x86_64)
    KDE: 4.3.5 (KDE 4.3.5) "release 0"

    uname -a
    Linux megpc26 2.6.31.12-0.2-desktop #1 SMP PREEMPT 2010-03-16 21:25:39
    +0100 x86_64 x86_64 x86_64 GNU/Linux

    >> >> perl -V
    Summary of my perl5 (revision 5 version 12 subversion 2) configuration:

    Platform:
    osname=linux, osvers=2.6.31.12-0.2-desktop,
    archname=x86_64-linux-thread-multi
    uname='linux megpc26 2.6.31.12-0.2-desktop #1 smp preempt 2010-03-16
    21:25:39 +0100 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dprefix=/data/meg_store2/megmip01/meg/local64/perl5122
    -Dusethreads'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
    Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
    -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
    -D_FILE_OFFSET_BITS=64',
    optimize='-O3',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
    -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.1 [gcc-4_4-branch revision 150839]',
    gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
    lseeksize=8
    alignbytes=8, prototype=define
    Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/data/meg_store2/megmip01/meg/local64/lib
    /data/meg_store2/megmip01/meg/local64/lib64 /usr/local/lib /lib /usr/lib
    /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.10.1.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.10.1'
    Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O3 -L/usr/local/lib
    -fstack-protector'

    Characteristics of this binary (from libperl):
    Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
    USE_64_BIT_ALL
    USE_64_BIT_INT USE_FAST_STDIO USE_ITHREADS
    USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF
    USE_REENTRANT_API
    Built under linux
    Compiled at Dec 8 2010 13:35:47
    %ENV:

    PERL5LIB="/data/meg_store2/megmip01/meg/perl/lib:/data/meg_store2/megmip01/meg/perl/lib/CPAN"
    PERL_DEV="/data/megmip01/meghome/fboers/programs/perl/lib"
    PERL_DEV_LIB="/data/megmip01/meghome/fboers/programs/perl/lib"
    PERL_DEV_SOURCE="/data/megmip01/meghome/fboers/programs/perl/source"
    PERL_INSTALL="/data/meg_store2/megmip01/meg/local64"
    @INC:
    /data/meg_store2/megmip01/meg/perl/lib
    /data/meg_store2/megmip01/meg/perl/lib/CPAN

    /data/meg_store2/megmip01/meg/local64/perl5122/lib/site_perl/5.12.2/x86_64-linux-thread-multi
    /data/meg_store2/megmip01/meg/local64/perl5122/lib/site_perl/5.12.2

    /data/meg_store2/megmip01/meg/local64/perl5122/lib/5.12.2/x86_64-linux-thread-multi
    /data/meg_store2/megmip01/meg/local64/perl5122/lib/5.12.2

     
  • Chris Marshall

    Chris Marshall - 2010-12-08

    Attachments to Frank Boers report

     
1 2 > >> (Page 1 of 2)

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks