Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#148 pdl constructor undefined behaviour

closed
nobody
core (120)
5
2006-07-19
2006-07-17
Bill Coffman
No

pdl constructor has undefined behaviour for list of
pdls of different dimensionality.

perldl> print pdl zeroes(5), zeroes(10)
*** glibc detected *** malloc(): memory corruption:
0x08632b98 ***
Aborted

Should die more gracefully.

Discussion

  • Bill Coffman
    Bill Coffman
    2006-07-17

    Logged In: YES
    user_id=1538801

    Oops! the above actually works, which is, in a way worse,
    but this one fails.

    perldl> print pdl zeroes(100), zeroes(10)
    *** glibc detected *** malloc(): memory corruption:
    0x08632b98 ***
    Aborted

     
  • Bill Coffman
    Bill Coffman
    2006-07-17

    Logged In: YES
    user_id=1538801

    I think the prescribed behavior is to zero fill the shorter
    of the vectors, when the pdl c'tor is given a list.
    You can get interesting combinations of failures and successes:

    print pdl([(0)x13], zeroes 9); # fails
    print pdl([(0)x13], zeroes 1); # kind of succeeds
    print pdl([(0)x11], zeroes 10); # fails
    print pdl(zeroes(11),[(0)x10]); # succeeds
    print pdl([(0)x9], zeroes 10); # succeeds
    print pdl(zeroes(99),zeroes(98)); # fails
    print pdl(zeroes(98),zeroes(99)); # succeeds
    print pdl(zeroes(6),[(0)x5]); # succeeds

    At least that's for i386, ubuntu, glibc 2.3.6

    We might want to rethink the default behaviour of the c'tor.
    At least one user thought 1D lists would append.

     
  • Craig DeForest
    Craig DeForest
    2006-07-18

    Logged In: YES
    user_id=20200

    Are you using the current CVS? The constructor follows the prescribed behavior
    on my systems -- all the examples you list yield zero-padded PDLs with no
    corruption. I just tested on MacOS X (PPC), Fedora (x86-32), and Fedora
    (x86-64).

     
  • Chris Marshall
    Chris Marshall
    2006-07-18

    Logged In: YES
    user_id=44920

    This works for current 2.4.2cvs code on cygwin/XP(32bit)

     
  • Craig DeForest
    Craig DeForest
    2006-07-18

    Logged In: YES
    user_id=20200

    Okay, I figured out why I couldn't reproduce the problem:
    I've been using a padded debugging malloc (for another
    module that I'm developing for another project) and the
    padding was sufficient to prevent the memory overrun from
    showing.

    The problem was in pdl_setav_$type, in the way it was
    calling pdl_setzero_$type. pdl_setzero_$type sets an entire
    subspace region of the target PDL to undefval (zero by
    default). It requires a dimension index to know which
    dimensions to thread over. The PDL-handling code in
    pdl_setav_$type did not increment the dimension when calling
    pdl_setzero_$type.

    The result is that (in the pdl( zeroes(100), ones(10))
    example) pdl_setzero_$type calculated that it needed to
    generate 180 zeroes rather than 90 zeroes to pad out the
    last element, overrunning the target PDL.

    The bug would only be apparent if the last top-level element
    of the PDL under construction needed padding -- otherwise
    the overrun would simply write over part of the data space
    that then got overwritten again with valid data -- so that
    it was not apparent in, e.g., pdl( zeroes(10), ones(100))
    because the extra 90 zeroes from the first element would be
    overwritten with ones from the second.

    So much trouble from one little "+1"...

     
  • Craig DeForest
    Craig DeForest
    2006-07-18

    • status: open --> closed-fixed
     
  • Bill Coffman
    Bill Coffman
    2006-07-18

    • status: closed-fixed --> open
     
  • Bill Coffman
    Bill Coffman
    2006-07-18

    Logged In: YES
    user_id=1538801

    This is for CVS. I just updated again today and rebuilt and
    installed, to verify, and the failures still occur for me,
    as they did for the user who reported this ("David Whysong"
    <dwhysong@gmail.com>, Subject: "[Perldl] Strange problem
    (internal bug?)").

    You might want to try more extreme examples to reproduce,
    perldl> $z=pdl(zeroes(100000),zeroes(10));

    $ uname -a
    Linux zulu 2.6.15-25-686 #1 SMP PREEMPT Wed Jun 14 11:34:19
    UTC 2006 i686 GNU/Linux

    Ubuntu 6.06, glibc 2.3.6 perl was compiled with 2.3.5
    however, and cpu info:
    $ cat /proc/cpuinfo
    processor : 0
    vendor_id : GenuineIntel
    cpu family : 15
    model : 4
    model name : Intel(R) Celeron(R) CPU 2.53GHz
    stepping : 1
    cpu MHz : 2527.126
    cache size : 256 KB
    fdiv_bug : no
    hlt_bug : no
    f00f_bug : no
    coma_bug : no
    fpu : yes
    fpu_exception : yes
    cpuid level : 3
    wp : yes
    flags : fpu vme de pse tsc msr pae mce cx8 apic
    sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr
    sse sse2 ss ht tm pbe nx pni monitor ds_cpl tm2 cid xtpr
    bogomips : 5059.11

    Finally, my .perldl.conf
    $ cat ~/.perldl.conf
    # -*-perl-*-

    # PDL Configuration options

    # You can edit this here or say 'perl Makefile.PL PDLCONF=file'
    # or use ~/.perldl.conf

    # Note in general "0" means False, "1" means "True" and "undef"
    # means "Try if possible (e.g. because the library is found)"
    #
    # You can also use a string that matches /^y/i to mean True or
    # one that matches /^n/i to mean False. It will be
    automatically
    # converted to 1 or 0 before being loaded into the Config.pm
    module.
    #

    %PDL_CONFIG = (
    #
    # Do we generate HTML documentation? This is normally a
    good idea,
    # as it's nice to browse -- but many folks don't use it,
    preferring
    # the man page and/or help versions of the documentation.
    Undef or 1
    # causes the HTML documentation to be built; 0 turns it off.
    #
    HTML_DOCS => 1,

    # Location of directory for temporary files created during the
    # build/test process. See the getpdl_config() routine in
    Makefile.PL
    # for the choice made if TEMPDIR is left as 'undef': it
    boils down to
    # the first value that is defined from
    # $TEMP, $TMP, or "/tmp" [a TEMP directory for MSWin users]
    #

    TEMPDIR => undef,

    # Decides if the output of attempts to link various function
    # during 'perl Makefile.PL' will be hidden when building PDL
    # should only be set to 0 for debugging purposes
    # see also L<trylink|PDL::Core::Dev/trylink>

    HIDE_TRYLINK => 1,

    # you can set machine specific optimizations here
    # the settings will be passed to the toplevel Makefile.PL
    # which *should* pass it to any recursively invoked ones

    OPTIMIZE => undef, # '-g',

    # Use posix threading to make use of multiprocessor machines
    # undef -> try if possible
    # 0 -> don't use
    # true -> force use

    WITH_POSIX_THREADS => undef,

    MALLOCDBG => undef,
    # {
    # include => '-I/home/csoelle/tmp',
    # libs => '-L/home/csoelle/tmp
    -lmymalloc',
    # define => << 'EOD',
    ##define malloc(n) dbgmalloc(n,__FILE__,__LINE__)
    ##define free(p) dbgfree(p)
    #EOD
    # include => '',
    # libs => '-lefence',
    # define => '',
    # },

    # Do we want routines to handle bad values?
    # saying no will make PDL a bit faster
    # true -> yes
    # false -> no, undef -> no
    #
    # WITH_BADVAL => 0,
    WITH_BADVAL => 1,

    # if WITH_BADVAL == 1, do we use NaN/Inf to represent badvalues
    # (not convinced setting this results in faster code)
    #
    # BADVAL_USENAN => 0,
    BADVAL_USENAN => 1,
    # BADVAL_USENAN => 1,

    # the original BADVAL implementation assigned bad-values on
    pdl-types,
    # not per pdl, setting the following to one will make it a
    pdl-variable
    # THIS IS AN EXPERIMENTAL FEATURE -- BEWARE...

    BADVAL_PER_PDL => 0,
    # BADVAL_PER_PDL => 1,

    # Try to build Graphics/TriD
    #
    # There are problems with the build on OS-X, so we turn it
    off by default
    # for such systems
    #
    WITH_3D => $^O eq "darwin" ? 0 : undef,

    #
    # For Mesa 3.2
    #
    # OPENGL_LIBS => '-L/usr/local/Mesa/lib -lGL -lGLU
    -lXext -lX11',
    # Mesa need -DGL_GLEXT_LEGACY for glPolygonOffsetEXT amoung
    others
    #
    # OPENGL_DEFINE => '-DGL_GLEXT_LEGACY',
    # OPENGL_INC => '-I/usr/include/GL/',
    #
    #
    # For SGI GL
    #
    # OPENGL_LIBS => '-lGL -lGLU -lXext -lX11',
    # OPENGL_INC => '',
    # OPENGL_DEFINE => '',
    #
    # Let perl try to figure it out
    # OPENGL_LIBS => undef,
    # OPENGL_INC => undef,
    # OPENGL_DEFINE => undef,

    # Whether or not to build the Karma interface module

    WITH_KARMA => undef, # Leave it up to PDL to decide
    WHERE_KARMA => undef, # you may explicitly specify
    directory location

    #
    ## Whether or not to build the PLplot interface module
    #

    # default settings do not build PLPLOT.
    WITH_PLPLOT => undef, # Leave it up to PDL
    to decide
    WHERE_PLPLOT_LIBS => undef, # let PDL search for
    plplot installation
    WHERE_PLPLOT_INCLUDE => undef, # let PDL search for
    plplot installation

    # example manual settings:
    # WITH_PLPLOT => 1, #
    Build PLPLOT interface
    # WHERE_PLPLOT_LIBS => '/usr/local/plplot/lib', #
    PLplot lib dir
    # WHERE_PLPLOT_INCLUDE => '/usr/local/plplot/include', #
    PLplot include dir

    # Whether or not to build the PDL::Slatec module
    # 0 -> don't use
    # true -> force use

    WITH_SLATEC => undef, # Leave it up to PDL to decide

    # Whether or not to build the PDL::GSL module
    # 0 -> don't use
    # true -> force use

    WITH_GSL => undef, # Leave it up to PDL to decide

    # link flags for the GSL libs, e.g. '-L/usr/local/lib -lgsl -lm'
    GSL_LIBS => undef, # use gsl-config
    # Location to find GSL includes:
    GSL_INC => undef, # use gsl-config

    # Whether or not to build the PDL::FFTW module
    # 0 -> don't use
    # true -> force use

    WITH_FFTW => undef, # Leave it up to PDL to decide

    # Location to search for the FFTW libs
    FFTW_LIBS => [ '/lib','/usr/lib','/usr/local/lib'],
    # Location to find FFTW includes:
    FFTW_INC => ['/usr/include/','/usr/local/include'],

    # FFTW Numeric Precision Type to link in: (double or single
    precision)
    FFTW_TYPE => 'double',

    # Whether or not to build the PDL::IO::HDF module
    # 0 -> don't use
    # true -> force use
    WITH_HDF => undef,
    HDF_LIBS => undef,
    HDF_INC => undef,

    # Whether or not to build the PDL::IO::GD module
    # 0 -> don't use
    # true -> force use
    WITH_GD => 0,
    GD_LIBS => undef,
    GD_INC => undef,

    # Whether or not to build the PDL::GIS::Proj module
    # 0 -> don't use
    # true -> force use
    WITH_PROJ => $^O eq "cygwin" ? 0 : undef,
    PROJ_LIBS => undef,
    PROJ_INC => undef,

    # do we build PDL::IO::Browser?
    # - on some systems (eg OS-X) it won't build and I don't
    know enough
    # about curses to fix it
    # - 1 builds, 0 or undef does not build
    #
    WITH_IO_BROWSER => 0,
    );

    1; # Return OK status on 'require'

     
  • Craig DeForest
    Craig DeForest
    2006-07-18

    Logged In: YES
    user_id=20200

    Hmmm... Bill, would you check your Changes file to see if
    you've got my note about the pdlcore.c.PL fix at the top?
    Your update came a scant 5 minutes after mine and I'm not
    sure whether our updates crossed "in the mail". I'd like to
    double-check that the fix works for you.

     
  • Bill Coffman
    Bill Coffman
    2006-07-19

    Logged In: YES
    user_id=1538801

    Zowie's latest change seems to fix all my examples, so closing.

     
  • Bill Coffman
    Bill Coffman
    2006-07-19

    • status: open --> closed