PyDST Radau on 64-bit Ubuntu

Help
2011-06-25
2013-11-04
  • I'm running Ubuntu 10.10 on a 64-bit machine. I'm trying out PyDSTool DAE_example.py and get the following output:

    gustavo@ubuntu:~/pydstool/PyDSTool/tests$ python DAE_example.py
    In file included from /usr/include/features.h:387,
                     from /usr/include/math.h:28,
                     from /home/gustavo/pydstool/PyDSTool/integrator/interface.h:4,
                     from /home/gustavo/pydstool/PyDSTool/integrator/interface.c:1:
    /usr/include/gnu/stubs.h:7: fatal error: gnu/stubs-32.h: No such file or directory
    compilation terminated.
    Error occurred in generating Radau system...
    <type 'exceptions.SystemExit'> error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/local/lib/python2.6/dist-packages/numpy/numarray/include -I/home/gustavo/pydstool/PyDSTool/tests -I/home/gustavo/pydstool/PyDSTool/tests/radau5_temp -I/home/gustavo/pydstool/PyDSTool/integrator -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/include/python2.6 -c /home/gustavo/pydstool/PyDSTool/integrator/interface.c -o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/interface.o -w -D__RADAU__ -m32" failed with exit status 1
    Traceback (most recent call last):
      File "DAE_example.py", line 36, in <module>
        dae = Generator.Radau_ODEsystem(DSargs)
      File "/home/gustavo/pydstool/PyDSTool/Generator/Radau_ODEsystem.py", line 547, in __init__
        self.compileLib()
      File "/home/gustavo/pydstool/PyDSTool/Generator/Radau_ODEsystem.py", line 943, in compileLib
        raise RuntimeError
    RuntimeError
    

    It's trying to load gnu/stubs-32.h, which apparently Ubuntu 64-bit does not ship with. So I installed the package "libc6-dev-i386"

    Running again gives the following error:

    gustavo@ubuntu:~/pydstool/PyDSTool/tests$ python DAE_example.py 
    In file included from /usr/include/python2.6/Python.h:58,
                     from radau5_temp/src.linux-x86_64-2.6/radau5_temp/radau5_DAE_test_vf_wrap.c:125:
    /usr/include/python2.6/pyport.h:694: error: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."
    Error occurred in generating Radau system...
    <type 'exceptions.SystemExit'> error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/local/lib/python2.6/dist-packages/numpy/numarray/include -I/home/gustavo/pydstool/PyDSTool/tests -I/home/gustavo/pydstool/PyDSTool/tests/radau5_temp -I/home/gustavo/pydstool/PyDSTool/integrator -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/include/python2.6 -c radau5_temp/src.linux-x86_64-2.6/radau5_temp/radau5_DAE_test_vf_wrap.c -o radau5_temp/radau5_temp/src.linux-x86_64-2.6/radau5_temp/radau5_DAE_test_vf_wrap.o -w -D__RADAU__ -m32" failed with exit status 1
    Traceback (most recent call last):
      File "DAE_example.py", line 36, in <module>
        dae = Generator.Radau_ODEsystem(DSargs)
      File "/home/gustavo/pydstool/PyDSTool/Generator/Radau_ODEsystem.py", line 547, in __init__
        self.compileLib()
      File "/home/gustavo/pydstool/PyDSTool/Generator/Radau_ODEsystem.py", line 943, in compileLib
        raise RuntimeError
    RuntimeError
    

    I haven't figured out how to fix it yet.

    Thanks,
    Gustavo

     
  • Rob Clewley
    Rob Clewley
    2011-06-25

    Please let me know if you tried all the suggestions in Section "1.1.3.1. 64-bit machines" of the GettingStarted doc page:

    http://www2.gsu.edu/~matrhc/GettingStarted.html

    including those on the link to details for AMD machines if that's what you have. The first thing to try that you may not have done is to remove all the "-m32" flags in the Dopri and Radau Generator .py files, as listed in the FAQ in the PyDSTool directory.

     
  • I performed the changes outlined the FAQ (removing the -m32 flags) and DAE_example.py now gives a Segmentation Fault with no other output. The same with Dopri_backwards_test.py and dopri_event_test.py

    I'm running on an Intel Core 2 Duo, so I figure I don't have to apply the changes relevant to AMD.

     
  • Rob Clewley
    Rob Clewley
    2011-06-25

    Make sure to delete all the temporary files before trying without the -m32 flags - the whole radau_temp directory, basically. Otherwise they won't be refreshed and the linker will get upset. Let me know if the segfault is still a problem. Did you install 64 bit versions of python, scipy, etc.? Because that's sometimes an issue too.

     
  • I deleted radau5_temp and dopri853_temp but still get a Segmentation Fault. My Python version is 64-bit (as indicated by the platform.architecture() ). How can I check the NumPy/SciPy bit versions? If I recall, I installed NumPy from source, so it should be 64-bit.

     
  • Actually, SciPy must be 64-bit if I'm using it in 64-bit Python, I think.

     
  • Rob Clewley
    Rob Clewley
    2011-06-25

    I'm not sure what to suggest. Some have had similar problems before, but I also know several who have got it working on 64 bit machines. Are you sure you got *all* the occurrences of -m32 (one for the linker and one for the compiler in each of the .py files) because I know that's been one source of segfaults in the past. My only other suggestion for now would be to re-install python, numpy and scipy in a 32 bit mode and put back the -m32 flags. There's some kind of incompatibility between the host and target architectures. Could you post the stdout compilation output again so that I can see the exact gcc command, please?

     
  • I pulled the latest NumPy version  ('2.0.0.dev-79a1db5') from git and compiled. I've put the compile output here: http://codepad.org/VVJrR7fq

    I think I must have broken something by installing the latest version of NumPy:

    gustavo@ubuntu:~/pydstool/PyDSTool/tests$ python DAE_example.py 
    Traceback (most recent call last):
      File "DAE_example.py", line 11, in <module>
        from PyDSTool import *
      File "/home/gustavo/pydstool/PyDSTool/__init__.py", line 93, in <module>
        import Generator as GenModule
      File "/home/gustavo/pydstool/PyDSTool/Generator/__init__.py", line 8, in <module>
        from Vode_ODEsystem import *
      File "/home/gustavo/pydstool/PyDSTool/Generator/Vode_ODEsystem.py", line 14, in <module>
        from numpy import Inf, NaN, isfinite, sometrue, alltrue, sign, all, any, \
    ImportError: cannot import name unique1d
    
     
  • numpy.unique1d was removed from NumPy in favor of numpy.unique. (I was previously running NumPy 1.6 - must've still existed then). I changed the calls in Vode_ODEsystem.py

    Now getting:

    gustavo@ubuntu:~/pydstool/PyDSTool/tests$ python DAE_example.py 
    /usr/local/lib/python2.6/dist-packages/scipy/io/matlab/mio4.py:15: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
      from mio_utils import squeeze_element, chars_to_strings
    /usr/local/lib/python2.6/dist-packages/scipy/io/matlab/mio5.py:96: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility
      from mio5_utils import VarReader5
    Segmentation fault
    

    I'm going to compile the latest revision of SciPy too, too then I guess.

     
  • I'm now running SciPy '0.10.0.dev7180' , and getting a segfault again with no other messages upon running DAE_example.py

     
  • Rob Clewley
    Rob Clewley
    2011-06-25

    Make sure to delete the .so DLL that's been created as well as the temp directories, and then please post the full output of PyDSTool's attempt at compiling a new DLL before it runs the DAE script contents and segfaults. I.e., the same kind of info about gcc calls that were in your very first post.

     
  • Sorry, I hadn't realized there was a .so file too.

    Here's the complete output:

    gustavo@ubuntu:~/pydstool/PyDSTool/tests$ python DAE_example.py 
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f: In function radcor:
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f:960: warning: ‘dynold’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f:1020: warning: erracc may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f:1020: warning: ‘hacc’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f:1066: warning: hopt may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/radau5.f:964: warning: ‘thqold’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/lapackc.f: In function ztbsv:
    /home/gustavo/pydstool/PyDSTool/integrator/lapackc.f:2248: warning: ‘kx’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:6.60:
         &            M1,M2,NM1,FAC1,E1,LDE1,IP1,IER,IJOB,CALHES,IPHES)     
                                                                1
    Warning: Unused dummy argument 'calhes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:6.66:
         &            M1,M2,NM1,FAC1,E1,LDE1,IP1,IER,IJOB,CALHES,IPHES)     
                                                                      1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:200.49:
         &            M1,M2,NM1,ALPHN,BETAN,E2R,DUMMY,LDE1,IP2,IER,IJOB)    
                                                     1
    Warning: Unused dummy argument 'dummy' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:395.54:
         &          M1,M2,NM1,FAC1,E1,LDE1,Z1,F1,IP1,IPHES,IER,IJOB)        
                                                          1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:576.47:
         &          M1,M2,NM1,ALPHN,BETAN,E2R,DUMMY,LDE1,Z2,Z3,             
                                                   1
    Warning: Unused dummy argument 'dummy' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:577.36:
         &          F2,F3,CONT,IP2,IPHES,IER,IJOB)                          
                                        1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:884.55:
         &          M1,M2,NM1,FAC1,ALPHN,BETAN,E1,E2R,DUMMY,LDE1,Z1,Z2,Z3,  
                                                           1
    Warning: Unused dummy argument 'dummy' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:885.43:
         &          F1,F2,F3,CONT,IP1,IP2,IPHES,IER,IJOB)                   
                                               1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:1236.53:
         &          E1,LDE1,Z1,Z2,Z3,CONT,F1,F2,IP1,IPHES,SCAL,ERR,         
                                                         1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:1518.44:
         &          E1,LDE1,ZZ,CONT,FF,IP1,IPHES,SCAL,ERR,                  
                                                1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:2048.53:
          SUBROUTINE SLVSEU(N,FJAC,LDJAC,MLJAC,MUJAC,FMAS,LDMAS,MLMAS,MUMAS,
                                                         1
    Warning: Unused dummy argument 'fmas' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:2049.45:
         &          M1,M2,NM1,FAC1,E,LDE,IP,IPHES,DEL,IJOB)                 
                                                 1
    Warning: Unused dummy argument 'iphes' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:2048.65:
          SUBROUTINE SLVSEU(N,FJAC,LDJAC,MLJAC,MUJAC,FMAS,LDMAS,MLMAS,MUMAS,
                                                                     1
    Warning: Unused dummy argument 'mlmas' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:2048.71:
          SUBROUTINE SLVSEU(N,FJAC,LDJAC,MLJAC,MUJAC,FMAS,LDMAS,MLMAS,MUMAS,
                                                                           1
    Warning: Unused dummy argument 'mumas' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f: In function estrav:
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:1551: warning: ‘mm’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f: In function estrad:
    /home/gustavo/pydstool/PyDSTool/integrator/dc_lapack.f:1264: warning: ‘mm’ may be used uninitialized in this function
    /home/gustavo/pydstool/PyDSTool/integrator/lapack.f:796.69:
          INTEGER          FUNCTION ILAENV( ISPEC, NAME, OPTS, N1, N2, N3,  
                                                                         1
    Warning: Unused dummy argument 'n3' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/lapack.f:796.57:
          INTEGER          FUNCTION ILAENV( ISPEC, NAME, OPTS, N1, N2, N3,  
                                                             1
    Warning: Unused dummy argument 'opts' at (1)
    /home/gustavo/pydstool/PyDSTool/integrator/lapack.f: In function dtbsv:
    /home/gustavo/pydstool/PyDSTool/integrator/lapack.f:3406: warning: ‘kx’ may be used uninitialized in this function
    Segmentation fault
    
     
  • Rob Clewley
    Rob Clewley
    2011-06-25

    There's no output from distutils showing the "gcc -pthread -fno-strict-aliasing …" etc. call that it is making. But I just remembered that the stream is deliberately blocked around line 924 of Radau_ODEsystem.py to reduce verbosity (sorry!). So, if you comment out that line you could just post the new part that shows up which is not included in the rest of the output you already posted! Hopefully that will show the gcc call. You could do the same for a Dopri example too (first commenting out the equivalent line in Dopri_ODEsystem.py), as occasionally Dopri will work but not Radau.

     
  • I removed the line

    rout.start()
    

    from both Radau_ODEsystem.py and Dopri_ODEsystem.py. I even deleted all the .pyc files just in case. I also deleted *.so and the temp directories too.

    But even so, I do not see any output from distutils as you mentioned. The only new output is at the beginning

    Could not locate executable g77
    Could not locate executable f77
    Could not locate executable ifort
    Could not locate executable ifc
    Could not locate executable lf95
    Could not locate executable pgf90
    Could not locate executable pgf77
    Could not locate executable f90
    Found executable /usr/bin/f95
    Could not locate executable fort
    Could not locate executable efort
    Could not locate executable efc
    Found executable /usr/bin/gfortran
    

    the Dopri examples give no output at all that looks like anything is being compiled.

    Is stdout being redirected by anything else?

    Thanks for the help so far!

     
  • Rob Clewley
    Rob Clewley
    2011-06-26

    Right. Alas, the only way to see the gcc call when it *doesn't* fail at compilation time is to make distutils fail to finish with an ugly hack :) If you first add an extra meaningless entry to the distutils call's compile arguments, then run the script (remembering to clear out the .so and temp dirs) then you'll get an error that shows you the gcc call. Then, clean out again, and put the meaningless entry to the linker arguments instead, and repeat.

        extra_compile_args=,
        extra_link_args=)]

    These are around line 794 of Dopri and 935 of Radau. Once you have the full gcc compile and link commands, you can see if there are any mismatched libraries or options that would cause some 32 bit elements to be linked with 64 bit elements, as I'm guessing that's what happened. You can then tweak the setup by copy-pasting those gcc commands directly to the linux prompt (or putting them in a shell script) and editing it. But I have limited experience playing with that stuff, so there's no guarantee that I can fix it, sorry!

     
  • Alright! In Radau…

    BOGUS_ENTRY in extra_compile_args:

    gcc: BOGUS_ENTRY: No such file or directory
    gcc: BOGUS_ENTRY: No such file or directory
    Error occurred in generating Radau system...
    <type 'exceptions.SystemExit'> error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/local/lib/python2.6/dist-packages/numpy/numarray/include -I/home/gustavo/pydstool/PyDSTool/tests -I/home/gustavo/pydstool/PyDSTool/tests/radau5_temp -I/home/gustavo/pydstool/PyDSTool/integrator -I/usr/local/lib/python2.6/dist-packages/numpy/core/include -I/usr/include/python2.6 -c /home/gustavo/pydstool/PyDSTool/integrator/interface.c -o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/interface.o -w -D__RADAU__ BOGUS_ENTRY" failed with exit status 1
    

    BOGUS_ENTRY in extra_link_args

    Error occurred in generating Radau system...
    <type 'exceptions.SystemExit'> error: Command "/usr/bin/gfortran -Wall -Wall -shared radau5_temp/radau5_temp/src.linux-x86_64-2.6/radau5_temp/radau5_DAE_test_vf_wrap.o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/radau5mod.o radau5_temp/home/gustavo/pydstool/PyDSTool/tests/radau5_temp/DAE_test_vf.o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/integration.o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/eventFinding.o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/interface.o radau5_temp/home/gustavo/pydstool/PyDSTool/integrator/memory.o -L/home/gustavo/pydstool/PyDSTool/integrator -L/home/gustavo/pydstool/PyDSTool/tests/radau5_temp -L/home/gustavo/pydstool/PyDSTool/tests -Lradau5_temp -lradau5 -lgfortran -o ./_radau5_DAE_test_vf.so -w BOGUS_ENTRY" failed with exit status 1
    

    Let me know if it screams anything out to you.

     
  • Rob Clewley
    Rob Clewley
    2011-06-26

    No, but you could try adding -m64 where there were -m32 options to the compiler and linker. Also, can you make sure that the libraries linked to in  /usr/include/python2.6,  /usr/local/lib/python2.6/dist-packages/numpy/core/include  and  /usr/local/lib/python2.6/dist-packages/numpy/numarray/include  are the 64 bit versions? If you look in that directory structure, is there a separate version for 64 bits? E.g. is there a /usr/local/lib64/ directory?

     
  • andram
    andram
    2012-07-24

    Just to ask if this has been resolved yet? I also see probably the same problem on AMD64 with the current bzr checkout. It looks as follows for me:
    >python2 dopri_event_test.py
    Computing trajectory:
    traj = testODE.compute('traj')
    Segmentation fault

    I suspect something in the distutils.setup does not work out. As the problem appears with dop853 and radau, it appears that it is not only a fortran issue.
    The above command generates the file _dop853_event_test_vf.so but it is not transparent how it is generated. Can I test the .so file independently somehow?

     
  • Rob Clewley
    Rob Clewley
    2012-07-25

    Sorry, no, I do not have any further information about this. I have not rigorously tested a fully 64 bit version and I don't know why it works for some people and not others.