PyDSTool / Discussion / Help: Radau/Dopri error: gcc exit status 1

Dear all,
I have Radau and Dopri working fine on the examples delivered by PyDSTool (e.g. DAE_example.py, interp_dopri_test.py etc.) but when I try to run my own system I get the following compiler error:

error: Command "gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/home/maurizio/Dropbox/StabilityAnalysis_tmp -I/usr/local/pydstool/PyDSTool/integrator -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/include/python2.7 -c /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c -o /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.o -w -Wno-return-type -Wall -shared -D__RADAU__" failed with exit status 1

My system works fine with the VODE generator. The problem seems just related to the Radau and Dopri.
I tried hacking utils.py as pointed out in this thread on this forum but this won't solve the issue.

I tried getting some insight by GDB but unfortunately I can't access the stack. Besides this the debugger produces a series of weird errors as if all the right hand side of the system is not recognized (perhaps there is a missing link?):

/home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c: In function ‘vfieldfunc’:
/home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:122:1: error: ‘dmedt_val’ undeclared (first use in this function)
     dmedt_val=ds.me_dot_network(me,se2,mi,si2,ma,sa2,gs,
     ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:122:1: note: each undeclared identifier is reported only once for each function it appears in
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:122:11: error: ‘ds’ undeclared (first use in this function)
     dmedt_val=ds.me_dot_network(me,se2,mi,si2,ma,sa2,gs,
               ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:131:24: error: ‘dse2dt_val’ undeclared (first use in this function)
                         ); dse2dt_val=ds.se2_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:140:24: error: ‘dmidt_val’ undeclared (first use in this function)
                         ); dmidt_val=ds.mi_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:149:24: error: ‘dsi2dt_val’ undeclared (first use in this function)
                         ); dsi2dt_val=ds.si2_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:158:24: error: ‘dmadt_val’ undeclared (first use in this function)
                         ); dmadt_val=ds.ma_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:167:24: error: ‘dsa2dt_val’ undeclared (first use in this function)
                         ); dsa2dt_val=ds.sa2_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:176:24: error: ‘dgsdt_val’ undeclared (first use in this function)
                         ); dgsdt_val=ds.gs_dot_network(me,se2,mi,si2,ma,sa2,gs,
                            ^
    /home/maurizio/Dropbox/StabilityAnalysis_tmp/radau5_temp/ei_network_vf.c:189:1: error: expected ‘;’ before ‘f_’
     f_[0] = dgsdt_val;
     ^

Note that the _vf.* files seems to be produced in the radau_tmp or dop83_tmp directories, with the exception of the file _vf.o.

What could be the problem? Can I fix it?

Maurizio De Pitta' - 2016-09-20

Maybe I should add, that in the model, the RHS is passed as a Python function that contains some C-code compiled through WEAVE, and is passed to PyDSTool by the attributes

DSargs.vfcodeinsert_start = ... DSargs.ignorespecial = ...

as outlined in this thread.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-20

What I find weird is that the system runs if I choose the VODE generator... So it does not seem to be a problem of syntax or specification of the model...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Florian Aspart - 2016-09-20
  
  Hi Maurizio,
  
  I'm a little bit rusty with PyDSTool, but here is my guess.
  
  I think it is a problem of specification of the model. The VODE generator
  is a python only generator. Which means that the RHS as you provides it is
  simply executed.
  In case of the Radau and Dopri Generators, the model is first converted to
  C code and then compiled. You get the error during the compilation,
  probably because it cannot convert the Python function to C.
  
  If this is the case, you need to specify the C functions in a separate file
  and include them while building the library source:
  
  gen.makeLibSource(include=['headerContainingFunctionDef.h'] # Create the
  library with your own function def in C
  gen.compileLib() # Compile the generator
  
  This is also documented in this (quite old) post:
  https://sourceforge.net/p/pydstool/discussion/472291/thread/64e47eaf/
  
  As I said, I haven't been using PyDSTool for a while so my knowledge might
  be a little bit outdated, though.
  
  Best,
  Florian
  
  2016-09-20 2:48 GMT+02:00 Maurizio De Pitta' mauriziodepitta@users.sf.net:
  
  What I find weird is that the system runs if I choose the VODE
  generator... So it does not seem to be a problem of syntax or specification
  of the model...
  
  Radau/Dopri error: gcc exit status 1
  https://sourceforge.net/p/pydstool/discussion/472291/thread/5c791cbe/?limit=25#8c2f
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/pydstool/discussion/472291/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-20

Thanks Florian,
indeed I was figuring that out that with C integrators you need to bypass the Python interface.

Before I put hands on it however I have one more question. In my case the C-code function is in a CPP function which in turn requires loading of several C/C++ custom libraries. Can I pass the whole chain of H and CPP files including library include directories and links to gen.makeLibSource(...)?

I suppose so, but then I assume that there are different input arguments for gen.makeLibSource... what are they?

To be specific, currently my C routine as a standalone compiles by the following:

gcc -std=c++11 -g -v -O -I/usr/include/python2.7 -lpython2.7 -lm -lgsl -lgslcblas -I/usr/include/numpy -I/home/maurizio/Dropbox/Ongoing.Projects/pycustommodules -I/home/maurizio/Dropbox/Ongoing.Projects/c_libraries -I/home/maurizio/Dropbox/Ongoing.Projects/c_libraries/models /home/maurizio/Dropbox/Ongoing.Projects/pycustommodules/pycapi_utils.cpp /home/maurizio/Dropbox/Ongoing.Projects/c_libraries/dcomplex.cpp /home/maurizio/Dropbox/Ongoing.Projects/c_libraries/special_functions.cpp /home/maurizio/Dropbox/Ongoing.Projects/c_libraries/prob_dist.cpp /home/maurizio/Dropbox/Ongoing.Projects/c_libraries/models/freq_cv.cpp asn_meanfield.cpp netowrk_meanfield_mex.cpp -output rhs_network
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-20

So I have not attempted it yet. I am going to do it shortly. Maybe Rob or Florian can quickly confirm it.

I was looking at some mixins.py on Rob's github and if I got it correctly:

gen.makeLibSources(include=['header1.h','header2.h',...]) gen.compileLib(libsources=['-lm','-lgsl','-lgsclcblas','source1.cpp','source2.cpp',...],libdir=['dir1','dir2',...])

I mean, I should be able to link libraries as -lm -lgsl -lgsclcblas by the input argument libsources of the compileLib method. Am I right? Whereas libdir basically passes to the compiler the -I flag, i.e. the directories to include in the search path for header/source files.

How can I pass then the -std=c++11 flag argument? Do I need to manually edit mixins.py?

Last edit: Maurizio De Pitta' 2016-09-20
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-20

By editing manually, basically adding -std=c++11 to extra_compile_args in mixins.py ... and hoping that the integrator code will then be compatible with c++11...

Last edit: Maurizio De Pitta' 2016-09-20

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Rob Clewley - 2016-09-20
  
  Yes, that's the right idea. You can try changing std but I don't
  know what will happen. Generally, I haven't stress tested this stuff,
  so please let us know what you find. If dopri/radau don't compile with
  a different std, you might want to research elsewhere how to mix
  library sources like this for general-purpose C/C++ projects.
  StackOverflow probably has some insights.
  
  On Tue, Sep 20, 2016 at 11:56 AM, Maurizio De Pitta'
  mauriziodepitta@users.sf.net wrote:
  
  By editing manually, basically replacing -std=c99 by -std=c++11 at line 184
  of mixins.py ... and hoping that the integrator code will then be compatible
  with c++11...
  
  Radau/Dopri error: gcc exit status 1
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/pydstool/discussion/472291/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-21

Uhm,
so I have not succeeded yet. The _vf.c file looks fine now. But the compilation fails. In particular the compiled cannot recognize any of the C++ libraries, even if I add -std=c++11 to extra_compiler_args (line 180 of mixins.py). Like I get errors of the type:

/usr/include/blitz/blitz.h:45:18: fatal error: string: No such file or directory #include <string> ^

I suspect that the order I am passing the arguments to the compiler could be the reason, and for this I was trying to add -std=c++11 right after the specification of gcc. But I don't know how... it seems that the first string is fixed and built by a script build_ext which customizes the compiler automatically.

I am attaching the build.log file for convenience, hoping that you can help me figure out what I am missing...

Last edit: Maurizio De Pitta' 2016-09-21

build.log
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Rob Clewley - 2016-09-21
  
  If you capture the gcc command to console that build_ext
  (setuptools/distutils, in particular) creates, you can recreate your
  own gcc command based on that one and run it manually outside of
  python. I forget now whether there are still stdout redirects in
  build_ext, but you can easily comment those out for a minute to see
  the command output.
  
  On Wed, Sep 21, 2016 at 12:57 AM, Maurizio De Pitta'
  mauriziodepitta@users.sf.net wrote:
  
  Uhm,
  so I have not succeeded yet. The _vf.c file looks fine now. But the
  compilation fails. In particular the compiled cannot recognize any of the
  C++ libraries, even if I add -std=c++11 to extra_compiler_args (line 180 of
  mixins.py). Like I get errors of the type:
  
  /usr/include/blitz/blitz.h:45:18: fatal error: string: No such file or
  directory
  #include <string>
  ^
  
  I suspect that the order I am passing the arguments to the compiler could be
  the reason, and for this I was trying to add -std=c++11 right after the
  specification of gcc. But I don't know how... it seems that the first string
  is fixed and built by a script build_ext which customizes the compiler
  automatically.
  
  Radau/Dopri error: gcc exit status 1
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/pydstool/discussion/472291/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-21

Hi Rob,
I am afraid I am not sure how to "capture" the gcc command from build_ext. I tried to run the whole gcc command (as in build.log) from command line, but nothing happened. Because there is a wrapping by SWIG also that is involved in the code...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-21

Nope indeed,
I spent the past 3 hours to figure out how to hack build_ext but no success. The real issue is that it seems that the compiler does not recognize the standard C++ libraries/headers... but libstdc++ is installed and works. There is something in the pipeline of concatenated flags that somehow screws the successful compilaltion. So any further help/input when you get the chance would be appreciated.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Ok a slight progress here... I forgot of the fact that mixins.py is generating itself *_vf*.c files... So I replaced these files's extension by .cpp and now the generator step seem to go through till it stops and produce the following error:

Error in importing compiled vector field and integrator.
Did you compile the RHS C code?
Traceback (most recent call last):
  File "/home/maurizio/Dropbox/StabilityAnalysis_tmp/asn_meanfield.py", line 1791, in <module>
    ng_network_traj(model='ei-network',plot=True)
  File "/home/maurizio/Dropbox/StabilityAnalysis_tmp/asn_meanfield.py", line 1502, in ng_network_traj
    traj = DS.compute(traj_name)
  File "/usr/local/pydstool/PyDSTool/Generator/Dopri_ODEsystem.py", line 463, in compute
    extraSpace=self.algparams['extraspace'],
  File "/usr/local/pydstool/PyDSTool/Generator/Dopri_ODEsystem.py", line 45, in __init__
    modname, *imp.find_module(modname, ["dop853_temp"]))
  File "dop853_temp/dop853_ei_network_vf.py", line 28, in <module>
    _dop853_ei_network_vf = swig_import_helper()
  File "dop853_temp/dop853_ei_network_vf.py", line 24, in swig_import_helper
    _mod = imp.load_module('_dop853_ei_network_vf', fp, pathname, description)
ImportError: dop853_temp/_dop853_ei_network_vf.so: undefined symbol: massMatrix

Any suggestion?

Maurizio De Pitta' - 2016-09-21

After some further trial and error, I am afraid it is impossible.
The whole integrators are coded in native C and trying to extend it to C++ makes the problem intractable at least for my expertise.

The only way to make it work, is to code my equations only in C as briefly described in this post.

Last edit: Maurizio De Pitta' 2016-09-22

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Florian Aspart - 2016-09-22
  
  Maybe instead of changing the generated code from C to C++ you could just
  create a C wrapper for your RHS function which itself calls a C++ library.
  
  http://stackoverflow.com/questions/2744181/how-to-call-c-function-from-c#2744206
  
  In this case, your function would be in C and it would call the model
  implementation coded in C++.
  
  I'm not sure what kind of parameters you should give to PyDSTool for the
  compiler (my knowledge of PyDSTool is not big enough for that), but this is
  probably the way I would try to do it.
  
  2016-09-21 23:33 GMT+02:00 Maurizio De Pitta' mauriziodepitta@users.sf.net
  :
  
  After some further trial and error, I am afraid it is impossible.
  The whole integrators are coded in native C and trying to extend it to C++
  makes the problem intractable at least for my expertise.
  
  I tried to refactor my code as plain C but in my case, as I heavily use
  function overloading and classes, I am afraid that the task exceeds my
  current time requirements.
  
  Radau/Dopri error: gcc exit status 1
  https://sourceforge.net/p/pydstool/discussion/472291/thread/5c791cbe/?limit=25#35d9
  
  Sent from sourceforge.net because you indicated interest in
  https://sourceforge.net/p/pydstool/discussion/472291/
  
  To unsubscribe from further messages, please visit
  https://sourceforge.net/auth/subscriptions/
  
  alternate
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Maurizio De Pitta' - 2016-09-22

Thanks Florian.
I did not know about this possibility. I am going to test it shortly on the part of my C++ routines that use classes.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Radau/Dopri error: gcc exit status 1

Forums

Help

Radau/Dopri error: gcc exit status 1 document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Radau/Dopri error: gcc exit status 1