Thread: [pygccxml-development] wrapping pod struct with reference member variables
Brought to you by:
mbaas,
roman_yakovenko
From: Roman Y. <rom...@gm...> - 2009-01-20 05:59:44
|
On Tue, Jan 20, 2009 at 12:24 AM, <sit...@li...> wrote: > The attached message was received as a bounce, but either the bounce > format was not recognized, or no member addresses could be extracted > from it. This mailing list has been configured to send all > unrecognized bounce messages to the list administrator(s). Maik, for some reason your message was bounced. > ---------- Forwarded message ---------- > From: Maik Beckmann <bec...@go...> > To: pyg...@li... > Date: Mon, 19 Jan 2009 22:34:50 +0100 > Subject: wrapping pod struct with reference member variables > > Hi, > > I use structs with reference members for convenience naming in my code. For example > struct LookupTables{ > LookupTable& Fx,Fy,Fz; > }; > The ctor is omitted by intention to be able to use it like an POD > LookupTables lt = > { > forcesLookupTables[0], forcesLookupTables[1], forcesLookupTables[1] > }; Boost.Python don't like such structs - it cannot track the memory usage reliably. You also force your users to be aware of the memory management issue ( keep LookupTable alive, until LookupTables class instance is alive ) > = Hand written bpl code = > Wrapping this code with boost python is possible. > Consider this code > struct Refee { }; > struct Refer { Refee& refee; }; > The hand written written wrapper code looks like this > {{{ > namespace boost { namespace python { > struct no_default_ctor_policy : default_call_policies > { > static bool precall(PyObject*) > { > PyErr_SetString(PyExc_StandardError, "no default ctor"); > throw_error_already_set(); > } > }; > }} // ns boost::python > namespace { > void set_refee(Refer& refer, Refee& refee) > { refer.refee = refee; } > Refee get_refee(Refer& refer) > { return refer.refee; } > std::auto_ptr<Refer> make_refer(Refee* refee) > { > Refer tmp = { *refee }; > return std::auto_ptr<Refer>(new Refer(tmp)); > } > } //anonymous ns > BOOST_PYTHON_MODULE(pyExample) > { > bp::class_<Refee>("Refee"); > bp::class_<Refer>("Refer", bp::init<>()[bp::no_default_ctor_policy()]) > .def("__init__", bp::make_constructor(make_refer)) > .add_property("refee", get_refee, set_refee); > } > }}} > There are two tricks. > 1. Calling the default __init__(self) would cause an dangling reference. no_default_ctor_policy raises an error if someone call Refer() > 2. getter and setter are free functions, nothing special > This code > {{{ > from pyExample import Refer, Refee > refee = Refee() > #refer = Refer(r)# raises error > refer = Refer(refee) > }}} > works. > = pyplusplus = > py++ however, generates this (reformatted to make it a little shorter) > {{{ > struct Refer_wrapper : Refer, bp::wrapper< Refer > { > Refer_wrapper(Refer const & arg ): Refer( arg ) , bp::wrapper< Refer >() > { } > Refer_wrapper() : Refer(), bp::wrapper< Refer >() > { } > static ::Refee & get_refee( Refer& inst ) { > return inst.refee; > } > static void set_refee( Refer& inst, ::Refee & new_value ){ > inst.refee = new_value; > } > }; > BOOST_PYTHON_MODULE(pyExample){ > bp::class_< Refee >( "Refee" ); > { //::Refer > typedef bp::class_< Refer_wrapper > Refer_exposer_t; > Refer_exposer_t Refer_exposer = Refer_exposer_t( "Refer" ); > bp::scope Refer_scope( Refer_exposer ); > Refer_exposer.def( "get_refee" > , (::Refee & (*)( ::Refer & ))(&Refer_wrapper::get_refee) > , bp::return_value_policy< bp::copy_non_const_reference >() ); > Refer_exposer.def( "set_refee" > , (void (*)( ::Refer &,::Refee & ))(&Refer_wrapper::set_refee) ); > } > } > }}} > == Problems == > There are three problems: > 1. > The wrapper ctors are trouble. It's default construct breaks compilation, because there is no way it can proper initialize the members of the Refer. Refer lacks the ctor to do this (by intention, see above). > Wrapping isn't an option AFAICT. Obviously this is a bug. I will check it against current SVN ( both gccxml and Py++ ). > 2. The call policies for the __init__(self) are not altered You have a bug in the relevant code: > defctor = foo.operator(arg_types=[None]) defctor = foo.find_trivial_constructor() if defctor: .... > 3. A fake constructor (via bp::make_constructor) is needed > === regarding 1. === > Excluding all reference members stops wrapping. But then I have to write the getter/setter functions myself. > === regarding 2. === > I played around and it seems setting custom call policies > like > {{{ > Refer = mb.class_('Refer') > Refer.variables().exclude() > defctor = foo.operator(arg_types=[None]) > defctor.call_policies = call_policies.custom_call_policies("no_default_ctor_policy", "call_policies.hpp") > }}} > doesn't have an effect!? See my previous comment > === regarding 3. === > So far I didn't get how class_t.add_fake_constructors works. This function takes a calldef_t as argument. If I inject code via mb.add_declaration_code mb.free_function('make_refer') doesn't work because gccxml newer saw the code. Any hints how to construct a calldef_t after gccxml ran? You don't. Tweaking declarations tree is not a good idea. > As a workaround I added the code to the parsed header file and did > mb.class_('Refer').add_fake_constructors(mb.free_function('make_refer')) > which results in > {{{ > File "/usr/lib/python2.6/site-packages/pyplusplus/decl_wrappers/class_wrapper.py", line 210, in add_fake_constructors > self._fake_constructors.add( f ) > AttributeError: 'list' object has no attribute 'add > }}} This is a bug, which I fixed right now. Thanks for reporting. It seems that I always used\tested the other path :-). > I know doing mb.class_('Refer').exclude() and creating the wrapper code from the information in mb.class_('Refer') works (I did this for template class template member member functions), but I wonder what are the other means to teach py++ what I want. I am going to play with your example, stay turned :-) P.S. Please format a little the code, so it could be easier to read. Thanks -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
From: Maik B. <bec...@go...> - 2009-01-20 06:43:51
|
Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 06:59: > Maik, for some reason your message was bounced. Yikes! I don't use SF mailing lists a lot, but I guess it was to long. > Boost.Python don't like such structs - it cannot track the memory > usage reliably. > You also force your users to be aware of the memory management issue ( > keep LookupTable alive, until LookupTables class instance is alive ) The generated bindings are for testing purposes only. Having python gives me numpy, matplotlib and vtk/tvtk. The data I'm passing around reaches the limits of 32Bit memory machines, so I decided to sacrifice the ease use to some extend. > > There are three problems: > > 1. > > The wrapper ctors are trouble. It's default construct breaks compilation, > > because there is no way it can proper initialize the members of the > > Refer. Refer lacks the ctor to do this (by intention, see above). > > Wrapping isn't an option AFAICT. > > Obviously this is a bug. I will check it against current SVN ( both > gccxml and Py++ ). Thanks for taking care > > 2. The call policies for the __init__(self) are not altered > > You have a bug in the relevant code: > > defctor = foo.operator(arg_types=[None]) > > defctor = foo.find_trivial_constructor() > if defctor: > .... I'll check it > > === regarding 3. === > > So far I didn't get how class_t.add_fake_constructors works. This > > function takes a calldef_t as argument. If I inject code via > > mb.add_declaration_code mb.free_function('make_refer') doesn't work > > because gccxml newer saw the code. Any hints how to construct a calldef_t > > after gccxml ran? > > You don't. Tweaking declarations tree is not a good idea. Roger > > As a workaround I added the code to the parsed header file and did > > mb.class_('Refer').add_fake_constructors(mb.free_function('make_refer')) > > which results in > > {{{ > > File > > "/usr/lib/python2.6/site-packages/pyplusplus/decl_wrappers/class_wrapper. > >py", line 210, in add_fake_constructors self._fake_constructors.add( f ) > > AttributeError: 'list' object has no attribute 'add > > }}} > > This is a bug, which I fixed right now. Thanks for reporting. > It seems that I always used\tested the other path :-). Well, there are downsides of dynamic languages as well ;) I don't wanna miss the static nature of C++, if its appropriate. > > I know doing mb.class_('Refer').exclude() and creating the wrapper code > > from the information in mb.class_('Refer') works (I did this for template > > class template member member functions), but I wonder what are the other > > means to teach py++ what I want. > > I am going to play with your example, stay turned :-) Thank you Roman! -- Maik > P.S. Please format a little the code, so it could be easier to read. Thanks The Mail in my output folder is all formatted. The bouncer must have removed the leading whitespace. I can send you the mail again off list, if you like to. Next time this happens, please refuse to read the mail and ask for the proper version(i.e. off-list). This is what I would have done. You are awesome :) |
From: Maik B. <bec...@go...> - 2009-01-20 09:31:50
|
Maik Beckmann schrieb am Dienstag 20 Januar 2009 um 07:44: > Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 06:59: > > Maik, for some reason your message was bounced. > > Yikes! I don't use SF mailing lists a lot, but I guess it was to long. > > > P.S. Please format a little the code, so it could be easier to read. > > Thanks > Ok, I think both happened because I didn't noticed kmail switched html- formatting on. Sorry! |
From: Roman Y. <rom...@gm...> - 2009-01-20 07:44:28
Attachments:
refee_refer.cpp
|
On Tue, Jan 20, 2009 at 8:44 AM, Maik Beckmann <bec...@go...> wrote: >> Obviously this is a bug. I will check it against current SVN ( both > >> gccxml and Py++ ). > > Thanks for taking care Done. It seems like you are using buggy GCCXML version or wrong Py++. I committed new test case for Py++ and everything works just fine ( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml?view=rev ) I also attached the generated code. I use current GCCXML cvs and current Py++ & pygccxml SVN. Happy upgrading :-) -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
From: Maik B. <bec...@go...> - 2009-01-20 09:22:31
|
Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 08:44: > Done. It seems like you are using buggy GCCXML version or wrong Py++. > I committed new test case for Py++ and everything works just fine ( > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml?view=rev ) > I also attached the generated code. > I use current GCCXML cvs and current Py++ & pygccxml SVN. > > Happy upgrading :-) gccxml-cvs is just updated, rebuild and installed. pygccxml is on rev 1590 pyplusplus is on revision 1602 it fails with {{{ File "/usr/lib/python2.6/site- packages/pyplusplus/code_repository/__init__.py", line 23, in <module> import indexing_suite ImportError: No module named indexing_suite }}} -- Maik |
From: Roman Y. <rom...@gm...> - 2009-01-20 09:33:44
|
On Tue, Jan 20, 2009 at 11:23 AM, Maik Beckmann <bec...@go...> wrote: > Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 08:44: > >> Done. It seems like you are using buggy GCCXML version or wrong Py++. > >> I committed new test case for Py++ and everything works just fine ( > >> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml?view=rev ) > >> I also attached the generated code. > >> I use current GCCXML cvs and current Py++ & pygccxml SVN. > >> > >> Happy upgrading :-) > > gccxml-cvs is just updated, rebuild and installed. > > pygccxml is on rev 1590 > > pyplusplus is on revision 1602 > > it fails with > > {{{ > > File > "/usr/lib/python2.6/site-packages/pyplusplus/code_repository/__init__.py", > line 23, in <module> > > import indexing_suite > > ImportError: No module named indexing_suite > > }}} Strange, the "indexing_suite" package exists in repository: http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/code_repository/ -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
From: Maik B. <bec...@go...> - 2009-01-20 09:56:31
|
Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 10:33: > Strange, the "indexing_suite" package exists in repository: > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pypluspl >us/code_repository/ Yes, its in my working copy too, but not installed by $ python setup.py install --root=$startdir/pkg = working copy = {{{ [maik@horst pyplusplus-svn]$ ls -1 src/pyplusplus/pyplusplus/code_repository/ __init__.py array_1.py call_policies.py convenience.py ctypes_integration.py ctypes_utils.py gil_guard.py indexing_suite named_tuple.py return_pointee_value.hpp return_range.py }}} = installed = {{{ [maik@horst pyplusplus-svn]$ ls -1 pkg/usr/lib/python2.6/site- packages/pyplusplus/code_repository/|grep -v pyc$ __init__.py array_1.py call_policies.py convenience.py ctypes_integration.py ctypes_utils.py gil_guard.py named_tuple.py return_range.py }}} |
From: Roman Y. <rom...@gm...> - 2009-01-20 10:04:33
|
On Tue, Jan 20, 2009 at 11:57 AM, Maik Beckmann <bec...@go...> wrote: > Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 10:33: > >> Strange, the "indexing_suite" package exists in repository: >> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pypluspl >>us/code_repository/ > > Yes, its in my working copy too, but not installed by > $ python setup.py install --root=$startdir/pkg Sorry, I always forget to add new packages to the setup.py script :-( The fix in SVN -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
From: Maik B. <bec...@go...> - 2009-01-20 10:34:58
Attachments:
pod.tar.gz
|
Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 11:04: > > Yes, its in my working copy too, but not installed by > > $ python setup.py install --root=$startdir/pkg > > Sorry, I always forget to add new packages to the setup.py script :-( > > The fix in SVN Ok. These are are versions I have installed: {{{ [maik@horst pod]$ pacman -Qs gccxml local/gccxml-cvs 20090120-1 GCC-XML generates an XML description of a C++ program from GCC's internal representation local/pygccxml-svn 1590-1 Python wrapper of gccxml [maik@horst pod]$ pacman -Qs pyplusplus-svn local/pyplusplus-svn 1603-1 Binding tool for boost.python }}} where the xxx in xxx-1 means the revision number for svn packages while cvs packages tagged with the build date. It doesn't work for me. The hole project dir is attached as pod.tar.gz including the build log pod/log The generated file is pod/build/pyMaik.cpp The python code for py++ is in pod/build/generate_code.py which is in turn generated by cmake from pod/generate_code.py.in. I guess I missed some adjustments for generate_code.py.in, sorry. If you want to build it yourself, make sure gccxml and cmake-2-6-x are on the path, pygccxml and py++ are installed for the python executable on the path and run at pod/ rm build -r mkdir build cd build #export BOOST_ROOT=/path/to/boost #optional, if its not installed cmake .. make On Windows, it works the same with cygwin or msys. For VisualStudio the cmake-gui.exe shipped with cmake2.6.x is more convenient. Thanks, -- Maik |
From: Roman Y. <rom...@gm...> - 2009-01-20 12:20:14
|
On Tue, Jan 20, 2009 at 12:35 PM, Maik Beckmann <bec...@go...> wrote: > Roman Yakovenko schrieb am Dienstag 20 Januar 2009 um 11:04: > >> > Yes, its in my working copy too, but not installed by >> > $ python setup.py install --root=$startdir/pkg >> >> Sorry, I always forget to add new packages to the setup.py script :-( >> >> The fix in SVN > > Ok. These are are versions I have installed: > {{{ > [maik@horst pod]$ pacman -Qs gccxml > local/gccxml-cvs 20090120-1 > GCC-XML generates an XML description of a C++ program from GCC's internal > representation > local/pygccxml-svn 1590-1 > Python wrapper of gccxml > [maik@horst pod]$ pacman -Qs pyplusplus-svn > local/pyplusplus-svn 1603-1 > Binding tool for boost.python > }}} > where the xxx in xxx-1 means the revision number for svn packages while cvs > packages tagged with the build date. > > > It doesn't work for me. The hole project dir is attached as pod.tar.gz Mike, all I need to see your problem is xml file generated by gccxml. During Py++ execution it prints command line used to generate xml file. Can you generate it and send to me? -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
From: Maik B. <bec...@go...> - 2009-01-20 13:09:33
|
2009/1/20 Roman Yakovenko <rom...@gm...>: > You don't use CVS gccxml version. Your gccxml version is 1.123, my is 1.127. > This information is written in the generated xml file. Oh Boy... To build and install gccxml from cvs I downloaded a build script. This checks out nicely, but doesn't do cvs update proper, it just sticked to the initial date That once had been checked out. I fixed it and will report the bug. I didn't got it earlier, because I never am not a frequent CVS user. I'm so sorry that I've wasted your time. Now it works just nicely! > If you use Windows you can download the binaries from the pygccxml > project on sourceforge. If Linux, than SVN tree contains executable > built on Ubuntu 8.10. While I prefer to build on linux myself, I'm very happy to take the windows Installer you've uploaded. Thank you Roman, -- Maik |