pygccxml-development Mailing List for C++ Python language bindings (Page 29)
Brought to you by:
mbaas,
roman_yakovenko
Menu
▾
▴
pygccxml-development
You can subscribe to this list here.
| 2006 |
Jan
|
Feb
(6) |
Mar
(160) |
Apr
(96) |
May
(152) |
Jun
(72) |
Jul
(99) |
Aug
(189) |
Sep
(161) |
Oct
(110) |
Nov
(9) |
Dec
(3) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2007 |
Jan
(13) |
Feb
(48) |
Mar
(35) |
Apr
(7) |
May
(37) |
Jun
(8) |
Jul
(15) |
Aug
(8) |
Sep
(2) |
Oct
(1) |
Nov
(2) |
Dec
(38) |
| 2008 |
Jan
(11) |
Feb
(29) |
Mar
(17) |
Apr
(3) |
May
|
Jun
(64) |
Jul
(49) |
Aug
(51) |
Sep
(18) |
Oct
(22) |
Nov
(9) |
Dec
(9) |
| 2009 |
Jan
(28) |
Feb
(15) |
Mar
(2) |
Apr
(11) |
May
(6) |
Jun
(2) |
Jul
(3) |
Aug
(34) |
Sep
(5) |
Oct
(7) |
Nov
(13) |
Dec
(14) |
| 2010 |
Jan
(39) |
Feb
(3) |
Mar
(3) |
Apr
(14) |
May
(11) |
Jun
(8) |
Jul
(9) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2011 |
Jan
|
Feb
|
Mar
(7) |
Apr
|
May
|
Jun
(3) |
Jul
(3) |
Aug
(3) |
Sep
|
Oct
|
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
(2) |
| 2016 |
Jan
(1) |
Feb
(1) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
(1) |
Aug
(1) |
Sep
|
Oct
|
Nov
(1) |
Dec
|
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
(1) |
| 2020 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
(1) |
| 2021 |
Jan
(1) |
Feb
(1) |
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(1) |
Nov
|
Dec
|
|
From: Roman Y. <rom...@gm...> - 2007-02-19 05:42:00
|
On 2/19/07, Gustavo Carneiro <gjc...@gm...> wrote:
> Even with the name mangling problem, something was not right:
>
> gjc@nazgul:python$ python test-schedule.py
> Traceback (most recent call last):
> File "test-schedule.py ", line 11, in <module>
> Simulator.Schedule_fbf096d706be19b392c1e3fbcd3bf544(t1,
> ev)
> Boost.Python.ArgumentError: Python argument types in
>
> Simulator.Schedule_fbf096d706be19b392c1e3fbcd3bf544(Time,
> MyEvent)
> did not match C++ signature:
> Schedule_fbf096d706be19b392c1e3fbcd3bf544(ns3::Time
> time, std::auto_ptr<ns3::EventImpl> event)
>
> Then I saw something I missed in the unit test, and added this code:
>
> EventImpl = mb.class_('EventImpl')
> EventImpl.held_type = 'std::auto_ptr< %s >' % EventImpl.decl_string
>
> (EventImpl is the type of the argument that is subject to the ownership
> transfer)
>
> But with the above changes the code doesn't compile (errors below). The
> generated source contains:
>
> bp::class_< EventImpl_wrapper, std::auto_ptr< ::ns3::EventImpl >,
> boost::noncopyable >( "EventImpl" )
> .def( bp::init< >() )
>
> The error comes from the ".def( bp::init< >() )" line. Manually
removing
> the "std::auto_ptr< ::ns3::EventImpl >" part from the first line solves
the
> compilation error.
>
> Any hints?
Yes:
1. change "std::auto_ptr< ::ns3::EventImpl >" to
"std::auto_ptr< EventImpl_wrapper >" and add next line to your code:
ei = mb.class_( 'EventImpl' )
ei.add_registration_code(
"boost::python::implicitly_convertible< std::auto_ptr< EventImpl_wrapper
>, std::auto_ptr< ::ns3::EventImpl > >();"
, False )
This should help.
2. If not, please create small standalone test case, which reproduce the
problem.
I will take a look on it.
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
|
|
From: Roman Y. <rom...@gm...> - 2007-02-19 05:28:29
|
On 2/19/07, Gustavo Carneiro <gjc...@gm...> wrote: > After much effort to rebuilt a patched boost library, it now compiles > fine. However, the methods that are affected by the transformation > (Schedule*) are receiving some kind of name mangling: > > >>> dir(ns3.Simulator) > ['Now', 'Run', > 'ScheduleDestroy_126390ccfc1f22f22289536ea47d3ecb', > 'ScheduleNow_3692da707df52052648f9d5435d958c4', > 'Schedule_fbf096d706be19b392c1e3fbcd3bf544', 'Stop', > '__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', > '__hash__', '__init__', '__module__', '__new__', '__reduce__', > '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__'] > > Bug? No, feature: file:///home/roman/language-binding/production/www/pyplusplus/documentation/functions/transformation/name_mangling.html You can use "alias" property to change it: Simulator = mb.class_( 'Simulator' ) Simulator.mem_fun( 'ScheduleDestroy' ).alias = .... -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Gustavo C. <gjc...@gm...> - 2007-02-18 22:48:33
|
On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > > On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > > On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > > > On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > > > I committed new function transformation: transfer ownership. The > > > documentation > > > > will come later. For usage example take a look on unittests: > > > > ( look for "transfer_ownership" ) > > > > > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/function_transformations_tester.py?view=markup > > > > > > > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/function_transformations_to_be_exported.hpp?view=markup > > > > > > > > Thanks! > > > > > > Unfortunately the generated code does not compile when I try to use > > that > > > feature: > > > > > > default/bindings/python/__call_policies.pypp.hpp:14:58: > > > error: boost/python/suite/indexing/iterator_range.hpp: No > > > such file or directory > > > default/bindings/python/__call_policies.pypp.hpp:172: > > > error: 'pyplusplus::call_policies::bpl::indexing' has not > > > been declared > > > default/bindings/python/__call_policies.pypp.hpp:172: > > > error: ISO C++ forbids declaration of 'iterator_range' with no type > > > default/bindings/python/__call_policies.pypp.hpp:172: > > > error: typedef name may not be a nested-name-specifier > > > default/bindings/python/__call_policies.pypp.hpp:172: > > > error: expected ';' before '<' token > > > [...etc...] > > > > > > From "boost/python/suite/indexing/iterator_range.hpp: No > > > such file or directory", does this sound like I need to get a newer > > > boost::python library or something? > > > > Something like that. Boost.Python, out of the box, comes with little > > support > > to STL containers. To make the long story short you have to install new > > indexing > > suite that comes with Py++: > > ( > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ > > ) > > > > Here you will find more information about new indexing suite > > http://language-binding.net/pyplusplus/documentation/containers.html. > > > > Thanks. > > After much effort to rebuilt a patched boost library, it now compiles > fine. However, the methods that are affected by the transformation > (Schedule*) are receiving some kind of name mangling: > > >>> dir(ns3.Simulator) > ['Now', 'Run', 'ScheduleDestroy_126390ccfc1f22f22289536ea47d3ecb', > 'ScheduleNow_3692da707df52052648f9d5435d958c4', > 'Schedule_fbf096d706be19b392c1e3fbcd3bf544', 'Stop', '__class__', > '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', > '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', > '__repr__', '__setattr__', '__str__', '__weakref__'] > Even with the name mangling problem, something was not right: gjc@nazgul:python$ python test-schedule.py Traceback (most recent call last): File "test-schedule.py", line 11, in <module> Simulator.Schedule_fbf096d706be19b392c1e3fbcd3bf544(t1, ev) Boost.Python.ArgumentError: Python argument types in Simulator.Schedule_fbf096d706be19b392c1e3fbcd3bf544(Time, MyEvent) did not match C++ signature: Schedule_fbf096d706be19b392c1e3fbcd3bf544(ns3::Time time, std::auto_ptr<ns3::EventImpl> event) Then I saw something I missed in the unit test, and added this code: EventImpl = mb.class_('EventImpl') EventImpl.held_type = 'std::auto_ptr< %s >' % EventImpl.decl_string (EventImpl is the type of the argument that is subject to the ownership transfer) But with the above changes the code doesn't compile (errors below). The generated source contains: bp::class_< EventImpl_wrapper, std::auto_ptr< ::ns3::EventImpl >, boost::noncopyable >( "EventImpl" ) .def( bp::init< >() ) The error comes from the ".def( bp::init< >() )" line. Manually removing the "std::auto_ptr< ::ns3::EventImpl >" part from the first line solves the compilation error. Any hints? /usr/include/boost/python/object/pointer_holder.hpp: In constructor 'boost::python::objects::pointer_holder<Pointer, Value>::pointer_holder(PyObject*) [with Pointer = std::auto_ptr<ns3::EventImpl>, Value = ns3::EventImpl]': /usr/include/boost/python/object/make_holder.hpp:83: instantiated from 'static void boost::python::objects::make_holder<0>::apply<Holder, ArgList>::execute(PyObject*) [with Holder = boost::python::objects::pointer_holder<std::auto_ptr<ns3::EventImpl>, ns3::EventImpl>, ArgList = boost::mpl::vector0<mpl_::na>]' /usr/include/boost/python/detail/make_keyword_range_fn.hpp:64: instantiated from 'boost::python::api::object boost::python::detail::make_keyword_range_constructor(const CallPolicies&, const boost::python::detail::keyword_range&, Holder*, ArgList*, Arity*) [with ArgList = boost::mpl::vector0<mpl_::na>, Arity = boost::mpl::size<boost::mpl::vector0<mpl_::na> >, Holder = boost::python::objects::pointer_holder<std::auto_ptr<ns3::EventImpl>, ns3::EventImpl>, CallPolicies = boost::python::default_call_policies]' /usr/include/boost/python/init.hpp:330: instantiated from 'void boost::python::detail::def_init_aux(ClassT&, const Signature&, NArgs, const CallPoliciesT&, const char*, const boost::python::detail::keyword_range&) [with ClassT = boost::python::class_<EventImpl_wrapper, std::auto_ptr<ns3::EventImpl>, boost::noncopyable_::noncopyable, boost::python::detail::not_specified>, CallPoliciesT = boost::python::default_call_policies, Signature = boost::mpl::vector0<mpl_::na>, NArgs = boost::mpl::size<boost::mpl::vector0<mpl_::na> >]' /usr/include/boost/python/init.hpp:399: instantiated from 'static void boost::python::detail::define_class_init_helper<0>::apply(ClassT&, const CallPoliciesT&, const Signature&, NArgs, const char*, const boost::python::detail::keyword_range&) [with ClassT = boost::python::class_<EventImpl_wrapper, std::auto_ptr<ns3::EventImpl>, boost::noncopyable_::noncopyable, boost::python::detail::not_specified>, CallPoliciesT = boost::python::default_call_policies, Signature = boost::mpl::vector0<mpl_::na>, NArgs = boost::mpl::size<boost::mpl::vector0<mpl_::na> >]' /usr/include/boost/python/init.hpp:171: instantiated from 'void boost::python::init_base<DerivedT>::visit(classT&) const [with classT = boost::python::class_<EventImpl_wrapper, std::auto_ptr<ns3::EventImpl>, boost::noncopyable_::noncopyable, boost::python::detail::not_specified>, DerivedT = boost::python::init<mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_>]' /usr/include/boost/python/def_visitor.hpp:31: instantiated from 'static void boost::python::def_visitor_access::visit(const V&, classT&) [with V = boost::python::def_visitor<boost::python::init<mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> >, classT = boost::python::class_<EventImpl_wrapper, std::auto_ptr<ns3::EventImpl>, boost::noncopyable_::noncopyable, boost::python::detail::not_specified>]' /usr/include/boost/python/def_visitor.hpp:67: instantiated from 'void boost::python::def_visitor<DerivedVisitor>::visit(classT&) const [with classT = boost::python::class_<EventImpl_wrapper, std::auto_ptr<ns3::EventImpl>, boost::noncopyable_::noncopyable, boost::python::detail::not_specified>, DerivedVisitor = boost::python::init<mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_>]' /usr/include/boost/python/class.hpp:225: instantiated from 'boost::python::class_<T, X1, X2, X3>& boost::python::class_<T, X1, X2, X3>::def(const boost::python::def_visitor<Derived>&) [with Derived = boost::python::init<mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_>, W = EventImpl_wrapper, X1 = std::auto_ptr<ns3::EventImpl>, X2 = boost::noncopyable_::noncopyable, X3 = boost::python::detail::not_specified]' default/bindings/python/ns3.cpp:76: instantiated from here /usr/include/boost/python/object/pointer_holder.hpp:179: error: cannot allocate an object of abstract type 'ns3::EventImpl' ./default/ns3/event-impl.h:28: note: because the following virtual functions are pure within 'ns3::EventImpl': ./default/ns3/event-impl.h:31: note: virtual ns3::EventImpl::~EventImpl() ./default/ns3/event-impl.h:36: note: virtual void ns3::EventImpl::Notify() -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." |
|
From: Gustavo C. <gjc...@gm...> - 2007-02-18 22:20:49
|
On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > > On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > > I committed new function transformation: transfer ownership. The > > documentation > > > will come later. For usage example take a look on unittests: > > > ( look for "transfer_ownership" ) > > > > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/function_transformations_tester.py?view=markup > > > > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/function_transformations_to_be_exported.hpp?view=markup > > > > Thanks! > > > > Unfortunately the generated code does not compile when I try to use > that > > feature: > > > > default/bindings/python/__call_policies.pypp.hpp:14:58: > > error: boost/python/suite/indexing/iterator_range.hpp: No > > such file or directory > > default/bindings/python/__call_policies.pypp.hpp:172: > > error: 'pyplusplus::call_policies::bpl::indexing' has not > > been declared > > default/bindings/python/__call_policies.pypp.hpp:172: > > error: ISO C++ forbids declaration of 'iterator_range' with no type > > default/bindings/python/__call_policies.pypp.hpp:172: > > error: typedef name may not be a nested-name-specifier > > default/bindings/python/__call_policies.pypp.hpp:172: > > error: expected ';' before '<' token > > [...etc...] > > > > From "boost/python/suite/indexing/iterator_range.hpp: No > > such file or directory", does this sound like I need to get a newer > > boost::python library or something? > > Something like that. Boost.Python, out of the box, comes with little > support > to STL containers. To make the long story short you have to install new > indexing > suite that comes with Py++: > ( > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ > ) > > Here you will find more information about new indexing suite > http://language-binding.net/pyplusplus/documentation/containers.html. Thanks. After much effort to rebuilt a patched boost library, it now compiles fine. However, the methods that are affected by the transformation (Schedule*) are receiving some kind of name mangling: >>> dir(ns3.Simulator) ['Now', 'Run', 'ScheduleDestroy_126390ccfc1f22f22289536ea47d3ecb', 'ScheduleNow_3692da707df52052648f9d5435d958c4', 'Schedule_fbf096d706be19b392c1e3fbcd3bf544', 'Stop', '__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__'] Bug? -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." |
|
From: Roman Y. <rom...@gm...> - 2007-02-18 19:56:37
|
On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > I committed new function transformation: transfer ownership. The > documentation > > will come later. For usage example take a look on unittests: > > ( look for "transfer_ownership" ) > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/function_transformations_tester.py?view=markup > > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/function_transformations_to_be_exported.hpp?view=markup > > Thanks! > > Unfortunately the generated code does not compile when I try to use that > feature: > > default/bindings/python/__call_policies.pypp.hpp:14:58: > error: boost/python/suite/indexing/iterator_range.hpp: No > such file or directory > default/bindings/python/__call_policies.pypp.hpp:172: > error: 'pyplusplus::call_policies::bpl::indexing' has not > been declared > default/bindings/python/__call_policies.pypp.hpp:172: > error: ISO C++ forbids declaration of 'iterator_range' with no type > default/bindings/python/__call_policies.pypp.hpp:172: > error: typedef name may not be a nested-name-specifier > default/bindings/python/__call_policies.pypp.hpp:172: > error: expected ';' before '<' token > [...etc...] > > From "boost/python/suite/indexing/iterator_range.hpp: No > such file or directory", does this sound like I need to get a newer > boost::python library or something? Something like that. Boost.Python, out of the box, comes with little support to STL containers. To make the long story short you have to install new indexing suite that comes with Py++: ( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ ) Here you will find more information about new indexing suite http://language-binding.net/pyplusplus/documentation/containers.html. -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Gustavo C. <gjc...@gm...> - 2007-02-18 19:50:14
|
On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > I committed new function transformation: transfer ownership. The > documentation > will come later. For usage example take a look on unittests: > ( look for "transfer_ownership" ) > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/function_transformations_tester.py?view=markup > > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/function_transformations_to_be_exported.hpp?view=markup Thanks! Unfortunately the generated code does not compile when I try to use that feature: default/bindings/python/__call_policies.pypp.hpp:14:58: error: boost/python/suite/indexing/iterator_range.hpp: No such file or directory default/bindings/python/__call_policies.pypp.hpp:172: error: 'pyplusplus::call_policies::bpl::indexing' has not been declared default/bindings/python/__call_policies.pypp.hpp:172: error: ISO C++ forbids declaration of 'iterator_range' with no type default/bindings/python/__call_policies.pypp.hpp:172: error: typedef name may not be a nested-name-specifier default/bindings/python/__call_policies.pypp.hpp:172: error: expected ';' before '<' token [...etc...] From "boost/python/suite/indexing/iterator_range.hpp: No such file or directory", does this sound like I need to get a newer boost::python library or something? Thanks in advance. -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." |
|
From: Roman Y. <rom...@gm...> - 2007-02-18 19:04:18
|
I committed new function transformation: transfer ownership. The documentation will come later. For usage example take a look on unittests: ( look for "transfer_ownership" ) http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/function_transformations_tester.py?view=markup http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/function_transformations_to_be_exported.hpp?view=markup -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Roman Y. <rom...@gm...> - 2007-02-18 18:24:37
|
On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > > I don't fully understand your situation. Take a look on this FAQ( > > http://boost.org/libs/python/doc/v2/faq.html#ownership ) > > > > Does it help? > > Yes, it seems to match my problem. You have function that takes ownership on a pointer, right? > Now, trying to implement the advice I stumble upon the "Make sure the C++ > object is held by auto_ptr" part, which I have no idea how to do. How do I > tell Py++ to do that? > > If not, please describe your situation with some C++ code. > > > > If it do help, than you can use "function transformation" feature to > > create the wrapper. > > > > ( > http://language-binding.net/pyplusplus/documentation/functions/transformation/built_in/built_in.html > > ) > > That doesn't seem to help; I don't know enough of Py++ internals to be > able to my own function transformation in useful time. Not a problem, I need this too, I will create it for you and will let you know ( +- 1 hour ) -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Gustavo C. <gjc...@gm...> - 2007-02-18 18:20:56
|
On 2/18/07, Roman Yakovenko <rom...@gm...> wrote: > > On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > > Take this program, where Simulator and EventImpl are C++ classes > wrapped > > by Py++ / boost python: > > > > class MyEvent(EventImpl): > > > > def Notify(self, *args): > > print "Notify!", args > > > > > > ev = MyEvent() > > t1 = Seconds(1) > > Simulator.Schedule(t1, ev) > > Simulator.Run() > > > > There's a memory error here because 'ev' is passed into Simulator, > which > > takes ownership of the value and eventually calls delete on C++ > counterpart > > to 'ev'. valgrind says that Address 0xsomething is 8 bytes inside the > > object allocated by boost python. > > > > If, OTOH, I do "Simulator.Schedule(Seconds(1), MyEvent())", then the > > python objects are prematurely destroyed, and another memory error > occurs . > > > > I notice that Py++ "call policies" are always related to the return > value; > > I don't see how to customize the parameter passing aspect of it all. > > > > Any advice is appreciated. Thanks in advance. > > I don't fully understand your situation. Take a look on this FAQ( > http://boost.org/libs/python/doc/v2/faq.html#ownership ) > > Does it help? Yes, it seems to match my problem. Now, trying to implement the advice I stumble upon the "Make sure the C++ object is held by auto_ptr" part, which I have no idea how to do. How do I tell Py++ to do that? If not, please describe your situation with some C++ code. > > If it do help, than you can use "function transformation" feature to > create the wrapper. > > ( > http://language-binding.net/pyplusplus/documentation/functions/transformation/built_in/built_in.html > ) That doesn't seem to help; I don't know enough of Py++ internals to be able to my own function transformation in useful time. -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic." |
|
From: Roman Y. <rom...@gm...> - 2007-02-18 17:36:48
|
On 2/18/07, Gustavo Carneiro <gjc...@gm...> wrote: > Take this program, where Simulator and EventImpl are C++ classes wrapped > by Py++ / boost python: > > class MyEvent(EventImpl): > > def Notify(self, *args): > print "Notify!", args > > > ev = MyEvent() > t1 = Seconds(1) > Simulator.Schedule(t1, ev) > Simulator.Run() > > There's a memory error here because 'ev' is passed into Simulator, which > takes ownership of the value and eventually calls delete on C++ counterpart > to 'ev'. valgrind says that Address 0xsomething is 8 bytes inside the > object allocated by boost python. > > If, OTOH, I do "Simulator.Schedule(Seconds(1), MyEvent())", then the > python objects are prematurely destroyed, and another memory error occurs . > > I notice that Py++ "call policies" are always related to the return value; > I don't see how to customize the parameter passing aspect of it all. > > Any advice is appreciated. Thanks in advance. I don't fully understand your situation. Take a look on this FAQ( http://boost.org/libs/python/doc/v2/faq.html#ownership ) Does it help? If not, please describe your situation with some C++ code. If it do help, than you can use "function transformation" feature to create the wrapper. ( http://language-binding.net/pyplusplus/documentation/functions/transformation/built_in/built_in.html ) Thanks. -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Gustavo C. <gjc...@gm...> - 2007-02-18 15:09:26
|
Take this program, where Simulator and EventImpl are C++ classes wrapped
by Py++ / boost python:
class MyEvent(EventImpl):
def Notify(self, *args):
print "Notify!", args
ev = MyEvent()
t1 = Seconds(1)
Simulator.Schedule(t1, ev)
Simulator.Run()
There's a memory error here because 'ev' is passed into Simulator, which
takes ownership of the value and eventually calls delete on C++ counterpart
to 'ev'. valgrind says that Address 0xsomething is 8 bytes inside the
object allocated by boost python.
If, OTOH, I do "Simulator.Schedule(Seconds(1), MyEvent())", then the
python objects are prematurely destroyed, and another memory error occurs.
I notice that Py++ "call policies" are always related to the return value;
I don't see how to customize the parameter passing aspect of it all.
Any advice is appreciated. Thanks in advance.
--
Gustavo J. A. M. Carneiro
"The universe is always one step beyond logic."
|
|
From: Roman Y. <rom...@gm...> - 2007-02-17 18:06:44
|
On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > > >>> I agree. When I saw that it surprised me. Is there anything you can > >>> think of that may have changed in the past several months that would > >>> increase the number of times that the full_name code is being called? > >>> In other words, is there a new algorithm in the code that needs to be > >>> made smarter to increase performance? > >>> > > > > Can you do me a favor: > > pygccxml.declarations.declaration_t. > > _get_name function implemented as > > > > > > def _get_name( self ): > > > > return self._get_name_impl() > > > > > > can you change the implementation to "return self._name" and to run this under > > profiler? I expect this change to increase performance. > > > That increased performance by about 5%. So a small speedup, but not a > significant speedup. Any other ideas? I thought about it and I don't know. May be you are doing something wrong. Python-Ogre project is not a smallest one, but it takes only 5 minutes( without cache and files ) to generate code. During code generation they also parse Ogre source tree to extract documentation. Can you show your script? -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Roman Y. <rom...@gm...> - 2007-02-17 17:51:49
|
On 2/17/07, Gustavo Carneiro <gjc...@gm...> wrote:
> Nevermind; it seems that I need to call mb.build_code_creator(module_name)
> after removing the declarations, not before.
This definitely could help, but better is to use include\exclude functionality
of declarations:
Simulator = mb.class_('Simulator')
Simulator.decls().exclude()
Take a look on this example:
http://language-binding.net/pyplusplus/documentation/tutorials/module_builder/generate_code.py.html
Also you can remove declarations, it is a bad practices. Py++ uses all
declarations
to verify that you create good and usable bindings.
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
|
|
From: Gustavo C. <gjc...@gm...> - 2007-02-17 16:58:27
|
Nevermind; it seems that I need to call mb.build_code_creator(module_name)
after removing the declarations, not before.
On 2/17/07, Gustavo Carneiro <gjc...@gm...> wrote:
>
> I want to make Py++ scan a header file but ignore all declarations
> except a few handpicked ones, since most of the declarations produce
> compilation errors. I am trying this code but it doesn't seem to have any
> effect:
>
> Simulator = mb.class_('Simulator')
> for decl in Simulator.get_members():
> print decl
> Simulator.remove_declaration(decl)
>
> Any ideas?
>
> --
> Gustavo J. A. M. Carneiro
> "The universe is always one step beyond logic."
--
Gustavo J. A. M. Carneiro
"The universe is always one step beyond logic."
|
|
From: Gustavo C. <gjc...@gm...> - 2007-02-17 16:24:02
|
I want to make Py++ scan a header file but ignore all declarations except
a few handpicked ones, since most of the declarations produce compilation
errors. I am trying this code but it doesn't seem to have any effect:
Simulator = mb.class_('Simulator')
for decl in Simulator.get_members():
print decl
Simulator.remove_declaration(decl)
Any ideas?
--
Gustavo J. A. M. Carneiro
"The universe is always one step beyond logic."
|
|
From: Allen B. <al...@vr...> - 2007-02-16 21:21:21
|
>>> I agree. When I saw that it surprised me. Is there anything you can >>> think of that may have changed in the past several months that would >>> increase the number of times that the full_name code is being called? >>> In other words, is there a new algorithm in the code that needs to be >>> made smarter to increase performance? >>> > > Can you do me a favor: > pygccxml.declarations.declaration_t. > _get_name function implemented as > > > def _get_name( self ): > > return self._get_name_impl() > > > can you change the implementation to "return self._name" and to run this under > profiler? I expect this change to increase performance. > That increased performance by about 5%. So a small speedup, but not a significant speedup. Any other ideas? -Allen |
|
From: Roman Y. <rom...@gm...> - 2007-02-16 20:48:45
|
On 2/16/07, Allen Bierbaum <al...@vr...> wrote:
> Roman Yakovenko wrote:
> > I need to study the report. The funny thing is that we do have cache for
> > "full_name" algorithm.
> >
> I agree. When I saw that it surprised me. Is there anything you can
> think of that may have changed in the past several months that would
> increase the number of times that the full_name code is being called?
> In other words, is there a new algorithm in the code that needs to be
> made smarter to increase performance?
Can you do me a favor:
pygccxml.declarations.declaration_t.
_get_name function implemented as
def _get_name( self ):
return self._get_name_impl()
can you change the implementation to "return self._name" and to run this under
profiler? I expect this change to increase performance.
Thanks
P.S. Please don't over quote.
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
|
|
From: Allen B. <al...@vr...> - 2007-02-16 20:40:29
|
Roman Yakovenko wrote: > I need to study the report. The funny thing is that we do have cache for > "full_name" algorithm. > I agree. When I saw that it surprised me. Is there anything you can think of that may have changed in the past several months that would increase the number of times that the full_name code is being called? In other words, is there a new algorithm in the code that needs to be made smarter to increase performance? -Allen > > May be solution could come from gccxml. It seems that gccxml reports > the class names right. I need to check this. Also I have to add some support to > gccxml version, so I can safety select needed functionality. > > On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > >> Roman Yakovenko wrote: >> >>> On 2/16/07, Allen Bierbaum <al...@vr...> wrote: >>> >>>> Is there any user API to disable the dependency manager (atleast during >>>> debugging and development)? >>>> >>> No, it does not exists. >>> >>> >>>> What is the basic structure of the dependency management search >>>> algorithm? Is it something like: >>>> >>>> - For every part of an interface exposed >>>> - For each item used in the interface >>>> - Look up the item and make sure that is exposed >>>> >>>> If it is something like this, then that could definitely be biting me. >>>> I have a *huge* number of symbols and something O(N^2) like this could >>>> really consume a lot of time. >>>> >>> Yes. >>> >>> >>>> In the meantime, I am trying to collect some profiler information to >>>> find out exactly where the time is being spent. Once I have those >>>> numbers collected I will pass them along in case there is anything that >>>> can be done to increase performance. >>>> >>> Lets wait for results. I want to be sure before I introduce >>> functionality, which >>> disables dependency manager. >>> >> You are right, removing the dependency manager did not help performance. >> >> I have attached the detailed performance profile results to this e-mail >> with the results sorted by total time in the given methods. >> >> The summary is: >> >> ncalls tottime percall cumtime percall >> filename:lineno(function) >> 49849182 452.827 0.000 755.786 0.000 >> .../pygccxml/declarations/algorithm.py:37(full_name) >> 50225914 338.388 0.000 1228.453 0.000 >> .../pygccxml/declarations/matchers.py:224(check_name) >> 50490184 244.846 0.000 1538.742 0.000 >> .../pygccxml/declarations/matchers.py:205(__call__) >> 151030576 198.249 0.000 198.249 0.000 >> .../pygccxml/declarations/matchers.py:160(_get_name) >> 102343991 169.654 0.000 169.654 0.000 >> .../pygccxml/declarations/declaration.py:241(cache) >> 49819921 153.485 0.000 1682.796 0.000 >> .../pygccxml/declarations/scopedef.py:273(<lambda>) >> 99681702 136.791 0.000 136.791 0.000 >> .../pygccxml/declarations/algorithms_cache.py:30(_get_full_name) >> 50591 84.602 0.002 1783.335 0.035 >> .../pygccxml/declarations/matcher.py:33(find) >> 1 46.046 46.046 46.047 46.047 >> .../pygccxml/parser/declarations_cache.py:166(flush) >> 1 32.485 32.485 32.535 32.535 >> .../pygccxml/parser/source_reader.py:151(create_xml_file) >> 6483/1 26.067 0.004 49.694 49.694 >> .../pygccxml/declarations/scopedef.py:144(init_optimizer) >> 6219137 12.618 0.000 12.618 0.000 >> .../pyplusplus/decl_wrappers/algorithm.py:88(<lambda>) >> 12960 9.767 0.001 27.079 0.002 >> .../pyplusplus/decl_wrappers/algorithm.py:80(create_identifier) >> 3353123 9.533 0.000 21.040 0.000 >> .../pygccxml/declarations/declaration.py:153(_get_name) >> >> I need to look into it closer, but it looks like there is something in >> the code that is checking and/or getting the name of decls a *huge* >> number of times as part of some algorithm. This may be a good >> opportunity for caching if we can be sure the value is set. >> >> I know from our previous discussions of caching that you had two big >> worries (that I share with you by the way). >> 1. Caching doesn't work well in some cases where user code may want to >> change the values being cached (runtime modification) >> 2. It makes the code ugly putting all these caches everywhere. >> >> That got me thinking about the python memoization pattern that can be >> handled by decorators. See: >> http://www.phyast.pitt.edu/~micheles/python/documentation.html#memoize and >> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325205 >> >> What I was thinking is that in the context of py++ it would be nice if >> the user could make whatever changes they want to the system and then >> once the state is "set", call a method that turns on caching from that >> point forward. >> >> Something like: >> >> ------------------------------ >> mb = ModuleBuilder(....) >> >> <do user changes> >> >> pyplusplus.memoizer.enable() # Everything we cache is set from here on >> ------------------------------ >> >> We could also introduce an enabled state that asserted that the cache >> value equals the value that would have been calculated. This would >> give a nice sanity check to use during debugging to make sure the cache >> is not breaking things. >> >> Anyway..... with a memoization pattern then the py++ internal code could >> look something like this: >> >> class declaration_t: >> .... >> @memoize >> def getName(...): >> return value as normal. >> >> So you see I think we could address both issues aboved. The code would >> allow for runtime modification by users until they say they are done. >> It would also allow for clean code because you could just put a >> decorator (@memoize) on methods that needed it and then they would >> automatically work. >> >> What do you think? Is this a way we could increase the performance but >> still keep the code clean and flexible? >> >> -Allen >> >> >> >> 617979712 function calls (616759372 primitive calls) in 2162.516 CPU seconds >> >> Ordered by: internal time >> List reduced from 1554 to 100 due to restriction <100> >> >> ncalls tottime percall cumtime percall filename:lineno(function) >> 49849182/49848887 452.827 0.000 755.786 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:37(full_name) >> 50225914 338.388 0.000 1228.453 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:224(check_name) >> 50490184 244.846 0.000 1538.742 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:205(__call__) >> 151030576 198.249 0.000 198.249 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:160(_get_name) >> 102343991 169.654 0.000 169.654 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:241(cache) >> 49819921 153.485 0.000 1682.796 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:273(<lambda>) >> 99681702 136.791 0.000 136.791 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:30(_get_full_name) >> 50591 84.602 0.002 1783.335 0.035 /home/allenb/python/lib/python/pygccxml/declarations/matcher.py:33(find) >> 1 46.046 46.046 46.047 46.047 /home/allenb/python/lib/python/pygccxml/parser/declarations_cache.py:166(flush) >> 1 32.485 32.485 32.535 32.535 /home/allenb/python/lib/python/pygccxml/parser/source_reader.py:151(create_xml_file) >> 6483/1 26.067 0.004 49.694 49.694 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:144(init_optimizer) >> 6219137 12.618 0.000 12.618 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:88(<lambda>) >> 12960 9.767 0.001 27.079 0.002 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:80(create_identifier) >> 3353123/3353028 9.533 0.000 21.040 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:153(_get_name) >> 1784028 9.249 0.000 22.834 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:42(remove_alias) >> 277051 7.772 0.000 7.772 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:112(__decl_types) >> 755882/755552 6.722 0.000 11.733 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:10(declaration_path) >> 487343 6.428 0.000 10.145 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:14(__init__) >> 2154824 5.373 0.000 10.755 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:40(make_flatten_generator) >> 2356010/2337404 5.234 0.000 5.382 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:42(proceed_single) >> 50296 5.103 0.000 1795.412 0.036 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:331(_find_multiple) >> 637 4.441 0.007 43.550 0.068 /usr/lib64/python2.4/site-packages/_xmlplus/sax/expatreader.py:205(feed) >> 291 4.374 0.015 16.067 0.055 /home/allenb/python/lib/python/pyplusplus/code_creators/class_declaration.py:113(_exported_base_classes) >> 2979382 4.353 0.000 4.353 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:150(_get_name_impl) >> 373741/373740 4.337 0.000 7.154 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:117(_get_name_impl) >> 487343 3.717 0.000 3.717 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:95(__init__) >> 2711357 3.436 0.000 3.436 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:99(_get_remove_alias) >> 174708 3.218 0.000 33.638 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:166(startElement) >> 46858 3.118 0.000 3.389 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:30(_get_left_siblings) >> 169606/102367 2.694 0.000 4.253 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:71(proceed_single) >> 96123 2.473 0.000 3.149 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:207(__read_location) >> 286719 2.422 0.000 11.120 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:45(__eq__) >> 879364/455629 2.339 0.000 8.790 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:40(_get_decl_string) >> 1145583 2.229 0.000 2.229 0.000 /home/allenb/python/lib/python/pygccxml/parser/patcher.py:113(<lambda>) >> 356 2.213 0.006 4.759 0.013 /home/allenb/python/lib/python/pygccxml/parser/patcher.py:109(__find_enum) >> 1389745 2.164 0.000 2.164 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:171(_get_parent) >> 1026249 2.124 0.000 2.124 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/xmlreader.py:324(get) >> 100654 2.004 0.000 2.004 0.000 /usr/lib64/python2.4/posixpath.py:373(normpath) >> 295853/86792 1.991 0.000 2.331 0.000 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:453(get_from_type) >> 1493074 1.872 0.000 1.872 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:60(_get_declaration_path) >> 96175 1.739 0.000 1.739 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/decl_wrapper.py:25(__init__) >> 290492 1.715 0.000 15.668 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:265(remove_cv) >> 1 1.671 1.671 72.154 72.154 /home/allenb/python/lib/python/pygccxml/parser/source_reader.py:310(__parse_gccxml_created_file) >> 34097 1.613 0.000 5.760 0.000 /home/allenb/python/lib/python/pyplusplus/module_creator/call_policies_resolver.py:53(__call__) >> 57738 1.597 0.000 4.142 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:319(__read_calldef) >> 1 1.536 1.536 6.534 6.534 /home/allenb/python/lib/python/pyplusplus/module_builder/builder.py:138(__filter_by_location) >> 146338 1.524 0.000 2.233 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:256(get_members) >> 96175 1.513 0.000 2.848 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:71(__init__) >> 985059 1.491 0.000 1.491 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:196(_get_location) >> 229891/116390 1.484 0.000 1.894 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:66(decompose_type) >> 52710 1.458 0.000 2.649 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:307(__read_argument) >> 114946 1.351 0.000 2.088 0.000 /home/allenb/python/lib/python/pygccxml/parser/linker.py:24(_set_inst) >> 96175 1.335 0.000 1.335 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:12(__init__) >> 1 1.330 1.330 1.346 1.346 /home/allenb/python/lib/python/pygccxml/parser/declarations_cache.py:141(__load) >> 947095 1.287 0.000 1.287 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:326(_get_base) >> 122541 1.261 0.000 4.489 0.000 /home/allenb/python/lib/python/pygccxml/declarations/function_traits.py:13(is_same_function) >> 217304 1.223 0.000 9.077 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:24(__eq__) >> 375145 1.209 0.000 7.670 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:216(is_const) >> 429350 1.183 0.000 1.655 0.000 extensions/pyexpat.c:465(CharacterData) >> 163346/70667 1.174 0.000 1.485 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:31(__remove_alias) >> 79084 1.153 0.000 3.959 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:282(adopt_declaration) >> 57738 1.151 0.000 1.999 0.000 /home/allenb/python/lib/python/pygccxml/parser/linker.py:61(__link_calldef) >> 581 1.118 0.002 1.408 0.002 /home/allenb/python/lib/python/pyplusplus/file_writers/writer.py:54(write_file) >> 624197 1.102 0.000 1.102 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/xmlreader.py:315(__getitem__) >> 254332 1.069 0.000 1.069 0.000 /usr/lib64/python2.4/logging/__init__.py:1136(getEffectiveLevel) >> 322753 1.062 0.000 1.062 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:148(does_match_exist) >> 174708 1.055 0.000 1.055 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:203(endElement) >> 174708 1.050 0.000 34.924 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/expatreader.py:310(start_element) >> 303849 1.031 0.000 6.214 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:322(__init__) >> 57752 1.003 0.000 2.576 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:371(function_type) >> 50590 0.998 0.000 4.635 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:262(__create_matcher) >> 6431 0.994 0.000 1.169 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:103(__init__) >> 53310 0.954 0.000 5.511 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:336(__read_member_function) >> 290492 0.953 0.000 2.812 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:249(is_volatile) >> 322753 0.949 0.000 2.011 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:171(__call__) >> 699219 0.945 0.000 0.945 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:26(enabled) >> 189878/189770 0.924 0.000 3.226 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:350(_create_decl_string) >> 724484 0.903 0.000 0.903 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:50(_get_demangled_name) >> 12960 0.891 0.000 4.366 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:37(_get_definition_set) >> 253539 0.889 0.000 1.954 0.000 /usr/lib64/python2.4/logging/__init__.py:943(debug) >> 241132 0.873 0.000 4.267 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:233(remove_declarated) >> 1 0.864 0.864 3.924 3.924 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:424(_relink_declarated_types) >> 81593 0.857 0.000 1.468 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:320(find_out_member_access_type) >> 1 0.852 0.852 12.578 12.578 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:359(_join_class_hierarchy) >> 269445 0.847 0.000 1.269 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/decl_wrapper.py:113(exclude) >> 159205 0.835 0.000 0.835 0.000 /home/allenb/python/lib/python/pygccxml/declarations/pattern_parser.py:24(has_pattern) >> 627455 0.825 0.000 0.825 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:60(_get_related_class) >> 10535 0.811 0.000 2.844 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:175(find_all_declarations) >> 57350 0.808 0.000 2.596 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:108(__init__) >> 96156 0.791 0.000 2.486 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:70(reset) >> 72203 0.753 0.000 4.109 0.000 /home/allenb/python/lib/python/pygccxml/declarations/container_traits.py:40(get_container_or_none) >> 435851 0.750 0.000 0.750 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:145(_get_arguments) >> 30997 0.729 0.000 3.546 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/__init__.py:118(create_member_function) >> 116184 0.723 0.000 1.176 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:233(__read_access) >> 388905 0.721 0.000 0.721 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:101(<lambda>) >> 159844 0.715 0.000 2.141 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:341(__call__) >> 457956 0.704 0.000 0.704 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:42(_get_file_name) >> 96125 0.703 0.000 1.609 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:297(__call__) >> 73767 0.694 0.000 2.680 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:118(__init__) >> 96175 0.667 0.000 1.048 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:210(__update_membership) >> >> >> >> ------------------------------------------------------------------------- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to share your >> opinions on IT & business topics through brief surveys-and earn cash >> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> pygccxml-development mailing list >> pyg...@li... >> https://lists.sourceforge.net/lists/listinfo/pygccxml-development >> >> >> > > > |
|
From: Roman Y. <rom...@gm...> - 2007-02-16 20:10:00
|
I need to study the report. The funny thing is that we do have cache for "full_name" algorithm. May be solution could come from gccxml. It seems that gccxml reports the class names right. I need to check this. Also I have to add some support to gccxml version, so I can safety select needed functionality. On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > Roman Yakovenko wrote: > > On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > >> Is there any user API to disable the dependency manager (atleast during > >> debugging and development)? > > > > No, it does not exists. > > > >> What is the basic structure of the dependency management search > >> algorithm? Is it something like: > >> > >> - For every part of an interface exposed > >> - For each item used in the interface > >> - Look up the item and make sure that is exposed > >> > >> If it is something like this, then that could definitely be biting me. > >> I have a *huge* number of symbols and something O(N^2) like this could > >> really consume a lot of time. > > > > Yes. > > > >> In the meantime, I am trying to collect some profiler information to > >> find out exactly where the time is being spent. Once I have those > >> numbers collected I will pass them along in case there is anything that > >> can be done to increase performance. > > > > Lets wait for results. I want to be sure before I introduce > > functionality, which > > disables dependency manager. > You are right, removing the dependency manager did not help performance. > > I have attached the detailed performance profile results to this e-mail > with the results sorted by total time in the given methods. > > The summary is: > > ncalls tottime percall cumtime percall > filename:lineno(function) > 49849182 452.827 0.000 755.786 0.000 > .../pygccxml/declarations/algorithm.py:37(full_name) > 50225914 338.388 0.000 1228.453 0.000 > .../pygccxml/declarations/matchers.py:224(check_name) > 50490184 244.846 0.000 1538.742 0.000 > .../pygccxml/declarations/matchers.py:205(__call__) > 151030576 198.249 0.000 198.249 0.000 > .../pygccxml/declarations/matchers.py:160(_get_name) > 102343991 169.654 0.000 169.654 0.000 > .../pygccxml/declarations/declaration.py:241(cache) > 49819921 153.485 0.000 1682.796 0.000 > .../pygccxml/declarations/scopedef.py:273(<lambda>) > 99681702 136.791 0.000 136.791 0.000 > .../pygccxml/declarations/algorithms_cache.py:30(_get_full_name) > 50591 84.602 0.002 1783.335 0.035 > .../pygccxml/declarations/matcher.py:33(find) > 1 46.046 46.046 46.047 46.047 > .../pygccxml/parser/declarations_cache.py:166(flush) > 1 32.485 32.485 32.535 32.535 > .../pygccxml/parser/source_reader.py:151(create_xml_file) > 6483/1 26.067 0.004 49.694 49.694 > .../pygccxml/declarations/scopedef.py:144(init_optimizer) > 6219137 12.618 0.000 12.618 0.000 > .../pyplusplus/decl_wrappers/algorithm.py:88(<lambda>) > 12960 9.767 0.001 27.079 0.002 > .../pyplusplus/decl_wrappers/algorithm.py:80(create_identifier) > 3353123 9.533 0.000 21.040 0.000 > .../pygccxml/declarations/declaration.py:153(_get_name) > > I need to look into it closer, but it looks like there is something in > the code that is checking and/or getting the name of decls a *huge* > number of times as part of some algorithm. This may be a good > opportunity for caching if we can be sure the value is set. > > I know from our previous discussions of caching that you had two big > worries (that I share with you by the way). > 1. Caching doesn't work well in some cases where user code may want to > change the values being cached (runtime modification) > 2. It makes the code ugly putting all these caches everywhere. > > That got me thinking about the python memoization pattern that can be > handled by decorators. See: > http://www.phyast.pitt.edu/~micheles/python/documentation.html#memoize and > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325205 > > What I was thinking is that in the context of py++ it would be nice if > the user could make whatever changes they want to the system and then > once the state is "set", call a method that turns on caching from that > point forward. > > Something like: > > ------------------------------ > mb = ModuleBuilder(....) > > <do user changes> > > pyplusplus.memoizer.enable() # Everything we cache is set from here on > ------------------------------ > > We could also introduce an enabled state that asserted that the cache > value equals the value that would have been calculated. This would > give a nice sanity check to use during debugging to make sure the cache > is not breaking things. > > Anyway..... with a memoization pattern then the py++ internal code could > look something like this: > > class declaration_t: > .... > @memoize > def getName(...): > return value as normal. > > So you see I think we could address both issues aboved. The code would > allow for runtime modification by users until they say they are done. > It would also allow for clean code because you could just put a > decorator (@memoize) on methods that needed it and then they would > automatically work. > > What do you think? Is this a way we could increase the performance but > still keep the code clean and flexible? > > -Allen > > > > 617979712 function calls (616759372 primitive calls) in 2162.516 CPU seconds > > Ordered by: internal time > List reduced from 1554 to 100 due to restriction <100> > > ncalls tottime percall cumtime percall filename:lineno(function) > 49849182/49848887 452.827 0.000 755.786 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:37(full_name) > 50225914 338.388 0.000 1228.453 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:224(check_name) > 50490184 244.846 0.000 1538.742 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:205(__call__) > 151030576 198.249 0.000 198.249 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:160(_get_name) > 102343991 169.654 0.000 169.654 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:241(cache) > 49819921 153.485 0.000 1682.796 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:273(<lambda>) > 99681702 136.791 0.000 136.791 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:30(_get_full_name) > 50591 84.602 0.002 1783.335 0.035 /home/allenb/python/lib/python/pygccxml/declarations/matcher.py:33(find) > 1 46.046 46.046 46.047 46.047 /home/allenb/python/lib/python/pygccxml/parser/declarations_cache.py:166(flush) > 1 32.485 32.485 32.535 32.535 /home/allenb/python/lib/python/pygccxml/parser/source_reader.py:151(create_xml_file) > 6483/1 26.067 0.004 49.694 49.694 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:144(init_optimizer) > 6219137 12.618 0.000 12.618 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:88(<lambda>) > 12960 9.767 0.001 27.079 0.002 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:80(create_identifier) > 3353123/3353028 9.533 0.000 21.040 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:153(_get_name) > 1784028 9.249 0.000 22.834 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:42(remove_alias) > 277051 7.772 0.000 7.772 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:112(__decl_types) > 755882/755552 6.722 0.000 11.733 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:10(declaration_path) > 487343 6.428 0.000 10.145 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:14(__init__) > 2154824 5.373 0.000 10.755 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:40(make_flatten_generator) > 2356010/2337404 5.234 0.000 5.382 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:42(proceed_single) > 50296 5.103 0.000 1795.412 0.036 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:331(_find_multiple) > 637 4.441 0.007 43.550 0.068 /usr/lib64/python2.4/site-packages/_xmlplus/sax/expatreader.py:205(feed) > 291 4.374 0.015 16.067 0.055 /home/allenb/python/lib/python/pyplusplus/code_creators/class_declaration.py:113(_exported_base_classes) > 2979382 4.353 0.000 4.353 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:150(_get_name_impl) > 373741/373740 4.337 0.000 7.154 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:117(_get_name_impl) > 487343 3.717 0.000 3.717 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:95(__init__) > 2711357 3.436 0.000 3.436 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:99(_get_remove_alias) > 174708 3.218 0.000 33.638 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:166(startElement) > 46858 3.118 0.000 3.389 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:30(_get_left_siblings) > 169606/102367 2.694 0.000 4.253 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:71(proceed_single) > 96123 2.473 0.000 3.149 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:207(__read_location) > 286719 2.422 0.000 11.120 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:45(__eq__) > 879364/455629 2.339 0.000 8.790 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:40(_get_decl_string) > 1145583 2.229 0.000 2.229 0.000 /home/allenb/python/lib/python/pygccxml/parser/patcher.py:113(<lambda>) > 356 2.213 0.006 4.759 0.013 /home/allenb/python/lib/python/pygccxml/parser/patcher.py:109(__find_enum) > 1389745 2.164 0.000 2.164 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:171(_get_parent) > 1026249 2.124 0.000 2.124 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/xmlreader.py:324(get) > 100654 2.004 0.000 2.004 0.000 /usr/lib64/python2.4/posixpath.py:373(normpath) > 295853/86792 1.991 0.000 2.331 0.000 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:453(get_from_type) > 1493074 1.872 0.000 1.872 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:60(_get_declaration_path) > 96175 1.739 0.000 1.739 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/decl_wrapper.py:25(__init__) > 290492 1.715 0.000 15.668 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:265(remove_cv) > 1 1.671 1.671 72.154 72.154 /home/allenb/python/lib/python/pygccxml/parser/source_reader.py:310(__parse_gccxml_created_file) > 34097 1.613 0.000 5.760 0.000 /home/allenb/python/lib/python/pyplusplus/module_creator/call_policies_resolver.py:53(__call__) > 57738 1.597 0.000 4.142 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:319(__read_calldef) > 1 1.536 1.536 6.534 6.534 /home/allenb/python/lib/python/pyplusplus/module_builder/builder.py:138(__filter_by_location) > 146338 1.524 0.000 2.233 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:256(get_members) > 96175 1.513 0.000 2.848 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:71(__init__) > 985059 1.491 0.000 1.491 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:196(_get_location) > 229891/116390 1.484 0.000 1.894 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:66(decompose_type) > 52710 1.458 0.000 2.649 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:307(__read_argument) > 114946 1.351 0.000 2.088 0.000 /home/allenb/python/lib/python/pygccxml/parser/linker.py:24(_set_inst) > 96175 1.335 0.000 1.335 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:12(__init__) > 1 1.330 1.330 1.346 1.346 /home/allenb/python/lib/python/pygccxml/parser/declarations_cache.py:141(__load) > 947095 1.287 0.000 1.287 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:326(_get_base) > 122541 1.261 0.000 4.489 0.000 /home/allenb/python/lib/python/pygccxml/declarations/function_traits.py:13(is_same_function) > 217304 1.223 0.000 9.077 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:24(__eq__) > 375145 1.209 0.000 7.670 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:216(is_const) > 429350 1.183 0.000 1.655 0.000 extensions/pyexpat.c:465(CharacterData) > 163346/70667 1.174 0.000 1.485 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:31(__remove_alias) > 79084 1.153 0.000 3.959 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:282(adopt_declaration) > 57738 1.151 0.000 1.999 0.000 /home/allenb/python/lib/python/pygccxml/parser/linker.py:61(__link_calldef) > 581 1.118 0.002 1.408 0.002 /home/allenb/python/lib/python/pyplusplus/file_writers/writer.py:54(write_file) > 624197 1.102 0.000 1.102 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/xmlreader.py:315(__getitem__) > 254332 1.069 0.000 1.069 0.000 /usr/lib64/python2.4/logging/__init__.py:1136(getEffectiveLevel) > 322753 1.062 0.000 1.062 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:148(does_match_exist) > 174708 1.055 0.000 1.055 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:203(endElement) > 174708 1.050 0.000 34.924 0.000 /usr/lib64/python2.4/site-packages/_xmlplus/sax/expatreader.py:310(start_element) > 303849 1.031 0.000 6.214 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:322(__init__) > 57752 1.003 0.000 2.576 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:371(function_type) > 50590 0.998 0.000 4.635 0.000 /home/allenb/python/lib/python/pygccxml/declarations/scopedef.py:262(__create_matcher) > 6431 0.994 0.000 1.169 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:103(__init__) > 53310 0.954 0.000 5.511 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:336(__read_member_function) > 290492 0.953 0.000 2.812 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:249(is_volatile) > 322753 0.949 0.000 2.011 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:171(__call__) > 699219 0.945 0.000 0.945 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:26(enabled) > 189878/189770 0.924 0.000 3.226 0.000 /home/allenb/python/lib/python/pygccxml/declarations/cpptypes.py:350(_create_decl_string) > 724484 0.903 0.000 0.903 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:50(_get_demangled_name) > 12960 0.891 0.000 4.366 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/algorithm.py:37(_get_definition_set) > 253539 0.889 0.000 1.954 0.000 /usr/lib64/python2.4/logging/__init__.py:943(debug) > 241132 0.873 0.000 4.267 0.000 /home/allenb/python/lib/python/pygccxml/declarations/type_traits.py:233(remove_declarated) > 1 0.864 0.864 3.924 3.924 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:424(_relink_declarated_types) > 81593 0.857 0.000 1.468 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:320(find_out_member_access_type) > 1 0.852 0.852 12.578 12.578 /home/allenb/python/lib/python/pygccxml/parser/project_reader.py:359(_join_class_hierarchy) > 269445 0.847 0.000 1.269 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/decl_wrapper.py:113(exclude) > 159205 0.835 0.000 0.835 0.000 /home/allenb/python/lib/python/pygccxml/declarations/pattern_parser.py:24(has_pattern) > 627455 0.825 0.000 0.825 0.000 /home/allenb/python/lib/python/pygccxml/declarations/class_declaration.py:60(_get_related_class) > 10535 0.811 0.000 2.844 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithm.py:175(find_all_declarations) > 57350 0.808 0.000 2.596 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:108(__init__) > 96156 0.791 0.000 2.486 0.000 /home/allenb/python/lib/python/pygccxml/declarations/algorithms_cache.py:70(reset) > 72203 0.753 0.000 4.109 0.000 /home/allenb/python/lib/python/pygccxml/declarations/container_traits.py:40(get_container_or_none) > 435851 0.750 0.000 0.750 0.000 /home/allenb/python/lib/python/pygccxml/declarations/calldef.py:145(_get_arguments) > 30997 0.729 0.000 3.546 0.000 /home/allenb/python/lib/python/pyplusplus/decl_wrappers/__init__.py:118(create_member_function) > 116184 0.723 0.000 1.176 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:233(__read_access) > 388905 0.721 0.000 0.721 0.000 /home/allenb/python/lib/python/pyplusplus/code_creators/algorithm.py:101(<lambda>) > 159844 0.715 0.000 2.141 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:341(__call__) > 457956 0.704 0.000 0.704 0.000 /home/allenb/python/lib/python/pygccxml/declarations/declaration.py:42(_get_file_name) > 96125 0.703 0.000 1.609 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:297(__call__) > 73767 0.694 0.000 2.680 0.000 /home/allenb/python/lib/python/pygccxml/declarations/matchers.py:118(__init__) > 96175 0.667 0.000 1.048 0.000 /home/allenb/python/lib/python/pygccxml/parser/scanner.py:210(__update_membership) > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys-and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > pygccxml-development mailing list > pyg...@li... > https://lists.sourceforge.net/lists/listinfo/pygccxml-development > > -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Allen B. <al...@vr...> - 2007-02-16 19:41:48
|
Roman Yakovenko wrote:
> On 2/16/07, Allen Bierbaum <al...@vr...> wrote:
>> Is there any user API to disable the dependency manager (atleast during
>> debugging and development)?
>
> No, it does not exists.
>
>> What is the basic structure of the dependency management search
>> algorithm? Is it something like:
>>
>> - For every part of an interface exposed
>> - For each item used in the interface
>> - Look up the item and make sure that is exposed
>>
>> If it is something like this, then that could definitely be biting me.
>> I have a *huge* number of symbols and something O(N^2) like this could
>> really consume a lot of time.
>
> Yes.
>
>> In the meantime, I am trying to collect some profiler information to
>> find out exactly where the time is being spent. Once I have those
>> numbers collected I will pass them along in case there is anything that
>> can be done to increase performance.
>
> Lets wait for results. I want to be sure before I introduce
> functionality, which
> disables dependency manager.
You are right, removing the dependency manager did not help performance.
I have attached the detailed performance profile results to this e-mail
with the results sorted by total time in the given methods.
The summary is:
ncalls tottime percall cumtime percall
filename:lineno(function)
49849182 452.827 0.000 755.786 0.000
.../pygccxml/declarations/algorithm.py:37(full_name)
50225914 338.388 0.000 1228.453 0.000
.../pygccxml/declarations/matchers.py:224(check_name)
50490184 244.846 0.000 1538.742 0.000
.../pygccxml/declarations/matchers.py:205(__call__)
151030576 198.249 0.000 198.249 0.000
.../pygccxml/declarations/matchers.py:160(_get_name)
102343991 169.654 0.000 169.654 0.000
.../pygccxml/declarations/declaration.py:241(cache)
49819921 153.485 0.000 1682.796 0.000
.../pygccxml/declarations/scopedef.py:273(<lambda>)
99681702 136.791 0.000 136.791 0.000
.../pygccxml/declarations/algorithms_cache.py:30(_get_full_name)
50591 84.602 0.002 1783.335 0.035
.../pygccxml/declarations/matcher.py:33(find)
1 46.046 46.046 46.047 46.047
.../pygccxml/parser/declarations_cache.py:166(flush)
1 32.485 32.485 32.535 32.535
.../pygccxml/parser/source_reader.py:151(create_xml_file)
6483/1 26.067 0.004 49.694 49.694
.../pygccxml/declarations/scopedef.py:144(init_optimizer)
6219137 12.618 0.000 12.618 0.000
.../pyplusplus/decl_wrappers/algorithm.py:88(<lambda>)
12960 9.767 0.001 27.079 0.002
.../pyplusplus/decl_wrappers/algorithm.py:80(create_identifier)
3353123 9.533 0.000 21.040 0.000
.../pygccxml/declarations/declaration.py:153(_get_name)
I need to look into it closer, but it looks like there is something in
the code that is checking and/or getting the name of decls a *huge*
number of times as part of some algorithm. This may be a good
opportunity for caching if we can be sure the value is set.
I know from our previous discussions of caching that you had two big
worries (that I share with you by the way).
1. Caching doesn't work well in some cases where user code may want to
change the values being cached (runtime modification)
2. It makes the code ugly putting all these caches everywhere.
That got me thinking about the python memoization pattern that can be
handled by decorators. See:
http://www.phyast.pitt.edu/~micheles/python/documentation.html#memoize and
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325205
What I was thinking is that in the context of py++ it would be nice if
the user could make whatever changes they want to the system and then
once the state is "set", call a method that turns on caching from that
point forward.
Something like:
------------------------------
mb = ModuleBuilder(....)
<do user changes>
pyplusplus.memoizer.enable() # Everything we cache is set from here on
------------------------------
We could also introduce an enabled state that asserted that the cache
value equals the value that would have been calculated. This would
give a nice sanity check to use during debugging to make sure the cache
is not breaking things.
Anyway..... with a memoization pattern then the py++ internal code could
look something like this:
class declaration_t:
....
@memoize
def getName(...):
return value as normal.
So you see I think we could address both issues aboved. The code would
allow for runtime modification by users until they say they are done.
It would also allow for clean code because you could just put a
decorator (@memoize) on methods that needed it and then they would
automatically work.
What do you think? Is this a way we could increase the performance but
still keep the code clean and flexible?
-Allen
|
|
From: Allen B. <al...@vr...> - 2007-02-16 15:27:50
|
Roman Yakovenko wrote:
> On 2/15/07, Allen Bierbaum <al...@vr...
> <mailto:al...@vr...>> wrote:
> > Have there been any recent commits (past 1-2 months) that may have had a
> > significant affect on performance in py++?
> >
> > I just update to the latest svn version and my code generation process
> > is back up around 15 minutes. Is there anything I should look for as a
> > cause?
>
> Yes, dependency manager. It checks that you expose all declarations
> that are used in your interface.
>
Is there any user API to disable the dependency manager (atleast during
debugging and development)?
What is the basic structure of the dependency management search
algorithm? Is it something like:
- For every part of an interface exposed
- For each item used in the interface
- Look up the item and make sure that is exposed
If it is something like this, then that could definitely be biting me.
I have a *huge* number of symbols and something O(N^2) like this could
really consume a lot of time.
In the meantime, I am trying to collect some profiler information to
find out exactly where the time is being spent. Once I have those
numbers collected I will pass them along in case there is anything that
can be done to increase performance.
-Allen
|
|
From: Allen B. <al...@vr...> - 2007-02-16 15:01:40
|
Roman Yakovenko wrote: > On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > >> Roman: >> >> The new warning support is very nice. I like it a lot and I think it >> will help me quite a bit. One suggestion though, when there is a >> warning related to a method or attribute of a class, could you output >> the name of that relevant declaration. For example when I get a warning >> about exposing a non-public member function it would be very helpful to >> know the name of the member that I exposed that was not public. >> >> What do you think about adding the member.name to all the warnings >> output by decl-wrappers.class_wrapper.class_t.is_wrapper_needed? Is >> there any reason not to support this type of thing? >> > > There is no reason, I will add this information. Can you devote few minutes > of your time and to fix the messages? There were few users who didn't get right > what the message tried to say them. > > ( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/messages/warnings_.py?view=markup > ) > > I may be one of those users. :) As I encounter the messages, if I find one that is confusing or unclear I will do my best to change it to make it better. Thanks, Allen > Thank you > > |
|
From: Roman Y. <rom...@gm...> - 2007-02-16 05:44:44
|
On 2/16/07, Allen Bierbaum <al...@vr...> wrote: > Roman: > > The new warning support is very nice. I like it a lot and I think it > will help me quite a bit. One suggestion though, when there is a > warning related to a method or attribute of a class, could you output > the name of that relevant declaration. For example when I get a warning > about exposing a non-public member function it would be very helpful to > know the name of the member that I exposed that was not public. > > What do you think about adding the member.name to all the warnings > output by decl-wrappers.class_wrapper.class_t.is_wrapper_needed? Is > there any reason not to support this type of thing? There is no reason, I will add this information. Can you devote few minutes of your time and to fix the messages? There were few users who didn't get right what the message tried to say them. ( http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/pyplusplus/messages/warnings_.py?view=markup ) Thank you -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Roman Y. <rom...@gm...> - 2007-02-16 05:37:56
|
On 2/15/07, Allen Bierbaum <al...@vr...> wrote: > Have there been any recent commits (past 1-2 months) that may have had a > significant affect on performance in py++? > > I just update to the latest svn version and my code generation process > is back up around 15 minutes. Is there anything I should look for as a > cause? Yes, dependency manager. It checks that you expose all declarations that are used in your interface. -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ |
|
From: Allen B. <al...@vr...> - 2007-02-16 00:14:47
|
Roman: The new warning support is very nice. I like it a lot and I think it will help me quite a bit. One suggestion though, when there is a warning related to a method or attribute of a class, could you output the name of that relevant declaration. For example when I get a warning about exposing a non-public member function it would be very helpful to know the name of the member that I exposed that was not public. What do you think about adding the member.name to all the warnings output by decl-wrappers.class_wrapper.class_t.is_wrapper_needed? Is there any reason not to support this type of thing? Thanks, Allen |
23 messages has been excluded from this view by a project administrator.
