Revision: 1218
http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1218&view=rev
Author: roman_yakovenko
Date: 2008-01-05 12:47:42 -0800 (Sat, 05 Jan 2008)
Log Message:
-----------
improving docs
Added Paths:
-----------
pyplusplus_dev/docs/documentation/how_to/
pyplusplus_dev/docs/documentation/how_to/best_practices.rest
pyplusplus_dev/docs/documentation/how_to/exception_translation.rest
pyplusplus_dev/docs/documentation/how_to/hints.rest
pyplusplus_dev/docs/documentation/how_to/how_to.rest
pyplusplus_dev/docs/documentation/how_to/templates.rest
pyplusplus_dev/docs/documentation/how_to/www_configuration.py
Added: pyplusplus_dev/docs/documentation/how_to/best_practices.rest
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/best_practices.rest (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/best_practices.rest 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,176 @@
+==============
+Best practices
+==============
+
+.. contents:: Table of contents
+
+------------
+Introduction
+------------
+
+`Py++`_ has reach interface and a lot of functionality. Sometimes reach
+interface helps, but sometimes it can confuse. This document will describe how
+effectively to use `Py++`_.
+
+------------
+Big projects
+------------
+
+Definition
+----------
+
+First of all, let me to define "big project". "Big project" is a project with
+few hundred of header files. `Py++`_ was born to create `Python`_ bindings
+for such projects. If you take a look `here`__ you will find few such projects.
+
+.. __ : ./../../pyplusplus/quotes.html
+
+Tips
+----
+
+* Create one header file, which will include all project header files.
+
+ Doing it this way makes it so `GCC-XML`_ is only called once and it reduces the
+ overhead that would occur if you pass `GCC-XML`_ all the files individually.
+ Namely `GCC-XML`_ would have to run hundreds of times and each call would
+ actually end up including quite a bit of common code anyway. This way takes a
+ `GCC-XML`_ processing time from multiple hours with gigabytes of caches to a
+ couple minutes with a reasonable cache size.
+
+ You can read more about different caches supported by `pygccxml`_ `here`__.
+ ``module_builder_t.__init__`` method takes reference to an instance of cache
+ class or ``None``:
+
+ .. code-block:: Python
+
+ from module_builder import *
+ mb = module_builder_t( ..., cache=file_cache_t( path to project cache file ), ... )
+
+* Single header file, will also improve performance compiling the generated bindings.
+
+ When `Py++`_ generated the bindings, you have a lot of .cpp files to
+ compile. The project you are working on is big. I am sure it takes a lot of
+ time to compile projects that depend on it. Generated code also depend on it,
+ more over this code contains a lot of template instantiations. So it could
+ take a great deal of time to compile it. Allen Bierbaum investigated this
+ problem. He found out that most of the time is really spent processing all the
+ headers, templates, macros from the project and from the boost library. So he
+ come to conclusion, that in order to improve compilation speed, user should
+ be able to control( to be able to generate ) precompiled header file. He
+ implemented an initial version of the functionality. After small discussion,
+ we agreed on next interface:
+
+ .. code-block:: Python
+
+ class module_builder_t( ... ):
+ ...
+ def split_module( self, directory_path, huge_classes=None, precompiled_header=None ):
+ ...
+
+ ``precompiled_header`` argument could be ``None`` or string, that contains
+ name of precompiled header file, which will be created in the directory.
+ `Py++`_ will add to it header files from `Boost.Python`_ library and
+ your header files.
+
+ What is ``huge_classes`` argument for? ``huge_classes`` could be ``None`` or
+ list of references to class declarations. It is there to provide a solution to
+ `this error`_. `Py++`_ will automatically split generated code for the
+ huge classes to few files:
+
+ .. code-block:: Python
+
+ mb = module_builder_t( ... )
+ ...
+ my_big_class = mb.class_( my_big_class )
+ mb.split_module( ..., huge_classes=[my_big_class], ... )
+
+ * **Caveats**
+
+ Consider next file layout:
+ ::
+
+ boost/
+ date_time/
+ ptime.hpp
+ time_duration.hpp
+ date_time.hpp //main header, which include all other header files
+
+ Py++ currently does not handle relative paths as input very well, so it is
+ recommended that you use "os.path.abspath()" to transform the header file to
+ be processed into an absolute path:
+
+ .. code-block:: Python
+
+ #Next code will expose nothing
+ mb = module_builder( [ 'date_time/date_time.hpp' ], ... )
+
+ #while this one will work as expected
+ import os
+ mb = module_builder( [ os.path.abspath('date_time/date_time.hpp') ], ... )
+
+.. _`this error` : http://boost.org/libs/python/doc/v2/faq.html#c1204
+.. __ : ./../../pygccxml/design.html
+
+* Keep the declaration tree small.
+
+ When parsing the header files to build the declaration tree, there will also
+ be the occasional "junk" declaration inside the tree that is not relevant to
+ the bindings you want to generate. These extra declarations come from header
+ files that were included somewhere in the header files that you were actually
+ parsing (e.g. if that library uses the STL or OpenGL or other system headers
+ then the final declaration tree will contain those declarations, too).
+ It can happen that the majority of declarations in your declaration tree are
+ such "junk" declarations that are not required for generating your bindings
+ and that just slow down the generation process (reading the declaration cache
+ and doing queries will take longer).
+
+ To speed up your generation process you might want to consider making the
+ declaration tree as small as possible and only store those declarations that
+ somehow have an influence on the bindings. Ideally, this is done as early
+ as possible and luckily gccxml provides an option that allows you to reduce
+ the number of declarations that it will store in the output XML file. You can
+ specify one or more declarations using the ``-fxml-start`` option and only
+ those sub-tree starting at the specified declarations will be written. For
+ example, if you specify the name of a particular class, only this class
+ and all its members will get written. Or if your project already uses
+ a dedicated namespace you can simply use this namespace as a starting point
+ and all declarations stemming from system headers will be ignored (except
+ for those declarations that are actually used within your library).
+
+ In the ``pygccxml`` package you can set the value for the ``-fxml-start``
+ option using the ``start_with_declarations`` attribute of the
+ ``pygccxml.parser.config_t`` object that you are passing to the parser.
+
+* Use `Py++`_ repository of generated files md5 sum.
+
+ `Py++`_ is able to store md5 sum of generated files in a file. Next time you
+ will generate code, `Py++`_ will compare generated file content against the sum,
+ instead of loading the content of the previously generated file from the disk
+ and comparing against it.
+
+ .. code-block:: Python
+
+ mb = module_builder_t( ... )
+ ...
+ my_big_class = mb.class_( my_big_class )
+ mb.split_module( ..., use_files_sum_repository=True )
+
+ `Py++`_ will generate file named "<your module name>.md5.sum" in the directory
+ it will generate all the files.
+
+ Enabling this functionality should give you 10-15% of performance boost.
+
+ * **Caveats**
+
+ If you changed manually some of the files - don't forget to delete the relevant
+ line from "md5.sum" file. You can also delete the whole file. If the file is
+ missing, `Py++`_ will use old plain method of comparing content of the files.
+ It will not re-write "unchanged" files and you will not be forced to recompile
+ the whole project.
+
+
+.. _`Py++` : ./../pyplusplus.html
+.. _`pygccxml` : ./../../pygccxml/pygccxml.html
+.. _`Boost.Python`: http://www.boost.org/libs/python/doc/index.html
+.. _`Python`: http://www.python.org
+.. _`GCC-XML`: http://www.gccxml.org
Added: pyplusplus_dev/docs/documentation/how_to/exception_translation.rest
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/exception_translation.rest (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/exception_translation.rest 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,65 @@
+=========================================
+How to register an exception translation?
+=========================================
+
+.. contents:: Table of contents
+
+------------
+Introduction
+------------
+
+Boost.Python provides functionality to translate any C++ exception to a Python one.
+`Py++`_ provides a convenient API to do this.
+
+By the way, be sure to take a look on "`troubleshooting guide - exceptions`_".
+The guide will introduces a complete solution for handling exceptions within
+Python scripts.
+
+.. _`troubleshooting guide - exceptions` : ./../../troubleshooting_guide/exceptions/exceptions.html
+
+--------
+Solution
+--------
+
+`Boost.Python exception translator documentation`_ contains a complete explanation
+what should be done. I will use that example, to show how it could be done with
+`Py++`_:
+
+.. code-block:: Python
+
+ from pyplusplus import module_builder_t
+
+ mb = module_builder_t( ... )
+ my_exception = mb.class_( 'my_exception' )
+
+ translate_code = 'PyErr_SetString(PyExc_RuntimeError, exc.what();'
+ my_exception.exception_translation_code = translate_code
+
+That's all, really. `Py++`_ will generate for you the ``translate`` function
+definition and than will register it.
+
+I think this is a most popular use case - translate a C++ exception to a string
+and than to create an instance of Python built-in exception. That is exactly why
+`Py++`_ provides additional API:
+
+.. code-block:: Python
+
+ mb = module_builder_t( ... )
+ my_exception = mb.class_( 'my_exception' )
+
+ my_exception.translate_exception_to_string( 'PyExc_RuntimeError', 'exc.what()')
+
+The first argument of ``translate_exception_to_string`` method is exception type,
+The second one is a string - code that converts your exception to ``const char*``.
+
+As you see, it is really simple to add exception translation to your project.
+
+One more point, in both pieces of code I used "``exc``" as the name of ``my_exception``
+class instance. This is a predefined name. I am not going to change it without
+any good reason, any time soon :-).
+
+.. _`Boost.Python exception translator documentation` : http://boost.org/libs/python/doc/v2/exception_translator.html
+.. _`Py++` : ./../../pyplusplus.html
+.. _`Boost.Python`: http://www.boost.org/libs/python/doc/index.html
+.. _`Python`: http://www.python.org
+.. _`GCC-XML`: http://www.gccxml.org
Added: pyplusplus_dev/docs/documentation/how_to/hints.rest
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/hints.rest (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/hints.rest 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,64 @@
+=====
+Hints
+=====
+
+.. contents:: Table of contents
+
+----------------------------------
+Class template instantiation alias
+----------------------------------
+
+`Py++`_ has nice feature. If you define ``typedef`` for instantiated class
+template, than `Py++`_ will use it as a `Python`_ class name.
+
+For example:
+
+.. code-block:: C++
+
+ #include <vector>
+ typedef std::vector< int > numbers;
+ numbers generate_n(){
+ ...
+ }
+
+`Py++`_ will use "numbers" as Python class name:
+
+.. code-block:: C++
+
+ using boost::python;
+ class_< std::vector< int > >( "numbers" )
+ ...
+ ;
+
+`Py++`_ will pick up the alias, only in case the class has single "typedef".
+
+``pyplusplus::aliases`` namespace
+---------------------------------
+
+The previous approach is "implicit" - `Py++`_ does something behind the scene.
+Recently (version 0.8.6 ), another approach was introduced:
+
+.. code-block:: C++
+
+ #include <vector>
+
+ namespace pyplusplus{ namespace aliases{
+ //^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ typedef std::vector< int > numbers;
+
+ } } //pyplusplus::aliases
+
+The idea is that you create namespace with a special name - ``pyplusplus::aliases``
+and `Py++`_ automatically picks the class aliases from it. In case you accidentally
+introduced two or more different aliases to the same class, it will pick the
+longest one and print a warning. Other advantages of the approach:
+
+* you are not forced to learn new API
+
+* you continue to use your favorite editor and familiar language
+
+.. _`Py++` : ./../pyplusplus.html
+.. _`Boost.Python`: http://www.boost.org/libs/python/doc/index.html
+.. _`Python`: http://www.python.org
+.. _`GCC-XML`: http://www.gccxml.org
+
Added: pyplusplus_dev/docs/documentation/how_to/how_to.rest
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/how_to.rest (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/how_to.rest 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,175 @@
+============
+How to ... ?
+============
+
+.. contents:: Table of contents
+
+`How to deal with templates?`_
+
+.. _`How to deal with templates?` : ./templates.html
+
+`How to add custom exception translation?`_
+
+.. _ `How to add custom exception translation?` : custom_exception_translation.html
+
+-------------------------------------------------------
+How to expose function, which has hand-written wrapper?
+-------------------------------------------------------
+.. code-block:: C++
+
+ struct window_t{
+ ...
+ void get_size( int& height, int& widht ) const;
+ };
+
+You can not expose ``get_size`` function as is - ``int`` is immutable type in
+Python. So, we need to create a wrapper to the function:
+
+.. code-block:: C++
+
+ boost::python::tuple get_size_wrapper( const window_t& win ){
+ int height(0), width( 0 );
+ win.get_size( height, widht );
+ return boost::python::make_tuple( height, width );
+ }
+
+.. code-block:: C++
+
+ class_<window_t>( ... )
+ .def( "get_size", &get_size_wrapper )
+ ...
+ ;
+
+Now, after you know how this problem is solved. I will show how this solution
+could be integrated with `Py++`_.
+
+.. code-block:: Python
+
+ wrapper_code = \
+ """
+ static boost::python::tuple get_size( const window_t& win ){
+ int height(0), width( 0 );
+ win.get_size( height, width );
+ return boost::python::make_tuple( height, width );
+ }
+ """
+
+.. code-block:: Python
+
+ registration_code = 'def( "get_size", &%s::get_size )' % window.wrapper_alias
+
+.. code-block:: Python
+
+ mb = module_builder_t( ... )
+ window = mb.class_( "window_t" )
+ window.member_function( "get_size" ).exclude()
+ window.add_wrapper_code( wrapper_code )
+ window.registration_code( registration_code )
+
+That's all.
+
+-------------------------------------------------------------
+Fatal error C1204:Compiler limit: internal structure overflow
+-------------------------------------------------------------
+
+If you get this error, than the generated file is too big. You will have to split
+it to few files. Well, not you but `Py++`_, you will only have to tell it to do
+that.
+
+If you are using ``module_builder_t.write_module`` method, consider to switch
+to ``module_builder_t.split_module``.
+
+If you are using ``split_module``, but still the generated code for some class
+could not be compiled, because of the error, you can ask `Py++`_ to split the
+code generated for class to be split to few cpp files.
+
+For more information, please read the documentation.
+
+
+-------------------------------------
+Py++ generated file name is too long!
+-------------------------------------
+There are use cases, when `Py++`_ generated file name is too long. In some cases
+the code generation process even fails because of this. What can you do in order
+to eliminate the problem?
+
+First of all the problem arises when you expose template instantiated classes
+and you did not set the class alias.
+
+Let me explain:
+
+.. code-block:: C++
+
+ template < class T>
+ struct holder{ ... };
+
+
+Lets say that you want to export ``holder< int >`` class. Class name in `Python`_
+has few `constraints`_. `Py++`_ is aware of the `constraints`_ and if you didn't
+set an alias to the class, `Py++`_ will do it for you. In this case,
+"holder_less_int_grate\_" is the generated alias. Obviously it is much longer
+and not readable.
+
+.. _`constraints` : http://www.python.org/doc/current/ref/identifiers.html
+
+There are few pretty good reasons for this behavior:
+
+* when you just start to work on `Python`_ bindings concentrate your attention
+ on really important things
+
+* if you forgot to set the class alias your users still can use the class
+ functionality, however the class name will be a little bit ugly.
+
+
+In this case the generate alias for ``holder`` instantiation is relatively short.
+Imagine how long it could be for ``std::map`` instantiation.
+
+`Py++`_ uses class alias for the file name. So if you want to force `Py++`_ to
+generate files with short name, you have to set class alias:
+
+.. code-block:: Python
+
+ from pyplusplus import module_builder
+
+ mb = module_builder_t( ... )
+ holder = mb.class_( 'holder< int >' )
+ holder.alias = 'IntHolder'
+ #next line has same effect as the previous one:
+ holder.rename( 'IntHolder' )
+
+The nice thing about this approach is that now `Python`_ users have "normal"
+class name and you have short file name.
+
+-------------------
+Full\relative paths
+-------------------
+
+Consider next file layout:
+::
+
+ boost/
+ date_time/
+ ptime.hpp
+ time_duration.hpp
+ date_time.hpp //main header, which include all other header files
+
+Py++ currently does not handle relative paths as input very well, so it is
+recommended that you use ``os.path.abspath()`` to transform the header file to
+be processed into an absolute path:
+
+.. code-block:: Python
+
+ #Next code will expose nothing
+ mb = module_builder( [ 'date_time/date_time.hpp' ], ... )
+ mb.split_module( ... )
+
+ #while this one will work as expected
+ import os
+ mb = module_builder( [ os.path.abspath('date_time/date_time.hpp') ], ... )
+ mb.split_module( ... )
+
+
+.. _`Py++` : ./../pyplusplus.html
+.. _`Boost.Python`: http://www.boost.org/libs/python/doc/index.html
+.. _`Python`: http://www.python.org
+.. _`GCC-XML`: http://www.gccxml.org
Added: pyplusplus_dev/docs/documentation/how_to/templates.rest
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/templates.rest (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/templates.rest 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,196 @@
+===========================
+How to deal with templates?
+===========================
+
+.. contents:: Table of contents
+
+------------
+Introduction
+------------
+
+I would like to introduce next piece of code I will use for most exlanations.
+
+.. code-block:: C++
+
+ // file point.h
+ template< class T>
+ struct point_t{
+ T x, y;
+ };
+
+ template <class T>
+ double distance( const point_t<T>& point ){
+ return sqrt( point.x * point.x + point.y*point.y );
+ }
+
+ struct environment_t{
+ ...
+ template< class T>
+ T get_value(const std::string& name);
+ ...
+ };
+
+----------------------
+Template instantiation
+----------------------
+
+First of all you should understand, that you can not export template itself, but
+only its instantiations.
+
+You can instantiate template class using operator ``sizeof``:
+
+.. code-block:: C++
+
+ sizeof( point_t<int> );
+
+
+In order to instantiate a function you have to call it:
+
+.. code-block:: C++
+
+ void instantiate(){
+ double x = distance( point_t<t>() );
+
+ environment_t env;
+ std::string path = env.get_value< std::string >( "PATH" );
+ int version = env.get_value< int >( "VERSION" );
+ }
+
+You should put that code in some header file, parsed by GCC-XML.
+
+"Dynamic" instantiation
+-----------------------
+If you have a template class, which should be instantiated with many types, you
+can create a small code generator, which will "instantiate the class". It is
+pretty easy to blend together the generated code and the existing one:
+
+.. code-block:: Python
+
+ from module_builder import module_builder_t, create_text_fc
+
+ def generate_instantiations_string( ... ):
+ ...
+
+ code = generate_instantiations_string( ... )
+
+ mb = module_builder_t( [ create_text_fc( code ), other header files ], ... )
+ ...
+
+Function ``create_text_fc`` allows you to extract declarations from the string,
+which contains valid C++ code. It creates temporal header file and compiles it.
+
+.. __ : ./../../../pygccxml/design.html#parser-configuration-classes
+
+----------------------------------
+Functions templated on return type
+----------------------------------
+
+.. code-block:: C++
+
+ environment_t env;
+ std::string path = env.get_value< std::string >( "PATH" );
+ int version = env.get_value< int >( "VERSION" );
+
+
+`GCC-XML`_ provides information for both instantiations:
+
+ * ``get_value<int>``
+
+ * ``get_value< std::string >``
+
+But, in this case there is a catch: the name of both functions is "**get_value**".
+The only difference is "return type".
+
+In this situation, `Py++`_ will generate code that contains errors. If your are
+lucky, it depends on the compiler you use, the generated code will not compile.
+Otherwise, you will discover the errors while testing the bindings.
+
+Generated code:
+
+.. code-block:: C++
+
+ bp::class_< environment_t >( "environment_t" )
+ ...
+ .def( "get_value"
+ , (int ( ::environment_t::* )( ::std::string const & ) )( &::environment_t::get_value ) )
+ .def( "get_value"
+ , (::std::string ( ::environment_t::* )( ::std::string const & ) )( &::environment_t::get_value ) );
+
+The correct code:
+
+.. code-block:: C++
+
+ bp::class_< environment_t >( "environment_t" )
+ .def( "get_value"
+ , (int ( ::environment_t::* )( ::std::string const & ) )( &::environment_t::get_value< int > ) )
+ //--------------------------------------------------------------------------^^^^^^^^^^^^^^^^
+ .def( "get_value"
+ , (::std::string ( ::environment_t::* )( ::std::string const & ) )( &::environment_t::get_value< std::string > ) );
+ //------------------------------------------------------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^
+
+The perfect one:
+
+.. code-block:: C++
+
+ bp::class_< environment_t >( "environment_t" )
+ ...
+ .def( "get_value", &::environment_t::get_value< int > )
+ .def( "get_value", &::environment_t::get_value< std::string > );
+
+
+Work-around
+-----------
+
+`Py++`_ contains a work-around to the problem:
+
+.. code-block:: Python
+
+ mb = module_builder_t( ..., optimize_queries=False, ... )
+ environment = mb.class_( "environment_t" )
+ for f in environment.member_functions( "get_value" ):
+ #set the function alias
+ f.alias = f.name + "_" + f.return_type.decl_string
+ #correct function name
+ f.name = f.demangled_name
+ #you still want the queries to run fast
+ mb.run_query_optimizer()
+
+Before you read the rest of the solution, you should understand what is
+"name mangling" means. If you don't, consider reading about it on `Wikipedia`__ .
+
+.. __ : http://en.wikipedia.org/wiki/Name_mangling
+
+The solution is pretty simple. `GCC-XML`_ reports mangled and demangled function
+names. The demangled function name contains "real" function name:
+``get_value< used type >``. You only have to instruct `Py++`_ to use it.
+
+`Py++`_ does not use by default demangled function name for mainly one reason.
+Demangled function name is a string that contains a lot of information. `Py++`_
+implements a parser, which extracts the only relevant one. The parser
+implementation is a little bit complex and was not heavily tested. By "heavily" I
+mean that I tested it on a lot of crazy use cases and on a real project, but
+there is always some new use case out there. I am almost sure it will work for
+you. The problem, we deal with, is rare, so by default "demangled_name"
+feature is turned off.
+
+By the way, almost the same problem exists for template classes. But, in the
+classes use case `Py++`_ uses demangled name by default.
+
+-----------
+Help wanted
+-----------
+I understand that the provided solutions are not perfect and that something
+better and simpler should be done. Unfortunatelly the priority of this task is
+low.
+
+Allen Bierbaum has few suggestion that could improve `Py++`_. He created a
+`wiki page`_, that discuss possible solutions. Your contribution is welcome too!
+
+.. _`wiki page` : https://realityforge.vrsource.org/view/PyppApi/TemplateSupport
+
+
+
+.. _`Py++` : ./../pyplusplus.html
+.. _`Boost.Python`: http://www.boost.org/libs/python/doc/index.html
+.. _`Python`: http://www.python.org
+.. _`GCC-XML`: http://www.gccxml.org
Added: pyplusplus_dev/docs/documentation/how_to/www_configuration.py
===================================================================
--- pyplusplus_dev/docs/documentation/how_to/www_configuration.py (rev 0)
+++ pyplusplus_dev/docs/documentation/how_to/www_configuration.py 2008-01-05 20:47:42 UTC (rev 1218)
@@ -0,0 +1,10 @@
+name = 'how to ... ?'
+#main_html_file = 'index.html'
+
+names = { 'hints' : 'hints'
+ , 'how_to' : 'how to'
+ , 'templates' : 'deal with templates'
+ , 'best_practices' : 'best practices'
+ , 'exception_translation' : 'exception translation'
+}
+
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|