pygccxml-development

[pygccxml-development] [FT] not exception safe generated code

[Roman Y. <rom...@gm...>]

Hi Matthias. I did quick review to part of the code and this is what I found:
mem_fun_v_transformed_wrapper_t.create_virtual_body method:

PyGILState_STATE gstate;

gstate = PyGILState_Ensure();



$DECLARATIONS



PyGILState_Release(gstate);


Obviously the code you generate here is not exception safe. I think
you should use
RIIA principle.

class gil_lock_t{
public:
    gil_lock_t( bool lock=true )
    : m_locked( lock )
    {
        if( m_locked )
            m_gstate = PyGILState_Ensure();
    }

    ~gil_lock_t(){
        if( m_locked )
            PyGILState_Release(m_gstate);
    }

     void lock(){
         if( !m_locked )
             m_gstate = PyGILState_Ensure();
     }

     void unlock(){
         if( m_locked )
             PyGILState_Release(m_gstate);
     }
private:
    bool m_locked;
    PyGILState_STATE m_gstate;
};

With this class you can change the code in create_virtual_body method.
Py++ has all functionality needed to add this code Take a look on
code_repository package.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] [FT] not exception safe generated code

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> Obviously the code you generate here is not exception safe. I think

You're right. But that code doesn't get generated anyway unless you 
explicitly enable it in the source code. I'm using this just to test 
whether it will work for the Maya bindings at all (I actually just got 
it working today :)
I agree that the final implementation should be more careful and using 
your proposed class for this looks like it might be a good choice.

- Matthias -

[pygccxml-development] code_repository package

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> With this class you can change the code in create_virtual_body method.
> Py++ has all functionality needed to add this code Take a look on
> code_repository package.

I've seen that those files get written in 
writer_t.write_code_repository(). But what does this line do:

      if self.__extmodule.is_system_header( cr.file_name ):

What exactly is a "system header" in this context?

And a second question: Where is the 'namespace' variable used?

- Matthias -

Re: [pygccxml-development] code_repository package

[Roman Y. <rom...@gm...>]

On 9/19/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> > With this class you can change the code in create_virtual_body method.
> > Py++ has all functionality needed to add this code Take a look on
> > code_repository package.
>
> I've seen that those files get written in
> writer_t.write_code_repository(). But what does this line do:
>
>       if self.__extmodule.is_system_header( cr.file_name ):
>
> What exactly is a "system header" in this context?

In whole context "system header" defined as header file that belongs to
Boost.Python or to Py++. When I build code creators tree, in creator_t, I add
"include" directives to the module_t code creator. In all cases I know
whether they are
system headers or

> And a second question: Where is the 'namespace' variable used?

When you use the functionality defined in the repository.

Take a look on array_1_registrator_t code creator and its construction
in creator_t class.

The namespace solves few problems:
1. pollution of global namespace
2. prevents Py++ functionality to conflict with user defined one
3. to say explicitly to the user that this code has been written by
Py++ developers and not
   by Boost.Python ( be careful :-) )


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] code_repository package

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> I've seen that those files get written in
>> writer_t.write_code_repository(). But what does this line do:
>>
>>       if self.__extmodule.is_system_header( cr.file_name ):
>>
>> What exactly is a "system header" in this context?
> 
> In whole context "system header" defined as header file that belongs to
> Boost.Python or to Py++. 

Aren't then all files from the code_repository package "system headers"?
How do I influence whether a file is considered a system header or not? 
I tried adding a new file to the code_repository package, but it doesn't 
get written.

- Matthias -

Re: [pygccxml-development] code_repository package

[Roman Y. <rom...@gm...>]

On 9/19/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> >> I've seen that those files get written in
> >> writer_t.write_code_repository(). But what does this line do:
> >>
> >>       if self.__extmodule.is_system_header( cr.file_name ):
> >>
> >> What exactly is a "system header" in this context?
> >
> > In whole context "system header" defined as header file that belongs to
> > Boost.Python or to Py++.
>
> Aren't then all files from the code_repository package "system headers"?

Yes they are

> How do I influence whether a file is considered a system header or not?

In creator_t, you add your header file using add_system_header API

> I tried adding a new file to the code_repository package, but it doesn't
> get written.

1. May be you did not add it as system header in creator_t class
2. May be you did not modified code_repository.__init__.py file

You can commit your code and I will take a look on it this evening.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] code_repository package

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> How do I influence whether a file is considered a system header or not?
> 
> In creator_t, you add your header file using add_system_header API

I suppose the header should only be created when it is required, right?
I want to use it for the class that can acquire/release the Python GIL 
(as you have suggested earlier).
So the actual decision whether the header is required or not is done by 
the code creator (mem_fun_v_transformed_wrapper_t). What is your 
recommendation how the code in creator_t (this is 
visit_member_function(), right?) will know about this decision?

Just to see if the file gets written or not I called add_system_header() 
and adopt_creator() on every call to visit_member_function(), but then I 
get a huge list of identical include statements in the generated files.
What I would actually like to do is to add the "include" code creator to 
the associated_decl_creators list of the corresponding class so that the 
include only shows up in the file where it is needed. But how can I 
check whether the file was already included or not?

I wonder why there is no specific API for managing include files that 
handles all those stuff? It appears to me that this would be quite a 
useful thing to have.

- Matthias -

Re: [pygccxml-development] code_repository package

[Roman Y. <rom...@gm...>]

On 9/19/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> >> How do I influence whether a file is considered a system header or not?
> >
> > In creator_t, you add your header file using add_system_header API
>
> I suppose the header should only be created when it is required, right?

file_writers package will do that for you.

> I want to use it for the class that can acquire/release the Python GIL
> (as you have suggested earlier).
> So the actual decision whether the header is required or not is done by
> the code creator (mem_fun_v_transformed_wrapper_t). What is your
> recommendation how the code in creator_t (this is
> visit_member_function(), right?) will know about this decision?

It can search for "thread safe" transformer. The better idea to
introduce next class:

class ftransformer_t:
    keeps a list of all transformations that should be applied on the
function at once

    provides some additional functionality:
        function signature
        has_thread_safe_transformer
        ...

Than
1. Creator will use it to ask the right question
2. It will be possible to have more than 1 "f" transformation applied
on the same
    function: for example "status as exception"

    struct status{};
    void do_smth( status& )

    In some cases it makes sence to expose "do_smth" function as is and to
    expose some convinience function that will raise an exception.


> Just to see if the file gets written or not I called add_system_header()
> and adopt_creator() on every call to visit_member_function(), but then I
> get a huge list of identical include statements in the generated files.

I don't fully understand what you are doing here.

> What I would actually like to do is to add the "include" code creator to
> the associated_decl_creators list of the corresponding class so that the
> include only shows up in the file where it is needed. But how can I
> check whether the file was already included or not?

If we talk about system header than you can ask module_t code creator
( creator_t.__extmodule ) is_system_header. You can also "include"
the header from "_create_includes" method.

> I wonder why there is no specific API for managing include files that
> handles all those stuff? It appears to me that this would be quite a
> useful thing to have.

Lack of time and low priority. Users learned to leave with "non
optimal" generated
code. I mean that all include appears in all header files. But they don't like
Py++, when it does not support the functionality they needed.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] code_repository package

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>>>> How do I influence whether a file is considered a system header or not?
>>> In creator_t, you add your header file using add_system_header API
>> I suppose the header should only be created when it is required, right?
> 
> file_writers package will do that for you.

How does it do that?

>> I want to use it for the class that can acquire/release the Python GIL
>> (as you have suggested earlier).
>> So the actual decision whether the header is required or not is done by
>> the code creator (mem_fun_v_transformed_wrapper_t). What is your
>> recommendation how the code in creator_t (this is
>> visit_member_function(), right?) will know about this decision?
> 
> It can search for "thread safe" transformer. The better idea to
> introduce next class:
> 
> class ftransformer_t:
>     keeps a list of all transformations that should be applied on the
> function at once
> 
>     provides some additional functionality:
>         function signature
>         has_thread_safe_transformer
>         ...
> 
> Than
> 1. Creator will use it to ask the right question
> 2. It will be possible to have more than 1 "f" transformation applied
> on the same
>     function: for example "status as exception"
> 
>     struct status{};
>     void do_smth( status& )
> 
>     In some cases it makes sence to expose "do_smth" function as is and to
>     expose some convinience function that will raise an exception.

I'm sorry, obviously I got misunderstood as this doesn't answer my
question. My question was actually related to the code repository
package and its usage, the function transformers and thread safety stuff 
was just an example. Sorry for any confusion. Maybe I should rephrase my
question:

The responsibility of a code creator object is to create a piece of C++ 
source code that will be written into one or more output files, right? 
Now my actual question probably boils down to the following question: 
Whose responsibility is it to make sure that the code that was generated 
by a particular code creator object can also be compiled without errors?

>> Just to see if the file gets written or not I called add_system_header()
>> and adopt_creator() on every call to visit_member_function(), but then I
>> get a huge list of identical include statements in the generated files.
> 
> I don't fully understand what you are doing here.

I was just doing what you were suggesting earlier, i.e. to call
add_system_header() to have Py++ write a particular header from the code
repository package. This did work but the file was nowhere included, so
I added an "include" code creator using adopt_creator() on the root node
(I was peeking at the array_1 stuff to see how it is done). But as I was
doing that in every call to visit_member_function() the file got
included again each time resulting in a lot of duplicate lines.

>> What I would actually like to do is to add the "include" code creator to
>> the associated_decl_creators list of the corresponding class so that the
>> include only shows up in the file where it is needed. But how can I
>> check whether the file was already included or not?
> 
> If we talk about system header than you can ask module_t code creator
> ( creator_t.__extmodule ) is_system_header. 

ok, this is what I'm doing now and which does work for now. But this 
makes the include file a "project wide" include file. How should I do 
this check if I would only like to add the include code creator to the 
associated_decl_creators list? (so that it only gets included where it 
is required?)

>> I wonder why there is no specific API for managing include files that
>> handles all those stuff? It appears to me that this would be quite a
>> useful thing to have.
> 
> Lack of time and low priority. Users learned to leave with "non
> optimal" generated
> code. I mean that all include appears in all header files. But they don't like
> Py++, when it does not support the functionality they needed.

That's not what I meant. I mean, even when the current functionality is 
exactly preserved, I would rather like to have an (internal) API that 
handles the details of how an include file is added to the output files. 
This is required at several places and it's not easy to understand how 
it is properly done. An additional benefit would be that it would pave 
the way for improving the handling of include files so that they could 
be set on a file-by-file basis.

- Matthias -

Re: [pygccxml-development] code_repository package

[Roman Y. <rom...@gm...>]

On 9/20/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> >>>> How do I influence whether a file is considered a system header or not?
> >>> In creator_t, you add your header file using add_system_header API
> >> I suppose the header should only be created when it is required, right?
> >
> > file_writers package will do that for you.
>
> How does it do that?

writer_t class defines write_code_repository method. It is a respnocibility of
derived file writers to call this function.

>
> The responsibility of a code creator object is to create a piece of C++
> source code that will be written into one or more output files, right?
> Now my actual question probably boils down to the following question:
> Whose responsibility is it to make sure that the code that was generated
> by a particular code creator object can also be compiled without errors?

File writer + code creator. File writer answer the question: where to put the
code, while code creator "what code to put". creator_t defines
"logical" place of the code creator, by placing it in the right place
within the tree. File writers insure that the "logical"
place of code creator is preserved.

My advice: implement your feature as is all code would be written in
single file.
Only then think about multiple files.


> >> Just to see if the file gets written or not I called add_system_header()
> >> and adopt_creator() on every call to visit_member_function(), but then I
> >> get a huge list of identical include statements in the generated files.
> >
> > I don't fully understand what you are doing here.
>
> I was just doing what you were suggesting earlier, i.e. to call
> add_system_header() to have Py++ write a particular header from the code
> repository package. This did work but the file was nowhere included, so
> I added an "include" code creator using adopt_creator() on the root node
> (I was peeking at the array_1 stuff to see how it is done). But as I was
> doing that in every call to visit_member_function() the file got
> included again each time resulting in a lot of duplicate lines.

Okey. You will have to prevent this and I think you already understand
how to do this
or already did. Am I right?

> >> What I would actually like to do is to add the "include" code creator to
> >> the associated_decl_creators list of the corresponding class so that the
> >> include only shows up in the file where it is needed. But how can I
> >> check whether the file was already included or not?
> >
> > If we talk about system header than you can ask module_t code creator
> > ( creator_t.__extmodule ) is_system_header.
>
> ok, this is what I'm doing now and which does work for now. But this
> makes the include file a "project wide" include file. How should I do
> this check if I would only like to add the include code creator to the
> associated_decl_creators list? (so that it only gets included where it
> is required?)

It is not possible. I tell you more, it will not be possible for few
next versions.
Py++ is missinig one concept - "bind" or "connect". Today user is not able to
"bind" between different declarations and different pieces of generated code.

Consider next cases:
* expose get\\set function as python properties
* expose begin\\end iterator as Boost.Python range
* add code "X" to the file, where declaration Y will be generated.
* register opaque type, when function has relevant call policy

I can continue the list, but I think you understand me. The small
refactoring I do on
code creators and file writers should improve (not fix ) the situation.

> >> I wonder why there is no specific API for managing include files that
> >> handles all those stuff? It appears to me that this would be quite a
> >> useful thing to have.
> >
> > Lack of time and low priority. Users learned to leave with "non
> > optimal" generated
> > code. I mean that all include appears in all header files. But they don't like
> > Py++, when it does not support the functionality they needed.
>
> That's not what I meant. I mean, even when the current functionality is
> exactly preserved, I would rather like to have an (internal) API that
> handles the details of how an include file is added to the output files.
> This is required at several places and it's not easy to understand how
> it is properly done. An additional benefit would be that it would pave
> the way for improving the handling of include files so that they could
> be set on a file-by-file basis.

I think I answered your question. In other words the problem is
"wider" and I want
to have 1 solution for it.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/