pygccxml-development

[pygccxml-development] array_t vs pointer_t

[Matthias B. <ba...@ir...>]

Hi again,

I'm wrapping a method that takes one single argument of type "double 
scale[3]". In the argument list of the corresponding declaration object 
this appears as a pointer_t object with a double_t object as base. What 
I'm missing is the '3'.
I noticed that pyplusplus defines a type array_t. Shouldn't the above 
argument be an array_t instead of a pointer_t to a double? Is this a bug 
in pyplusplus, in gccxml or something else?

- Matthias -

Re: [pygccxml-development] array_t vs pointer_t

[Roman Y. <rom...@gm...>]

On 4/11/06, Matthias Baas <ba...@ir...> wrote:
> Hi again,
>
> I'm wrapping a method that takes one single argument of type "double
> scale[3]". In the argument list of the corresponding declaration object
> this appears as a pointer_t object with a double_t object as base. What
> I'm missing is the '3'.
> I noticed that pyplusplus defines a type array_t. Shouldn't the above
> argument be an array_t instead of a pointer_t to a double? Is this a bug
> in pyplusplus, in gccxml or something else?

I could be wrong, but this is how C/C++ works. Array is passed by pointers.
Am I wrong? Even if I wrong, GCC-XML reports this.

I will think what can be done.

> - Matthias -

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] array_t vs pointer_t

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> I'm wrapping a method that takes one single argument of type "double
>> scale[3]". In the argument list of the corresponding declaration object
>> this appears as a pointer_t object with a double_t object as base. What
>> I'm missing is the '3'.
>> I noticed that pyplusplus defines a type array_t. Shouldn't the above
>> argument be an array_t instead of a pointer_t to a double? Is this a bug
>> in pyplusplus, in gccxml or something else?
> 
> I could be wrong, but this is how C/C++ works. Array is passed by pointers.
> Am I wrong? Even if I wrong, GCC-XML reports this.

That's too bad. :( The array size would have been a valuable piece of 
information for creating the wrappers. Without them I have to wade 
through ~250 methods manually to check if the arguments are really plain 
pointers or fixed size arrays as above (which seems to be the majority 
of cases in the Maya SDK)...

In which case does pygccxml create an array_t object then?

- Matthias -

Re: [pygccxml-development] array_t vs pointer_t

[Roman Y. <rom...@gm...>]

On 4/11/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> >> I'm wrapping a method that takes one single argument of type "double
> >> scale[3]". In the argument list of the corresponding declaration objec=
t
> >> this appears as a pointer_t object with a double_t object as base. Wha=
t
> >> I'm missing is the '3'.
> >> I noticed that pyplusplus defines a type array_t. Shouldn't the above
> >> argument be an array_t instead of a pointer_t to a double? Is this a b=
ug
> >> in pyplusplus, in gccxml or something else?
> >
> > I could be wrong, but this is how C/C++ works. Array is passed by point=
ers.
> > Am I wrong? Even if I wrong, GCC-XML reports this.
>
> That's too bad. :( The array size would have been a valuable piece of
> information for creating the wrappers. Without them I have to wade
> through ~250 methods manually to check if the arguments are really plain
> pointers or fixed size arrays as above (which seems to be the majority
> of cases in the Maya SDK)...

:-(.

First of all I will ask on GCC-XML list what could be done.

Second it is possible to implement solution using pygccxml only.
We can implement it and see how well it will work.
Description:
    1. Please take a look on declarations/call_invocation.py module.
        ( There is some doc strings within it )
    2. Please take a look on parser/patcher.py module.
        Basically GCC-XML has few bugs/features that pygccxml corrects.
        patch_it function is invoked from
        source_reader_t.__parse_gccxml_created_file method for every parsed
        file.

The idea is next: you can create your own patcher and rewrite/replace patch=
_it
with an other one. Every calldef declaration has location property initiali=
zed.
Thus you can get file path and line of declaration. Using type_traits
you can find
whether function has relevant type of arguments or not. After this you can =
go to
file, read original declaration, using call_invocation parser to split
it's arguments
and then find out whether it is array or not. After this you can replace ty=
pe of
argument to be an array of relevant type.

I don't say that this is a perfect method. But I think it will work in
most cases.

An other way to solve this issue is to create small parser, that will parse=
 all
header files and will create table with relevant information. Then you
can just use
that information from your script. I use this trick with tnfox. Take a look=
 on
examples/tnfox/declarations_to_exclude.py/find_deprecated function.

> In which case does pygccxml create an array_t object then?

When you have member/free variable declared as array:

const int items[23];

> - Matthias -

Hope this will help you.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] array_t vs pointer_t

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>>>> I'm wrapping a method that takes one single argument of type "double
>>>> scale[3]". In the argument list of the corresponding declaration object
>>>> this appears as a pointer_t object with a double_t object as base. What
>>>> I'm missing is the '3'.
>>>> I noticed that pyplusplus defines a type array_t. Shouldn't the above
>>>> argument be an array_t instead of a pointer_t to a double? Is this a bug
>>>> in pyplusplus, in gccxml or something else?
>>> I could be wrong, but this is how C/C++ works. Array is passed by pointers.
>>> Am I wrong? Even if I wrong, GCC-XML reports this.
>> That's too bad. :( The array size would have been a valuable piece of
>> information for creating the wrappers. Without them I have to wade
>> through ~250 methods manually to check if the arguments are really plain
>> pointers or fixed size arrays as above (which seems to be the majority
>> of cases in the Maya SDK)...
> 
> [...]
> An other way to solve this issue is to create small parser, that will parse all
> header files and will create table with relevant information. Then you
> can just use
> that information from your script. 

I took this route and meanwhile I've implemented it. So obtaining the 
array size is no problem anymore and I could assign argument policies 
automatically to most methods.

Now I have the problem that the argument policy mechanism sometimes 
produces wrapper methods that are ambiguous. Looks like I have to 
improve my prototype so that it recognizes these cases and asks for user 
help.

But at least obtaining the array size (what this thread was actually 
about) works fine.
By the way, maybe this code might be useful to some users at some point, 
so I think it might be worth sharing eventually. But I don't think it's 
really general and stable enough to be part of the official pyplusplus 
package, so how about adding a "contrib" directory when the code is 
moved to svn (maybe pyplusplus_dev/contrib) where contributions from 
users could be put that is not "good enough" yet for the official 
package but that might still be useful to the community either as an 
example for something or as a basis for tailoring it to their own 
projects? This will be code that does not have to be maintained or 
supported by yourself (Roman) but users have the option to use it "at 
their own risk".
Such a scheme is used by ODE (Open Dynamics Engine) and only 
occasionally is code from contrib moved to the main branch when it 
proves to be useful and becomes stable.

- Matthias -

Re: [pygccxml-development] array_t vs pointer_t

[Roman Y. <rom...@gm...>]

On 4/20/06, Matthias Baas <ba...@ir...> wrote:
> > An other way to solve this issue is to create small parser, that will p=
arse all
> > header files and will create table with relevant information. Then you
> > can just use
> > that information from your script.

I glad you found this solution simple and useful.

> I took this route and meanwhile I've implemented it. So obtaining the
> array size is no problem anymore and I could assign argument policies
> automatically to most methods.

Those are good news. But as for me, the better news is that you did it
using some custom program and then share this knowledge with code generator=
.
This is really grate. This was one of my goals.

> Now I have the problem that the argument policy mechanism sometimes
> produces wrapper methods that are ambiguous. Looks like I have to
> improve my prototype so that it recognizes these cases and asks for user
> help.

Can you post example in relevant thread? ( Where we discuss the feature ).

> But at least obtaining the array size (what this thread was actually
> about) works fine.
> By the way, maybe this code might be useful to some users at some point,
> so I think it might be worth sharing eventually. But I don't think it's
> really general and stable enough to be part of the official pyplusplus
> package, so how about adding a "contrib" directory when the code is
> moved to svn (maybe pyplusplus_dev/contrib) where contributions from
> users could be put that is not "good enough" yet for the official
> package but that might still be useful to the community either as an
> example for something or as a basis for tailoring it to their own
> projects? This will be code that does not have to be maintained or
> supported by yourself (Roman) but users have the option to use it "at
> their own risk".
> Such a scheme is used by ODE (Open Dynamics Engine) and only
> occasionally is code from contrib moved to the main branch when it
> proves to be useful and becomes stable.

Go ahead. I like this idea. I saw it in docutils.

> - Matthias -

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] array_t vs pointer_t

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>>> An other way to solve this issue is to create small parser, that will parse all
>>> header files and will create table with relevant information. Then you
>>> can just use
>>> that information from your script.
> 
> I glad you found this solution simple and useful.

Not really simple, but definitely useful... ;)

>> Now I have the problem that the argument policy mechanism sometimes
>> produces wrapper methods that are ambiguous. Looks like I have to
>> improve my prototype so that it recognizes these cases and asks for user
>> help.
> 
> Can you post example in relevant thread? ( Where we discuss the feature ).

The problem is not in pyplusplus, it's in my own code. For example, I 
have functions like this:

  void getValue(int& val)
  void getValue(double& val)

When I apply the Output(1) policy, both wrappers look identical (except 
for their return value, but there are also examples where even the 
return value is identical):

  int getValue_wrapper()
  double getValue_wrapper()

But meanwhile, I've updated my code so that these clashes are recognized 
and reported (and can then be resolved manually).
After fixing a series of smaller issues in my code it seems to compile 
again.... (but I can only say for sure in about an hour or so when 
compilation is done)...

[contrib dir]
> Go ahead. I like this idea. I saw it in docutils.

ok. Do you already know when you will move the code over to svn? (I will 
wait until this is done. By the way, I will also wait with updating the 
setup scripts (so that they can be used for the source distributions) 
until the transition is finished)

- Matthias -

Re: [pygccxml-development] array_t vs pointer_t

[Roman Y. <rom...@gm...>]

On 4/20/06, Matthias Baas <ba...@ir...> wrote:
> The problem is not in pyplusplus, it's in my own code. For example, I
> have functions like this:
>
>   void getValue(int& val)
>   void getValue(double& val)
>
> When I apply the Output(1) policy, both wrappers look identical (except
> for their return value, but there are also examples where even the
> return value is identical):
>
>   int getValue_wrapper()
>   double getValue_wrapper()
>
> But meanwhile, I've updated my code so that these clashes are recognized
> and reported (and can then be resolved manually).
> After fixing a series of smaller issues in my code it seems to compile
> again.... (but I can only say for sure in about an hour or so when
> compilation is done)...

:-) All you need is reliable way to give function unique name. You can
construct
name of wrapper from from name + line number. Every declaration has
location property. Or another option: every declaration has mangled
property, you can use
it within the name.

Please consider this idea and post the result. This is important, because p=
y++
will be able to generate less readable code, but it will compile faster.

> [contrib dir]
> > Go ahead. I like this idea. I saw it in docutils.
>
> ok. Do you already know when you will move the code over to svn? (I will
> wait until this is done. By the way, I will also wait with updating the
> setup scripts (so that they can be used for the source distributions)
> until the transition is finished)

Next week or two. I want to fix some bugs found by people and then to move.

> - Matthias -


--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] array_t vs pointer_t

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> On 4/20/06, Matthias Baas <ba...@ir...> wrote:
>> The problem is not in pyplusplus, it's in my own code. For example, I
>> have functions like this:
>>
>>   void getValue(int& val)
>>   void getValue(double& val)
>>
>> When I apply the Output(1) policy, both wrappers look identical (except
>> for their return value, but there are also examples where even the
>> return value is identical):
>>
>>   int getValue_wrapper()
>>   double getValue_wrapper()
>>
>> But meanwhile, I've updated my code so that these clashes are recognized
>> and reported (and can then be resolved manually).
>> After fixing a series of smaller issues in my code it seems to compile
>> again.... (but I can only say for sure in about an hour or so when
>> compilation is done)...
> 
> :-) All you need is reliable way to give function unique name. You can
> construct
> name of wrapper from from name + line number. Every declaration has
> location property. Or another option: every declaration has mangled
> property, you can use it within the name.

Actually, I do have unique names now (using id(decl)), but for different 
reasons. In the above case, this would not be a solution as long as the 
wrappers have the same name in Python (and when they do not have the 
same name, then the above problem doesn't exist anyway). Which version 
should Boost.Python pick when I call getValue() in Python? So the 
wrappers must have different names in Python.

>> [contrib dir]
>>> Go ahead. I like this idea. I saw it in docutils.
>> ok. Do you already know when you will move the code over to svn? (I will
>> wait until this is done. By the way, I will also wait with updating the
>> setup scripts (so that they can be used for the source distributions)
>> until the transition is finished)
> 
> Next week or two. I want to fix some bugs found by people and then to move.

ok

- Matthias -