pygccxml-development

Re: [pygccxml-development] CVS GUI

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> I know, this is not directly related to the list, but still:
> what CVS GUI on Linux do you use or can recommend?

I'm only using the cvs command line utility.

- Matthias -

Re: [pygccxml-development] Printing declarations...

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> Should the fix for the copy constructor also be available yet? (they are
>> still not wrapped automatically)
> 
> I am doing something wrong with constructors. I do not know what :-(.

What's wrong with the other constructors? For me, it's only the copy 
constructor that doesn't get wrapped, all other constructors have been 
fine so far.

>> The call operator also doesn't seem to
>> get wrapped (hasn't that been already done by earlier versions)?
> 
> I think you find a bug.  I just run special_operators_tester.py and it pass.
> Can you reproduce it? Thanks

Ahem, I forgot that I was explicitly ignoring it.....
The actual problem is that the generated code won't compile on Windows 
(using VC7.1). pyplusplus generates the following code:

         MVector_exposer.def( "__call__"
                 , (double ( ::MVector::* )( unsigned int ) 
const)(&MVector::operator())
                 , ( bp::arg("i") )
                 , bp::default_call_policies() );

When I try to compile this I get a syntax error because of the 
"operator()" (VC7.1 gets confused by the brackets). Whereas when I 
create the following typedef

   typedef double (::MVector::*bracket_op)(unsigned int i) const;

and then rewrite the above as

         MVector_exposer.def( "__call__"
                 , (bracket_op)(&MVector::operator())
                 , ( bp::arg("i") )
                 , bp::default_call_policies() );

then it compiles fine. Could this scheme be incorporated in pyplusplus?

>> Now to my main question for this mail: Is there already a way to obtain
>> a "full" string representation of a declaration (for printing)? I'm only
>> aware of the full_name() function but (as the name already implies) this
>> only returns the name whereas for methods I'd like to see the full
>> signature as well (to be able to distinguish between overloaded methods).
> 
> x.decl_string
> 
> This should do it.

Almost, the signature is there but the actual method name is missing.

- Matthias -

[pygccxml-development] Status:

[Roman Y. <rom...@gm...>]

Plan for next week or two
1. adding code creators to decl wrappers

Done.

2. resolving function overload issues

Matthias, I would like to see your solution before I make any decision. Tha=
nks.

3. fixing finalize concept.

I will do it this evening

4. add few function/properties to module builder class:
   parser configuration
   code creators ( factory ) configuration

I hope I will be able to add this tomorrow

5. Documentation. Here I really need your help. If you can write small
tutorial, I will be
    glad. I think this will be the only stopper for making release.

Also take a look on pyplusplus boost.date_time example. It looks much,
much, much better.

Thank you very much.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] High level API

[Roman Y. <rom...@gm...>]

Hi. We have small problem in all versions of API's. ( I think )

Module builder class implicitly done some steps if user did not them.
For example if user didn't called "parse" method, then it will be
called from next step by framework. The same will happen with
code_creators tree. What is wrong with this?

There are some customization that user can do on declarations, even
after code creators
tree creation. But there are set of customizations that will not have
effect on code creators.
For example ignore. In big project it cause a problem, because
customize code is not
written in one place.

I think, we should change the behaviour in next way:

__init__ function will take as parameters:
    module name
    list of files
    gccxml parser configuration

__init__ function will parse file and put global namespace decl in
global_ns member variable

declarations member variable will be deleted

From this point if user tries to work with code creators tree or just
write code to
disk, every step should be done explicitly by user, otherwise
exception will be raised.

create_code_creators_tree ( please give better name for the function )
function will
create/initialize module_creator instance

write_file/write_multiple_files will write code to disk

What do you think? I just debug my own program that had the bug like
this. I did not
like the fact that it is easy to be confused.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Another status report...

[Roman Y. <rom...@gm...>]

On 3/19/06, Allen Bierbaum <al...@vr...> wrote:
> > So what are your plans for the next couple of days? What does our next
> > "milestone" look like? I'd like to see the basic principles of
> > selection/decoration to be resolved so that we can move on with
> > thinking about features that have only been touched so far but that are=
 no
>
> I think a good milestone would be an API that would allow the
> capabilities that my original simple prototype allowed.  In other words,
> basic selection, exposing, renaming, and policy.  In a way this will
> really be the core capabilities that most users will need.

I think, that today, after I commit add_code functionality we will have it.

> -Allen
>

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Another status report...

[Allen B. <al...@vr...>]

Matthias Baas wrote:

> Hi,
>
> unless I find some bugs I'm currently done with the selection stuff in 
> the experimental folder. With the current version I can now almost 
> create the same bindings as before with only slight adjustments to my 
> script. By default, the selection is similar to Allen's initial 
> version  but you can "activate" more flexible queries if desired to 
> get something that rather resembles my initial version.
> The next thing I need to get the same bindings as before is the 
> ability to add custom methods, I haven't checked yet if Allen's 
> addMethod() method still works or not, I'll try to do that tomorrow....
>
> I have also done some updates in the wiki:
>
> - There's one more goal I added and I even dared adding it to the top 
> of the list as I would consider that this is the "ultimate" goal for 
> me: pyplusplus  (and its API) should minimize the time it takes to 
> create the final bindings for a C++ library. Do you agree on that?

Yes, I agree with the condition that we mean minimize the time to create 
the final bindings assuming you already know something about 
boost.python and pyplusplus.  If the user doesn't know either tool yet 
then it may take a while to understand the basics.

>
> - To most of the issues I added an "Experimental version" section that 
> provides a short outline how the current code in the experimental 
> folder handles the respective issue. Of course, as this code is still 
> "experimental" this doesn't mean that this is how the issue will 
> definitely be resolved, but it's rather one proposal that exists in 
> code so that it can be tried out to see how well it works in practice.

Good idea.

>
> - I've removed my comments from the "What operations does the 
> selection interface provide?" topic as there is no general 
> disagreement on my part. But I added a new "open issue" about the 
> overloaded methods.

>
>
> How should we proceed now to resolve the issues in the wiki? There 
> still seems to be some disagreement, but I don't always see how an 
> alternative proposal would look like. So should we schedule another 
> chat session and go over those issues trying to find a resolution? Or 
> do you already have some comments about my latest changes? Could you 
> try them out? Do you need more documentation about the usage? Does the 
> selection work for you or were you confused when you tried using it? 
> (for me, it's still the overloaded methods that can lead to some 
> "surprises")

I really need to get some time to test the new interface.  I have gotten 
swamped by a couple of big projects and have not had time to devote to 
this in about a week. :(

>
> So what are your plans for the next couple of days? What does our next 
> "milestone" look like? I'd like to see the basic principles of 
> selection/decoration to be resolved so that we can move on with 
> thinking about features that have only been touched so far but that are no

I think a good milestone would be an API that would allow the 
capabilities that my original simple prototype allowed.  In other words, 
basic selection, exposing, renaming, and policy.  In a way this will 
really be the core capabilities that most users will need.

-Allen

> t yet implemented at all.
>
> - Matthias -
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> pygccxml-development mailing list
> pyg...@li...
> https://lists.sourceforge.net/lists/listinfo/pygccxml-development
>

[pygccxml-development] CVS GUI

[Roman Y. <rom...@gm...>]

I know, this is not directly related to the list, but still:
what CVS GUI on Linux do you use or can recommend?

Thanks

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Adding code creators to decl_wrappers...

[Roman Y. <rom...@gm...>]

On 3/17/06, Matthias Baas <ba...@ir...> wrote:
> I realized I'm still missing the "global view" on the implementation of
>   pyplusplus and that my approach was too short-sighted (for example, I
> totally neglected that someone might want to add stuff to the class
> wrappers) and I agree that a more general mechanism should be implemented=
.

If you want I can find some time to explain you design and main concept of =
C++.
I mean we can chat on irc, from my home computer :-)!

> However, I'm afraid I'm not able to implement that myself yet as your
> explanations contained some stuff I'm either not familiar with or I
> don't see how a concrete implementation would look like.

I will try to commit first version on Sanday or Monday.

> > In general this is what I expected so much time :-). Also does it mean =
that
> > you use module_builder_t class and you more or less happy with it's int=
erface?
>
> I'm still using the module builder from the experimental module.
> But I did notice that you've added the selection interface to the
> appropriate decl_wrappers.

and to the module builder itself.

> - Matthias -

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Printing declarations...

[Roman Y. <rom...@gm...>]

On 3/17/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> finally I can send a mail as a *user* of pyplusplus again and not as a
> developer... :)

Well, it took some time, but ...

> I was pleased to see that the optional<> stuff is in cvs and working!

:-). Enjoy.

> Should the fix for the copy constructor also be available yet? (they are
> still not wrapped automatically)

I am doing something wrong with constructors. I do not know what :-(.
In order to export constructors I need to take into account too many facts.
So, until somebody will write comprehensive test case it will not be done i=
n few
weeks or may be months. :-(

>The call operator also doesn't seem to
> get wrapped (hasn't that been already done by earlier versions)?

I think you find a bug.  I just run special_operators_tester.py and it pass=
.
Can you reproduce it? Thanks

> Now to my main question for this mail: Is there already a way to obtain
> a "full" string representation of a declaration (for printing)? I'm only
> aware of the full_name() function but (as the name already implies) this
> only returns the name whereas for methods I'd like to see the full
> signature as well (to be able to distinguish between overloaded methods).

x.decl_string

This should do it.

> - Matthias -

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Printing declarations...

[Matthias B. <ba...@ir...>]

Hi,

finally I can send a mail as a *user* of pyplusplus again and not as a 
developer... :)

I was pleased to see that the optional<> stuff is in cvs and working! 
Should the fix for the copy constructor also be available yet? (they are 
still not wrapped automatically) The call operator also doesn't seem to 
get wrapped (hasn't that been already done by earlier versions)?

Now to my main question for this mail: Is there already a way to obtain 
a "full" string representation of a declaration (for printing)? I'm only 
aware of the full_name() function but (as the name already implies) this 
only returns the name whereas for methods I'd like to see the full 
signature as well (to be able to distinguish between overloaded methods).

- Matthias -

Re: [pygccxml-development] Adding code creators to decl_wrappers...

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> Before I start, I want you to know that I thought about this feature
> from the moment I saw it. It was not clear to me how it should be implemented.
> [...]

I realized I'm still missing the "global view" on the implementation of 
  pyplusplus and that my approach was too short-sighted (for example, I 
totally neglected that someone might want to add stuff to the class 
wrappers) and I agree that a more general mechanism should be implemented.
However, I'm afraid I'm not able to implement that myself yet as your 
explanations contained some stuff I'm either not familiar with or I 
don't see how a concrete implementation would look like.
But as I do have a working solution for the moment, I can live with that 
until the "official" version is in. I suppose it would also be no big 
deal to move the (limited) functionality that I currently need to the 
high level layer, so that I can again use the same version of pyplusplus 
that everyone else is using as well (instead of having a "private" branch).

> In general this is what I expected so much time :-). Also does it mean that
> you use module_builder_t class and you more or less happy with it's interface?

I'm still using the module builder from the experimental module.
But I did notice that you've added the selection interface to the 
appropriate decl_wrappers.

> P.S. Do you see the hour I sent this message? - As you can guess
> I have Internet at home.

Ah, welcome back to the online community! :)

- Matthias -

Re: [pygccxml-development] Adding code creators to decl_wrappers...

[Roman Y. <rom...@gm...>]

On 3/16/06, Matthias Baas <ba...@ir...> wrote:
> Hi,

Before I start, I want you to know that I thought about this feature
from the moment I saw it. It was not clear to me how it should be implement=
ed.
I don't think that code creators and decl wrapper should have cross
dependencies.
This is technical reason. There is another reason: code creator is concept.
Every code creator has parent code creator and only one.  Code creators is
a data structure that have an order. I don't want to mix decl_wrapper and
code creators. I am afraid we will have some requests from the user, we
will not be able to implement with code creators within decl wrappers.
I think I find solution,  while I am writing this mail.

> the last "high priority" thing (at least for me) that I was missing was
> the ability to add code creators to declarations, so I gave it a try
> myself and modified pyplusplus.

In general this is what I expected so much time :-). Also does it mean that
you use module_builder_t class and you more or less happy with it's interfa=
ce?

> Here is what I did:
>
> - In decl_wrappers.scopedef_t I added a method "append_creator()" that
> adds a code creator to an internal list (which is made available as a
> read-only property).

I prefer add_text, where add_text defined as

def add_text( self, text, works_on_instance=3DTrue ):
    self.additional_text.apend( class_text_t( text,  works_on_instance ) )

where class_text_t is next:
class class_text_t:
    def __init__( self, text, works_on_instance ):
        self.text =3D text
        self.works_on_instance =3D works_on_instance

If you take a look on code_creators/custom.py you will find class very
similar to
this one. But that one is real code creator. May be I choose bad name, but
I hope you've got the idea. Also you can suggest your name, but it should
not contain "code creator" words. :-)

> - In code_creators.scoped_t I re-implemented the create() method so that
> it first adds the creators from the declaration to the local list of
> creators and then it calls the inherited create() method. I did that
> here and not in the constructor because I thought it's better to add the
> custom code creators to the *end* of the list instead of the beginning.

You have 2 little mistakes here, sorry :-(
1. You can not add them from the constructor. module_creator.creator_t
class does not create all internals code creators before external one.

2. Why do you want to give an user only one place, where user can add
custom text? For example in class_text_t you can add new argument - positio=
n.
Thus user will be able to say position, relative to code creators. I
don't ask you
to implement this behaviour, but I think you should be aware about new futu=
res
the user  can ask for.

> Roman, is that along the lines as you were thinking to implement it
> yourself? (should append_creator() rather be called adopt_creator() as
> they are already called in the code creators?)
> If you have no objections against the above approach I can commit the
> changes.

I am not sure whether we should change code_creators.scopedef_t and
decl_wrappers.scopedef_t classes. You can not add new code to
namespace.
There is no such thing in code creators policy right now. Also, If I rememb=
er
right all your use cases is to add new code to some class, right?
So, I prefer you will change decl_wrapper.class_t. I think we should add 2
lists here. The first one will contain code for boost::python::class_ code.
And the second will contain code for class wrapper. I think it will cover 8=
0% of
cases, when and where user need to add some code. Also I think we should
add an other 2 lists for module and module body. Thus user will be
able to modify
"main" function and code before it.

Also I think that you should not touch create function. Really, the
better approach
it to override code_creators.compaund_t._get_creators function. It will ret=
urn
new list, that contains from real code creators. You will create custom_tex=
t_t
code creator on fly.

> - Matthias -

What do you think? Please critic this solution. If you agree with this solu=
tion,
can you implement it and commit?

Thanks


P.S. Do you see the hour I sent this message? - As you can guess
I have Internet at home.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Adding code creators to decl_wrappers...

[Matthias B. <ba...@ir...>]

Hi,

the last "high priority" thing (at least for me) that I was missing was 
the ability to add code creators to declarations, so I gave it a try 
myself and modified pyplusplus.
Here is what I did:

- In decl_wrappers.scopedef_t I added a method "append_creator()" that 
adds a code creator to an internal list (which is made available as a 
read-only property).

- In code_creators.scoped_t I re-implemented the create() method so that 
it first adds the creators from the declaration to the local list of 
creators and then it calls the inherited create() method. I did that 
here and not in the constructor because I thought it's better to add the 
custom code creators to the *end* of the list instead of the beginning.

Roman, is that along the lines as you were thinking to implement it 
yourself? (should append_creator() rather be called adopt_creator() as 
they are already called in the code creators?)
If you have no objections against the above approach I can commit the 
changes.

- Matthias -

Re: [pygccxml-development] Another status report...

[Roman Y. <rom...@gm...>]

On 3/15/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> unless I find some bugs I'm currently done with the selection stuff in
> the experimental folder. With the current version I can now almost
> create the same bindings as before with only slight adjustments to my
> script. By default, the selection is similar to Allen's initial version
>   but you can "activate" more flexible queries if desired to get
> something that rather resembles my initial version.
> The next thing I need to get the same bindings as before is the ability
> to add custom methods, I haven't checked yet if Allen's addMethod()
> method still works or not, I'll try to do that tomorrow....

Can you work directly on decl_wrappers? Now I understand that this is a ver=
y
important future.

> I have also done some updates in the wiki:
>
> - There's one more goal I added and I even dared adding it to the top of
> the list as I would consider that this is the "ultimate" goal for me:
> pyplusplus  (and its API) should minimize the time it takes to create
> the final bindings for a C++ library. Do you agree on that?

I don't know, if we achieve all other goals this will be achieved as well.

> - To most of the issues I added an "Experimental version" section that
> provides a short outline how the current code in the experimental folder
> handles the respective issue. Of course, as this code is still
> "experimental" this doesn't mean that this is how the issue will
> definitely be resolved, but it's rather one proposal that exists in code
> so that it can be tried out to see how well it works in practice.

Good.

> - I've removed my comments from the "What operations does the selection
> interface provide?" topic as there is no general disagreement on my
> part. But I added a new "open issue" about the overloaded methods.

I will take a look, later

> How should we proceed now to resolve the issues in the wiki? There still
> seems to be some disagreement, but I don't always see how an alternative
> proposal would look like. So should we schedule another chat session and
> go over those issues trying to find a resolution? Or do you already have
> some comments about my latest changes?

Yes. I have. But I prefer to tell you good news. Almost all features
you need for Maya SDK
already in main branch. There 2 missing features:

adding code creators to decl_wrappers

resolving issue with overloaded functions

> Could you try them out?

I tried them and read your code.

> Do you need more documentation about the usage?

No

> Does the selection work for you or were you confused when you tried using=
 it?

Yes, a little. You mix 2 concepts:
1. what user is searching for
2. where it should be searched.

If you want I can explain what is wrong with it. But I really don't
want. You are doing great
job and I only tell you these or that thing is wrong. I took almost
all your ideas and
added them to main branch.

1. filters, I made for example only cosmetic changes and add more power in =
some
    places

2. select interface. it is now fully implemented.

>(for me, it's still the
> overloaded methods that can lead to some "surprises")
>
> So what are your plans for the next couple of days?

1. First of all I will port all my code to new API. This step will
find bugs and weaknesses.

>What does our next "milestone" look like?

There are few issues need to be resolved in next week or two:
1. adding code creators to decl wrappers
2. resolving function overload issues
3. fixing finalize concept.
4. add few function/properties to module builder class:
    parser configuration
    code creators ( factory ) configuration

After this point: we should stay backward compatible
( =3D=3D you can use source code from CVS in production ). It means that=20
Maya bindings should work!

After this I plan to release "technology preview" version to
boost.python community.
( Hint: this is the first time that documentation will be integrated
and release :-) )

>I'd like to see the basic principles of
> selection/decoration to be resolved so that we can move on with thinking
> about features that have only been touched so far but that are not yet
> implemented at all.

They already resolved. In order to understand how all works take a look on:
pygccxml/unittests/
    filters_tester.py
    variable_matcher_tester.py
    namespace_matcher_tester.py

pyplusplus/unittests/ relevant code in customize method:
    member_functions_tester.py
    member_variables_tester.py
    operators_tester.py
    regression2_tester.py
    recursive_tester.py

pyplusplus/examples/py_easybmp/create_easybmp.py

> - Matthias -

What do you think?

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Another status report...

[Matthias B. <ba...@ir...>]

Hi,

unless I find some bugs I'm currently done with the selection stuff in 
the experimental folder. With the current version I can now almost 
create the same bindings as before with only slight adjustments to my 
script. By default, the selection is similar to Allen's initial version 
  but you can "activate" more flexible queries if desired to get 
something that rather resembles my initial version.
The next thing I need to get the same bindings as before is the ability 
to add custom methods, I haven't checked yet if Allen's addMethod() 
method still works or not, I'll try to do that tomorrow....

I have also done some updates in the wiki:

- There's one more goal I added and I even dared adding it to the top of 
the list as I would consider that this is the "ultimate" goal for me: 
pyplusplus  (and its API) should minimize the time it takes to create 
the final bindings for a C++ library. Do you agree on that?

- To most of the issues I added an "Experimental version" section that 
provides a short outline how the current code in the experimental folder 
handles the respective issue. Of course, as this code is still 
"experimental" this doesn't mean that this is how the issue will 
definitely be resolved, but it's rather one proposal that exists in code 
so that it can be tried out to see how well it works in practice.

- I've removed my comments from the "What operations does the selection 
interface provide?" topic as there is no general disagreement on my 
part. But I added a new "open issue" about the overloaded methods.


How should we proceed now to resolve the issues in the wiki? There still 
seems to be some disagreement, but I don't always see how an alternative 
proposal would look like. So should we schedule another chat session and 
go over those issues trying to find a resolution? Or do you already have 
some comments about my latest changes? Could you try them out? Do you 
need more documentation about the usage? Does the selection work for you 
or were you confused when you tried using it? (for me, it's still the 
overloaded methods that can lead to some "surprises")

So what are your plans for the next couple of days? What does our next 
"milestone" look like? I'd like to see the basic principles of 
selection/decoration to be resolved so that we can move on with thinking 
about features that have only been touched so far but that are not yet 
implemented at all.

- Matthias -

Re: [pygccxml-development] Serious bug in pygccxml (scopedef.py)

[Roman Y. <rom...@gm...>]

On 3/15/06, Matthias Baas <ba...@ir...> wrote:
> But if those special =3D=3D and !=3D operators are only required for unit
> testing, wouldn't it then be better to move them to the unit tests and
> keep the standard operators for the regular users? From what I've seen
> the operators implement rather expensive operations (copying, sorting,
> comparing several values, etc.) and if this could just as well be done
> by comparing the reference of the instance, this would be much more
> efficient.

I can argue about this point, but I understand you.

> >> But I think for someone else reading the code this is not really
> >> obvious why I have to do that...
> >
> > Matthias, it is not the first time I provide "not so user friendly" int=
erface.
> > Can you show some code you have to write now and how I think it should
> > be written.
>
> It's in the __call__() method of the ParentFilter class in
> experimental/filters.py. First, the method looked like this:
>
>      def __call__(self, decl):
>          return decl.parent=3D=3Dself.parent
>
> Because of the bug in the =3D=3D operator I had a look at the implementat=
ion
> and found out that the operator is doing quite a lot things. With the
> latest fix you've applied to scopedef.py the above code would work
> again, but it would not be as efficient as the following line:
>
>          return id(decl.parent)=3D=3Did(self.parent)

:-)  return decl.parent is self.decl.parent is it better ? :-)))))

> >> PS: I also don't understand the purpose of the (static!) method
> >> _sorted_list() as this method just calls sort() and does nothing else.
> >> Why wasn't sort() used in the first place?
> >
> > Convenience. Python sorts list in place and return value of sort
> > function is None.
> > This function allows you to write something like this:
> >
> > soorted_list ( x ) =3D=3D sorted_list( y )
> >
> > while without function you need to write
> >
> > x.sort()
> > y.sort()
> > x =3D=3D y
>
> ok, I see. But sorted_list() has the side effect of modifying its
> argument (which might not have been intentional). That's why the Python
> developers chose not to return the list from sort() (see note (7) here:
> http://docs.python.org/lib/typesseq-mutable.html)
> Maybe the semantics you actually had in mind was this:
>
>      def _sorted_list( some_list ):
>          result =3D copy.copy(some_list)
>          result.sort()
>          return result
>
> But of course, this makes the operation more expensive...

I prefer expensive and work, to quick and may be work :-)

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Serious bug in pygccxml (scopedef.py)

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> I committed proposed fixed

Thanks!

>> I have to admit that I dont't really understand why those operators are
>> overridden at all.
> 
> pygccxml.project_reader_t has 2 modes to parse group of files. I
> needed to write some
> test to check that both methods return same answer.
> 
> Also I needed to test cache algorithm.
> 
> In general those 2 methods exists for unit tests.
> 
>> In which case does it happen that two separate
>> declaration objects refer to the same declaration?
> 
> There is no such case
> 
>> (probably when a
>> declaration is included in two separate include files, right? But why
>> isn't the first declaration then reused in the first place?).
> 
> It is do re-used.

But if those special == and != operators are only required for unit 
testing, wouldn't it then be better to move them to the unit tests and 
keep the standard operators for the regular users? From what I've seen 
the operators implement rather expensive operations (copying, sorting, 
comparing several values, etc.) and if this could just as well be done 
by comparing the reference of the instance, this would be much more 
efficient.

>> But I think for someone else reading the code this is not really
>> obvious why I have to do that...
> 
> Matthias, it is not the first time I provide "not so user friendly" interface.
> Can you show some code you have to write now and how I think it should
> be written.

It's in the __call__() method of the ParentFilter class in 
experimental/filters.py. First, the method looked like this:

     def __call__(self, decl):
         return decl.parent==self.parent

Because of the bug in the == operator I had a look at the implementation 
and found out that the operator is doing quite a lot things. With the 
latest fix you've applied to scopedef.py the above code would work 
again, but it would not be as efficient as the following line:

         return id(decl.parent)==id(self.parent)

>> PS: I also don't understand the purpose of the (static!) method
>> _sorted_list() as this method just calls sort() and does nothing else.
>> Why wasn't sort() used in the first place?
> 
> Convenience. Python sorts list in place and return value of sort
> function is None.
> This function allows you to write something like this:
> 
> soorted_list ( x ) == sorted_list( y )
> 
> while without function you need to write
> 
> x.sort()
> y.sort()
> x == y

ok, I see. But sorted_list() has the side effect of modifying its 
argument (which might not have been intentional). That's why the Python 
developers chose not to return the list from sort() (see note (7) here: 
http://docs.python.org/lib/typesseq-mutable.html)
Maybe the semantics you actually had in mind was this:

     def _sorted_list( some_list ):
         result = copy.copy(some_list)
         result.sort()
         return result

But of course, this makes the operation more expensive...

- Matthias -

Re: [pygccxml-development] need help with query

[Roman Y. <rom...@gm...>]

On 3/15/06, Matthias Baas <ba...@ir...> wrote:
> > mb.classes( f1 | f2 )
>
> So you prefer this last line over the above two ways, right? I do agree
> with you and this is already how you can use my filter classes... :)

Ooooooooooops. It seems that I did not paid enough attention to your
classes (=3DFilterBase)
I just took a look on Or|And|Not Filters and did not understand why do
you need them.
Shame on me. I will add your classes to pygccxml.

Thank you very much. I am glad, that we talk.

> (By the way, as I see it, using the filter classes directly is already
> an advanced feature anyway. The "normal" user would just use the
> convenience arguments of the query methods (which are converted into
> filter classes internally))

Yes, but to write "mb.classes( f1 | f2 )" is cool.

> - Matthias -
>

Thank you very much.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] need help with query

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> f1 = filter(....)
> f2 = filter(...)
> 
> mb = module_builder_t(...)
> mb.classes( OrFilter( [f1,f2] ) )
> 
> but rather:
> 
> mb.classes( lambda decl: f1(decl) or f2(decl) )
> 
> I could be wrong, but in this case he will not learn OrFilter. If we
> implement your idea
> in a different way, then user can write next line
> 
> mb.classes( f1 | f2 )

So you prefer this last line over the above two ways, right? I do agree 
with you and this is already how you can use my filter classes... :)

(By the way, as I see it, using the filter classes directly is already 
an advanced feature anyway. The "normal" user would just use the 
convenience arguments of the query methods (which are converted into 
filter classes internally))

- Matthias -

Re: [pygccxml-development] Serious bug in pygccxml (scopedef.py)

[Roman Y. <rom...@gm...>]

On 3/15/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> I just had a case where a query for a single class produced a result
> that contained that class twice (this something that shouldn't be
> possible at all).
> It took me about an hour to find that bug. The problem is the scopedef_t
> class (from which class_t and namespace_t are derived). This class
> overrides the equality operator __eq__() and the implementation of that
> operator has unwanted side effects, i.e. it modifies the instances! The
> method calls self._sorted_list() with its declaration list as argument.
> _sorted_list() sorts the list inplace, so this means the declaration
> list gets modified. If you do that while iterating over the list you get
> a messed up result where a declaration may appear twice (while other
> declarations haven't been seen at all). I think this is quite a serious
> bug (that is difficult to track down).
>
> A quick fix would be to replace the last line in scopedef_t.__eq__() with=
:
>
>          return self._sorted_list( self.declarations[:] ) \
>                 =3D=3D other._sorted_list( other.declarations[:] )
>
> That is, instead of passing declarations directly we pass a copy of that
> list (using slice notation, you might also use the copy module).
> But of course, these involves two copy operations which makes the
> operator even more inefficient.

I committed proposed fixed

> I have to admit that I dont't really understand why those operators are
> overridden at all.

pygccxml.project_reader_t has 2 modes to parse group of files. I
needed to write some
test to check that both methods return same answer.

Also I needed to test cache algorithm.

In general those 2 methods exists for unit tests.

> In which case does it happen that two separate
> declaration objects refer to the same declaration?

There is no such case

>(probably when a
> declaration is included in two separate include files, right? But why
> isn't the first declaration then reused in the first place?).

It is do re-used.

> I think in my case I could switch to comparing id(decl) instead of decl
> itself.

I think that this is a better solution, than comparing decls

>But I think for someone else reading the code this is not really
> obvious why I have to do that...

Matthias, it is not the first time I provide "not so user friendly" interfa=
ce.
Can you show some code you have to write now and how I think it should
be written.

> - Matthias -

Thank you.

> PS: I also don't understand the purpose of the (static!) method
> _sorted_list() as this method just calls sort() and does nothing else.
> Why wasn't sort() used in the first place?

Convenience. Python sorts list in place and return value of sort
function is None.
This function allows you to write something like this:

soorted_list ( x ) =3D=3D sorted_list( y )

while without function you need to write

x.sort()
y.sort()
x =3D=3D y



--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Serious bug in pygccxml (scopedef.py)

[Matthias B. <ba...@ir...>]

Hi,

I just had a case where a query for a single class produced a result 
that contained that class twice (this something that shouldn't be 
possible at all).
It took me about an hour to find that bug. The problem is the scopedef_t 
class (from which class_t and namespace_t are derived). This class 
overrides the equality operator __eq__() and the implementation of that 
operator has unwanted side effects, i.e. it modifies the instances! The 
method calls self._sorted_list() with its declaration list as argument. 
_sorted_list() sorts the list inplace, so this means the declaration 
list gets modified. If you do that while iterating over the list you get 
a messed up result where a declaration may appear twice (while other 
declarations haven't been seen at all). I think this is quite a serious 
bug (that is difficult to track down).

A quick fix would be to replace the last line in scopedef_t.__eq__() with:

         return self._sorted_list( self.declarations[:] ) \
                == other._sorted_list( other.declarations[:] )

That is, instead of passing declarations directly we pass a copy of that 
list (using slice notation, you might also use the copy module).
But of course, these involves two copy operations which makes the 
operator even more inefficient.

I have to admit that I dont't really understand why those operators are 
overridden at all. In which case does it happen that two separate 
declaration objects refer to the same declaration? (probably when a 
declaration is included in two separate include files, right? But why 
isn't the first declaration then reused in the first place?).

I think in my case I could switch to comparing id(decl) instead of decl 
itself. But I think for someone else reading the code this is not really 
obvious why I have to do that...

- Matthias -

PS: I also don't understand the purpose of the (static!) method 
_sorted_list() as this method just calls sort() and does nothing else. 
Why wasn't sort() used in the first place?

Re: [pygccxml-development] need help with query

[Roman Y. <rom...@gm...>]

On 3/15/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> > I saw Matthias filters ( Or, Not and And ) and liked the idea. But (
> > there is always some 'but' :-( ).
> >
> > I don't think they are useful in the form like this. People are to
> > lazy to learn. So instead
> > of composing queries using Matthias classes, they will right small
> > lambda function that will
> > do the job. May be I wrong, may be not. I propose an other
> > implementation. Take a look on attached file. Right now it does not
> > work. This is the area I need help. Can you see what I am doing wrong?
> >
> > Lets say I have query class, that has __call__ method, that takes 1
> > argument - some text
> >
> > qa =3D query_t( 'a' ) #returns true if 'a' in text
> > qb =3D query_t( 'b' ) #returns true if 'b' in text
> >
> > qab =3D qa and qb
> > [...]
> > qaqb =3D qa or qb
>
> The above lines should read:
>
> qab =3D qa & qb
> ...
> qaqb =3D qa | qb
>
> "and", "or" is not the same as the &,| operators.
>
> > Do you like such approach? Matthias what do you think?
>
> I don't understand what problem you want to solve. Your query class just
> does the same than my filters, but above you say that this is too
> difficult to learn...? So in what respect is query_t easier to use than
> my filters?

By the way, I could be wrong. I just want a discuss this a little.

OrFilter and AndFilter takes as argument list of filters right. I
think that in most case
an user does not have more then 2 or 3 filters. I could be wrong. So I
think, user will not use next code

f1 =3D filter(....)
f2 =3D filter(...)

mb =3D module_builder_t(...)
mb.classes( OrFilter( [f1,f2] ) )

but rather:

mb.classes( lambda decl: f1(decl) or f2(decl) )

I could be wrong, but in this case he will not learn OrFilter. If we
implement your idea
in a different way, then user can write next line

mb.classes( f1 | f2 )

> - Matthias -

I hope, I was clear. I do like your idea, but I think it should be
improved a little.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] need help with query

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> I saw Matthias filters ( Or, Not and And ) and liked the idea. But (
> there is always some 'but' :-( ).
> 
> I don't think they are useful in the form like this. People are to
> lazy to learn. So instead
> of composing queries using Matthias classes, they will right small
> lambda function that will
> do the job. May be I wrong, may be not. I propose an other
> implementation. Take a look on attached file. Right now it does not
> work. This is the area I need help. Can you see what I am doing wrong?
> 
> Lets say I have query class, that has __call__ method, that takes 1
> argument - some text
> 
> qa = query_t( 'a' ) #returns true if 'a' in text
> qb = query_t( 'b' ) #returns true if 'b' in text
> 
> qab = qa and qb
> [...]
> qaqb = qa or qb

The above lines should read:

qab = qa & qb
...
qaqb = qa | qb

"and", "or" is not the same as the &,| operators.

> Do you like such approach? Matthias what do you think?

I don't understand what problem you want to solve. Your query class just 
does the same than my filters, but above you say that this is too 
difficult to learn...? So in what respect is query_t easier to use than 
my filters?

- Matthias -

[pygccxml-development] need help with query

[Roman Y. <rom...@gm...>]

I saw Matthias filters ( Or, Not and And ) and liked the idea. But (
there is always some 'but' :-( ).

I don't think they are useful in the form like this. People are to
lazy to learn. So instead
of composing queries using Matthias classes, they will right small
lambda function that will
do the job. May be I wrong, may be not. I propose an other
implementation. Take a look on attached file. Right now it does not
work. This is the area I need help. Can you see what I am doing wrong?

Lets say I have query class, that has __call__ method, that takes 1
argument - some text

qa =3D query_t( 'a' ) #returns true if 'a' in text
qb =3D query_t( 'b' ) #returns true if 'b' in text

qab =3D qa and qb
print 'a and b in text ac?', qab( 'ac' ) #False
print 'a and b in text ab?',qab( 'ab' ) #True

qaqb =3D qa or qb
print 'a or b in text a', qaqb('a') #True
print 'a or b in text b', qaqb('b')#True
print 'a or b in text c', qaqb('c')#False

Do you like such approach? Matthias what do you think?

Thanks for help.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Current status...

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> I implemented some stuff today that mainly deals with the selection of
>> declarations (all additions are beneath the experimental directory, but
>> I didn't commit anything yet). It's not finished but I just wanted to
>> keep you informed about what I'm currently doing.
> 
> I am a little bit surprised. I struggle with my self for 3 or more
> weeks not to write
> high level api. But this is okay. 

I thought we (Allen and I) are supposed to try out ideas and come up 
with prototype implementations (within the experimental directory)....?

> I have something like 10 hours in a
> week to work on
> py++. It would be nice to know how many hours do you have to work on
> it, so we could plan features. 

As I'm still not able to re-create the full Maya bindings as I had them 
2 or 3 weeks ago, I'm currently dedicating quite a lot of time to get 
that back.

> Can you commit your changes? I would like to analyze your code, as I
> did with Allen one. I am sure I can find some piece of code that
> should be placed in an other place :-).

Please, let's wait with moving stuff from the experimental code to the 
actual pyplusplus code base. I think it's too early to do that when it's 
still not clear whether that code is really "stable" or not. Our main 
interest is what an optimal *high level API* looks like, right? So 
currently, I think the actual implementation is not so important. Let's 
optimize that once the API is stable. Immediate changes to pyplusplus 
are only required when we want to do something in the high level API 
that is impossible to do with the current version.

The last change to pyplusplus was already like an "earthquake" that has 
shattered our APIs from which they still haven't fully recovered (using 
some sort of "dramatic" language here... ;-) And instead of spending my 
time "repairing" the API I'd rather like to try out *new* ideas and move 
the API forward. The newly introduced decl_wrappers are fine as they 
provide a common interface to do the decorations, but I think some other 
stuff was moved there too quickly.

>> - I implemented an alternative version of the
>> DeclWrapper/MultiDeclWrapper classes
>>
>> - To prevent chaos I didn't actually change anything in pypp_api but
>> implemented the new DeclWrapper in a separate module and replaced the
>> DeclWrapper in pypp_api at runtime
> 
> I hope you read my post about adding some teaching/guiding functionality to
> decl_wrappers. Did you created DeclWrapper base on my work or you wrote
> something new? 

Of course, I'm using the decl_wrappers from pyplusplus to decorate the 
declarations. Otherwise, pyplusplus wouldn't work anyway, would it? But 
as they are not the same as the DeclWrapper class I don't return the 
decl_wrappers to the user directly but instead "wrap" them (actually, 
I'd still vote for renaming the decl_wrappers into decl_decorators. I 
think this is a more precise name).
For example, the decl_wrappers don't provide the selection interface, 
which is already the main reason why they cannot be passed to the user 
directly.

> Also what is wrong with mdecl_wrapper_t class?

Well, it's not entirely clear to me how this class eases the 
implementation of the high level API...
(that's an example of a class where I'd say it was moved too quickly 
down to pyplusplus)

>> - I only dealt with the selection so far, so the decoration still works
>> as before. The heart of the selection is a single function select() that
>> takes as input the root of the pygccml declaration tree and a single
>> filter "function" and returns a list of declarations that matched the
>> filter. Conceptually, the search is always done recursively.
> 
> What is wrong with pygccxml.declarations.matcher? Even if you say that
> search is always done recursively you still can use it. Am I wrong?

I guess the find() method could be used, but this method always searches 
the entire tree whereas I have some code that can restrict the search to 
certain parts of the tree.
Also, I didn't use the matcher classes from the declarations.filters 
module because they also differ somewhat in the way they work 
(especially the regex stuff. I'm not really sure yet how this is 
supposed to be used).

> First of all I like you idea about using and\or\not for
> matchers\select function.
> Why did not you modify matchers? This is exactly how things should work.

Because they are part of pyplusplus which is your domain. I won't change 
anything "down there" without your approval. Isn't that what the 
experimental folder is for? I don't consider the experimental folder to 
be part of the official pyplusplus code base, I rather see it as a 
"common playground" where Allen and I can mess around and try out new stuff.
And if we end up with something useful then we can decide how this is 
best moved to pyplusplus. Isn't that how it was supposed to work?

>> There is a global option that controls whether queries producing no
>> results should be considered an error or not (for both cases). By
>> default, empty queries are considered an error.
> 
> Global variables? -brrr :-). Can I propose solution for this? Can you
> add all those global variables as class members? 

To which class?

> Please commit your changes. Do not forget license. People should know
> who is guilty :-).

:)

I've committed what I have so far. Note however, that this is still much 
work in progress! I still left Allen's original version intact, so to 
activate the new "DeclWrapper" I did the following in my driver script:

from pypp_api import *
# Replace with test version
import pypp_api
import declwrapper
pypp_api.DeclWrapper = declwrapper.IDecl

This "IDecl" class is based on my previous version with additions from 
Allen's DeclWrappers. Most stuff should be compatible to Allen's 
version. Here are some things that differ:

- I left out the "expose" flag in the Class, Method, etc. methods. This 
actually led to some confusion here because I didn't know why a class 
was exposed even though I wasn't exposing it. (But of course, it would 
be easy to add that feature again if somebody insists on it)

- Is a return value policy also a "call policy" in Boost.Python terms? 
(I was calling my version setPolicy() whereas Allen's version is called 
setCallPolicy(). I thought that a call policy is already a specialized 
policy but I could be wrong)

- addMethod()/wrapMethod() and everything else with custom code creators 
is not implemented yet.


- Matthias -