pygccxml-development

[pygccxml-development] Current status...

[Matthias B. <ba...@ir...>]

Hi,

I implemented some stuff today that mainly deals with the selection of 
declarations (all additions are beneath the experimental directory, but 
I didn't commit anything yet). It's not finished but I just wanted to 
keep you informed about what I'm currently doing.

What I did:

- I implemented an alternative version of the 
DeclWrapper/MultiDeclWrapper classes

- To prevent chaos I didn't actually change anything in pypp_api but 
implemented the new DeclWrapper in a separate module and replaced the 
DeclWrapper in pypp_api at runtime

- I only dealt with the selection so far, so the decoration still works 
as before. The heart of the selection is a single function select() that 
takes as input the root of the pygccml declaration tree and a single 
filter "function" and returns a list of declarations that matched the 
filter. Conceptually, the search is always done recursively. The filter 
function is actually a special object which itself can be a tree of 
filter functions (think of it as the tree representation of a boolean 
expression). The nodes of the tree are the functions AND, OR, NOT and 
the leaves are the actual filters similar to what I had before or to 
Roman's matcher objects. Filters can be concatenated using the regular 
&, | and ~ operators. This select() function is supposed to be the most 
generic selection function that all specialized selection functions can 
be based on.

- Based on the above select() function there are the query methods 
(still using the naming convention from pypp_api):

   * Class / Classes
   * Method / Methods
   * Function / Functions
   * ...etc...

The first version will always reference exactly 1 declaration whereas 
the second version may reference an arbitrary number of declarations (if 
desired the user can still add an assert_count argument).
There is a global option that controls whether queries producing no 
results should be considered an error or not (for both cases). By 
default, empty queries are considered an error.
What I still have to do is make the above queries only work on the 
direct children of the parent node by default. I think this can again be 
done by a global option as default value which can be overridden by a 
local recursive argument.

There's no special treatment of overloaded methods yet. This is still an 
unresolved issue.

I hope that this version meets most (if not all) of the requirements we 
had so far. By default, the queries are local and the "1-result" version 
guarantees that you will get exactly one declaration. This is what a new 
user will see first. Then, when he feels bold enough he can use the 
"n-result" queries and can opt to enable global searches. If desired he 
can also turn off the checks for empty results.
In any case, the decoration interface is the same, so when the beginner 
gets advanced he doesn't have to learn a new interface.


- Matthias -

Re: [pygccxml-development] Current status...

[Roman Y. <rom...@gm...>]

On 3/13/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> I implemented some stuff today that mainly deals with the selection of
> declarations (all additions are beneath the experimental directory, but
> I didn't commit anything yet). It's not finished but I just wanted to
> keep you informed about what I'm currently doing.

I am a little bit surprised. I struggle with my self for 3 or more
weeks not to write
high level api. But this is okay. I have something like 10 hours in a
week to work on
py++. It would be nice to know how many hours do you have to work on
it, so we could
plan features. Also until end of this week I should have Internet
connection at home.
It will be easier to discuss our efforts.

Can you commit your changes? I would like to analyze your code, as I
did with Allen one. I am sure I can find some piece of code that
should be placed in an other place :-).


> What I did:
>
> - I implemented an alternative version of the
> DeclWrapper/MultiDeclWrapper classes
>
> - To prevent chaos I didn't actually change anything in pypp_api but
> implemented the new DeclWrapper in a separate module and replaced the
> DeclWrapper in pypp_api at runtime

I hope you read my post about adding some teaching/guiding functionality to
decl_wrappers. Did you created DeclWrapper base on my work or you wrote
something new? Also what is wrong with mdecl_wrapper_t class?

> - I only dealt with the selection so far, so the decoration still works
> as before. The heart of the selection is a single function select() that
> takes as input the root of the pygccml declaration tree and a single
> filter "function" and returns a list of declarations that matched the
> filter. Conceptually, the search is always done recursively.

What is wrong with pygccxml.declarations.matcher? Even if you say that
search is always done recursively you still can use it. Am I wrong?

>The filter
> function is actually a special object which itself can be a tree of
> filter functions (think of it as the tree representation of a boolean
> expression). The nodes of the tree are the functions AND, OR, NOT and
> the leaves are the actual filters similar to what I had before or to
> Roman's matcher objects. Filters can be concatenated using the regular
> &, | and ~ operators. This select() function is supposed to be the most
> generic selection function that all specialized selection functions can
> be based on.

First of all I like you idea about using and\or\not for
matchers\select function.
Why did not you modify matchers? This is exactly how things should work.
I thought that the user will use lambda. But this is a great idea,
lets port that piece of
code to pygccxml.declaration.matchers object. I think I have an idea
how to do it
in single place. If you want we can discuss this in separate thread.

> - Based on the above select() function there are the query methods
> (still using the naming convention from pypp_api):
>
>    * Class / Classes
>    * Method / Methods
>    * Function / Functions
>    * ...etc...

Good.

> The first version will always reference exactly 1 declaration whereas
> the second version may reference an arbitrary number of declarations (if
> desired the user can still add an assert_count argument).

Good.

> There is a global option that controls whether queries producing no
> results should be considered an error or not (for both cases). By
> default, empty queries are considered an error.

Global variables? -brrr :-). Can I propose solution for this? Can you
add all those
global variables as class members? No global variables + all instances will
behave the same.

> What I still have to do is make the above queries only work on the
> direct children of the parent node by default. I think this can again be
> done by a global option as default value which can be overridden by a
> local recursive argument.

If you would use my decl_wrapper class hierarchy you could put those
select functions on scope/namespace/class wrappers.

> There's no special treatment of overloaded methods yet. This is still an
> unresolved issue.

This is okay, We need something to comment/try.

> I hope that this version meets most (if not all) of the requirements we
> had so far. By default, the queries are local and the "1-result" version
> guarantees that you will get exactly one declaration. This is what a new
> user will see first. Then, when he feels bold enough he can use the
> "n-result" queries and can opt to enable global searches. If desired he
> can also turn off the checks for empty results.
> In any case, the decoration interface is the same, so when the beginner
> gets advanced he doesn't have to learn a new interface.
>
> - Matthias -
>

Please commit your changes. Do not forget license. People should know
who is guilty :-).
I will try it with boost.date_time library.

Thanks

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Current status...

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> I implemented some stuff today that mainly deals with the selection of
>> declarations (all additions are beneath the experimental directory, but
>> I didn't commit anything yet). It's not finished but I just wanted to
>> keep you informed about what I'm currently doing.
> 
> I am a little bit surprised. I struggle with my self for 3 or more
> weeks not to write
> high level api. But this is okay. 

I thought we (Allen and I) are supposed to try out ideas and come up 
with prototype implementations (within the experimental directory)....?

> I have something like 10 hours in a
> week to work on
> py++. It would be nice to know how many hours do you have to work on
> it, so we could plan features. 

As I'm still not able to re-create the full Maya bindings as I had them 
2 or 3 weeks ago, I'm currently dedicating quite a lot of time to get 
that back.

> Can you commit your changes? I would like to analyze your code, as I
> did with Allen one. I am sure I can find some piece of code that
> should be placed in an other place :-).

Please, let's wait with moving stuff from the experimental code to the 
actual pyplusplus code base. I think it's too early to do that when it's 
still not clear whether that code is really "stable" or not. Our main 
interest is what an optimal *high level API* looks like, right? So 
currently, I think the actual implementation is not so important. Let's 
optimize that once the API is stable. Immediate changes to pyplusplus 
are only required when we want to do something in the high level API 
that is impossible to do with the current version.

The last change to pyplusplus was already like an "earthquake" that has 
shattered our APIs from which they still haven't fully recovered (using 
some sort of "dramatic" language here... ;-) And instead of spending my 
time "repairing" the API I'd rather like to try out *new* ideas and move 
the API forward. The newly introduced decl_wrappers are fine as they 
provide a common interface to do the decorations, but I think some other 
stuff was moved there too quickly.

>> - I implemented an alternative version of the
>> DeclWrapper/MultiDeclWrapper classes
>>
>> - To prevent chaos I didn't actually change anything in pypp_api but
>> implemented the new DeclWrapper in a separate module and replaced the
>> DeclWrapper in pypp_api at runtime
> 
> I hope you read my post about adding some teaching/guiding functionality to
> decl_wrappers. Did you created DeclWrapper base on my work or you wrote
> something new? 

Of course, I'm using the decl_wrappers from pyplusplus to decorate the 
declarations. Otherwise, pyplusplus wouldn't work anyway, would it? But 
as they are not the same as the DeclWrapper class I don't return the 
decl_wrappers to the user directly but instead "wrap" them (actually, 
I'd still vote for renaming the decl_wrappers into decl_decorators. I 
think this is a more precise name).
For example, the decl_wrappers don't provide the selection interface, 
which is already the main reason why they cannot be passed to the user 
directly.

> Also what is wrong with mdecl_wrapper_t class?

Well, it's not entirely clear to me how this class eases the 
implementation of the high level API...
(that's an example of a class where I'd say it was moved too quickly 
down to pyplusplus)

>> - I only dealt with the selection so far, so the decoration still works
>> as before. The heart of the selection is a single function select() that
>> takes as input the root of the pygccml declaration tree and a single
>> filter "function" and returns a list of declarations that matched the
>> filter. Conceptually, the search is always done recursively.
> 
> What is wrong with pygccxml.declarations.matcher? Even if you say that
> search is always done recursively you still can use it. Am I wrong?

I guess the find() method could be used, but this method always searches 
the entire tree whereas I have some code that can restrict the search to 
certain parts of the tree.
Also, I didn't use the matcher classes from the declarations.filters 
module because they also differ somewhat in the way they work 
(especially the regex stuff. I'm not really sure yet how this is 
supposed to be used).

> First of all I like you idea about using and\or\not for
> matchers\select function.
> Why did not you modify matchers? This is exactly how things should work.

Because they are part of pyplusplus which is your domain. I won't change 
anything "down there" without your approval. Isn't that what the 
experimental folder is for? I don't consider the experimental folder to 
be part of the official pyplusplus code base, I rather see it as a 
"common playground" where Allen and I can mess around and try out new stuff.
And if we end up with something useful then we can decide how this is 
best moved to pyplusplus. Isn't that how it was supposed to work?

>> There is a global option that controls whether queries producing no
>> results should be considered an error or not (for both cases). By
>> default, empty queries are considered an error.
> 
> Global variables? -brrr :-). Can I propose solution for this? Can you
> add all those global variables as class members? 

To which class?

> Please commit your changes. Do not forget license. People should know
> who is guilty :-).

:)

I've committed what I have so far. Note however, that this is still much 
work in progress! I still left Allen's original version intact, so to 
activate the new "DeclWrapper" I did the following in my driver script:

from pypp_api import *
# Replace with test version
import pypp_api
import declwrapper
pypp_api.DeclWrapper = declwrapper.IDecl

This "IDecl" class is based on my previous version with additions from 
Allen's DeclWrappers. Most stuff should be compatible to Allen's 
version. Here are some things that differ:

- I left out the "expose" flag in the Class, Method, etc. methods. This 
actually led to some confusion here because I didn't know why a class 
was exposed even though I wasn't exposing it. (But of course, it would 
be easy to add that feature again if somebody insists on it)

- Is a return value policy also a "call policy" in Boost.Python terms? 
(I was calling my version setPolicy() whereas Allen's version is called 
setCallPolicy(). I thought that a call policy is already a specialized 
policy but I could be wrong)

- addMethod()/wrapMethod() and everything else with custom code creators 
is not implemented yet.


- Matthias -

Re: [pygccxml-development] Current status...

[Allen B. <al...@vr...>]

Matthias Baas wrote:

> Hi,
>
> I implemented some stuff today that mainly deals with the selection of 
> declarations (all additions are beneath the experimental directory, 
> but I didn't commit anything yet). It's not finished but I just wanted 
> to keep you informed about what I'm currently doing.

Great.  I was feeling guilty that I got tied up with work this weekend 
and hadn't been able to get to this.  I am glad someone is making 
progress. :)

>
> What I did:
>
> - I implemented an alternative version of the 
> DeclWrapper/MultiDeclWrapper classes
>
> - To prevent chaos I didn't actually change anything in pypp_api but 
> implemented the new DeclWrapper in a separate module and replaced the 
> DeclWrapper in pypp_api at runtime
>
If you go ahead and check this in I don't think there will be chaos.  I 
think in fact that it would help us to better understand what we need to do.

On question I have related to this implementation though:

 From our discussion the other day I think it sounds like we don't need 
a DeclWrapper/MultiDeclWrapper class but instead should maybe be 
modifying/extending/renaming/etc the decl_wrapper that are being created 
by the pyplusplus code creator now.

I think this may be the direction to move because:

- It seems like this is what Roman wants
- The interface for everything but the query methods in *DeclWrapper 
will just call straight through to the pyplusplus _wrappers
- Using the pyplusplus wrappers could give us better type safety in the 
form of errors when the user calls something that doesn't make sense for 
the currently wrapped decl type.

Doing this would move a great deal of the high-level API into the 
pyplusplus decl_wrappers but it just seems like a good move right now.

Thoughts?

> - I only dealt with the selection so far, so the decoration still 
> works as before. The heart of the selection is a single function 
> select() that takes as input the root of the pygccml declaration tree 
> and a single filter "function" and returns a list of declarations that 
> matched the filter. Conceptually, the search is always done 
> recursively. The filter function is actually a special object which 
> itself can be a tree of filter functions (think of it as the tree 
> representation of a boolean expression). The nodes of the tree are the 
> functions AND, OR, NOT and the leaves are the actual filters similar 
> to what I had before or to Roman's matcher objects. Filters can be 
> concatenated using the regular &, | and ~ operators. This select() 
> function is supposed to be the most generic selection function that 
> all specialized selection functions can be based on.

Sounds very interesting.  I can't wait to see it. :)

>
> - Based on the above select() function there are the query methods 
> (still using the naming convention from pypp_api):
>
>   * Class / Classes
>   * Method / Methods
>   * Function / Functions
>   * ...etc...
>
> The first version will always reference exactly 1 declaration whereas 
> the second version may reference an arbitrary number of declarations 
> (if desired the user can still add an assert_count argument).
> There is a global option that controls whether queries producing no 
> results should be considered an error or not (for both cases). By 
> default, empty queries are considered an error.
> What I still have to do is make the above queries only work on the 
> direct children of the parent node by default. I think this can again 
> be done by a global option as default value which can be overridden by 
> a local recursive argument.

Sounds good.  Once this is all in there I think we can really start 
pushing the API and finding areas that we want to extend further.

>
> There's no special treatment of overloaded methods yet. This is still 
> an unresolved issue.
>
> I hope that this version meets most (if not all) of the requirements 
> we had so far. By default, the queries are local and the "1-result" 
> version guarantees that you will get exactly one declaration. This is 
> what a new user will see first. Then, when he feels bold enough he can 
> use the "n-result" queries and can opt to enable global searches. If 
> desired he can also turn off the checks for empty results.
> In any case, the decoration interface is the same, so when the 
> beginner gets advanced he doesn't have to learn a new interface.
>
Sounds great.  Thanks for working so hard on this.

-Allen

>
> - Matthias -
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> pygccxml-development mailing list
> pyg...@li...
> https://lists.sourceforge.net/lists/listinfo/pygccxml-development
>

Re: [pygccxml-development] Current status...

[Roman Y. <rom...@gm...>]

Matthias, it seems that you are watching too match TV :-). You don't
have to keep me/us under presure.

Please commit your changes, as they are. I want to contribute too :-)

Thanks