[pygccxml-development] Re: Patches and complete pyste replacement prototype for pyplusplus

[Matthias B. <ba...@ir...>]

Allen Bierbaum wrote:
[global "abbreviation" functions]
> Agreed.  How about we get the object-based interface finalized first
> and then we come back and introduce an optional way to use global
> methods later.

Agreed.

>> code in each method). I also didn't base the interface on the number of
>> contained declarations because I thought whenever I call a method on a
>> MultiDecl object I could just as well iterate over the contained
>> declarations and call that method on each of them individually. And
>> that's basically what I'm doing, relieving the user from having to write
>> that loop himself.
> 
> There were some methods that I found I didn't think this was a good
> thing to do.  For example, add_method and rename.  I didn't think it
> made much sense to rename multiple declarations to the same name 

How do you rename a set of overloaded methods?

> or to add the same method to multiple found classes in a namespace.

I admit that this is probably a rare case (not so much because it might 
not be useful but rather because you could only specify one C/C++ 
function that implements the method whereas you would have to pass 
different functions (because the type of the "self" argument differs)).

In the Maya SDK there are a couple of related classes that basically 
have the same interface (e.g. vector, float vector, point, float point, 
color, then the same thing for array versions etc). When decorating 
those classes I can treat every class of such a group the same and apply 
the same operations. This is where being able to select stuff from 
several classes at once can be quite handy.

>>> ns = mod.Namespace("test_ns")
>>> class1 = ns.Class("class1")
>>> class1_method1 = class1.Method("method1")
>>> class2 = ns.Class("class2")
>>> class2_method1 = class2.Method("method1")
>>>
>>> In Matthias's API I believe you could do something where you could ask
>>> for all methods named "method1" across the entire decl tree.
>> Right, you *could*, but you don't *have to*. The above code would work
>> in my version just as well with the same semantics, i.e. class1_method
>> would only contain the method of class1 and not the one from class2
>> because you called Method() on a previous selection of exactly one
>> class. Only if you would call Method() on the main namespace (which by
>> default also contains all children nodes) or on a class selection that
>> references both classes would you get the "method1" methods from both
>> classes.
> 
> This "all children nodes" or include children interface in your API
> was one of the biggest things that confused me.  Can you describe how
> it is meant to work 

I was regarding the decoration process as a 2-step process: First, a 
selection is done which is based on matching some filters. The result is 
a set of declarations. Then, a decoration operation is applied to this 
selection which means the operation is done on each individual item.

The result of doing a selection was an IDecl object which can be seen as 
a container for declarations. But what declarations are actually 
"contained"? Strictly applying the above scheme would mean that only 
those declarations are considered "contained" that matched the filter 
functions (by the way, I always regarded a declaration as sort of one 
"atomic" thing and not as a node in a tree that may have children). This 
would allow something like

Foo = Class("Foo")
Foo.rename("Spam")

which would only rename the class, and not its methods. But I couldn't 
go on and write

method1 = Foo.Method("method1")

because I wanted a selection on a previous selection only to return 
elements from that previous selection and in the above case Foo only 
contains 1 class declaration, so any further queries for methods would 
always return an empty set. That's when I thought that the user should 
have the possibility to specify whether he wants to include children of 
a matched declaration in the resulting set as well (that's the 
include_children flag) even though those children might not match the 
filter functions. So far, the idea was to consider everything as being 
"contained" in the set if it either matched the filter functions 
directly or if it had a parent that matched the filter functions (if 
include_children was set to True). But this was problematic as well 
because then the lines

Foo = Class("Foo")
Foo.rename("Spam")

would assign the new name not only to the class but to all of its 
methods as well.
To solve this, I made a distinction between "contained" declarations and 
"matched" declarations. The "contained" declarations are all that are 
considered to be part of the set whereas the "matched" declarations are 
only those that matched the filter functions directly (the "matched" set 
is a subset of the "contained" set and when include_children is False 
both sets are always identical). In the above case, the class 
declaration for "Foo" would be the only "matched" declaration whereas 
all the methods are still "contained". By default, I applied all 
declaration operations (and refining filter operations) to the set of 
"contained" methods, but some operations, like rename(), are only 
applied to the matched methods because applying them to children as well 
will hardly ever make any sense. Now the above lines actually did what I 
wanted them to do:

Foo = Class("Foo")
Foo.rename("Spam")  # only renames the class
method1 = Foo.Method("method1") # but the methods are still there to be 
selected


I admit that it is somewhat complicated to explain but once you got it, 
I found it consistent and straightforward to predict the results of 
doing selection/decoration operations. And in practice it turned out 
that I never had to explicitly specify the flag anyway as the default 
always did the "right" thing (because something like rename() is already 
restricted to "matched" declarations).

I'm not arguing that this way of seeing it is superior to the 
"recursive" flag, it's just one way I was trying out and it did work for 
me rather well. But I admit that the "matched" vs "contained" thing and 
operations like rename() being slightly different than other operations 
adds to the complexity. So I think it's still worthwhile to try other 
approaches as well.

> or perhaps another similar question: do you think
> that using a recursive flag as suggested below could allow the same
> functionality without having to say include_children?  Instead we
> would just be telling the search methods to recursive  deeply into all
> children which I think gives the same functionality if I understand
> your goal here.

I suppose it can be made so that it's equivalent to what I described 
above (but I'm not sure). It depends on the details:

- Is the result of a selection conceptually a flat list (as it was in my 
case) or is it a tree or a list of trees?

- When there's a "match" at some point (i.e. a particular declaration 
matches the filters) and recursive is set to True, will all children 
make it into the selection or only those that also match the filters?

- On which declarations is a decoration operation (rename, expose) 
applied to? (i.e. which declarations are considered "contained"?)


And once these questions are answered I will go on with the following 
question: Will the following sequence of statements have the effects as 
stated in the comments? If not, what would I have to change?

   # Obtain a "handle" to a class
   Foo = Class("Foo")
   # Expose the entire class (assuming that by default everything
   # is ignored, otherwise replace expose() by ignore())
   Foo.expose()
   # Rename the class
   Foo.rename("Spam")
   # Select a method for further processing
   method1 = Foo.Method("method1")


(That's how I proceeded in creating my version of the API. I already had 
in mind how I wanted an "optimal" decoration sequence to look like and 
then I was trying to come up with something that allowed me write it 
like that and that also had a consistent set of rules that make it easy 
for the user to predict the outcome of his code. Of course, you might 
argue that the above sequence is not the optimal way of expressing these 
things. Then we should take a step back and discuss how the sequence can 
be improved and what the "optimality criterion" actually is.)


By the way, another thing to think about when dealing with the recursive 
flag is if it would be better to add the flag to the decoration 
operations instead of the selection operations...


> Okay.  I have made some modifications to my version.  I would like to
> get it into a public VCS as soon as possible (see below) so we can
> refine further.

Right, could you please check it in into the "pyplusplus/experimental" 
folder that Roman has created?


- Matthias -