pygccxml-development

[pygccxml-development] Re: Patches and complete pyste replacement prototype for pyplusplus

[Roman Y. <rom...@gm...>]

On 3/2/06, Matthias Baas <ba...@ir...> wrote:
> How do you rename a set of overloaded methods?

I think, I understand what are you talking about. I think that this is
a convenience
functionality and could be implemented as standalone function

def rename_overloaded( function_or_functions, new_name ):
...

I prefer not to insert such functionality to the first draft of interface.
I think it will pollute it and we will not be able to evaluate the interfac=
e.

I do think that in future versions we will\should add such functionality.

>
> > or to add the same method to multiple found classes in a namespace.

I don't understand this case at all.

>
> In the Maya SDK there are a couple of related classes that basically
> have the same interface (e.g. vector, float vector, point, float point,
> color, then the same thing for array versions etc). When decorating
> those classes I can treat every class of such a group the same and apply
> the same operations. This is where being able to select stuff from
> several classes at once can be quite handy.

May I give you a small advice? You can combine between power of pyplusplus =
and
power of C++. I think that using creating single template for every
group is better solution.
But I could be wrong.

> I'm not arguing that this way of seeing it is superior to the
> "recursive" flag, it's just one way I was trying out and it did work for
> me rather well. But I admit that the "matched" vs "contained" thing and
> operations like rename() being slightly different than other operations
> adds to the complexity. So I think it's still worthwhile to try other
> approaches as well.

Well, I read it and I think I've got the idea. Wiki is the way do discuss t=
his.
I will try to add new entry/ edit existing one that explains the functional=
ity.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Roman Y. <rom...@gm...>]

On 3/2/06, Matthias Baas <ba...@ir...> wrote:
> However, apart from that I was implementing a new cache class as
> mentioned in another mail. It's not yet finished because of that "dict
> vs list" bug in the handling of the "included_files" parameter. This
> first needs to be resolved before I can finish the class (so is
> source_reader_t.__parse_gccxml_created_file() supposed to return the
> files as a dictionary or as a list?)

May be I missed something, but what cache do you implement?
Any way the answer: list, I already fixed this bug.

> >> currently breaks my "driver" script for creating my bindings (that's
> >> what I meant the other day on the c++-sig list when I was against
> >> exposing the entire internal API as public API because the chances a
> >> future pyplusplus version will break a user's script are much higher
> >> then. And here we are... ;)
> >
> > :-(. I tested my changes. They should work as is without any changes
> > from user code.
> > It seems that I introduce a bug.
>
> It seems that for your tests you're using pyplusplus right from the
> source tree (instead of building/installing them using distutils), is
> this correct? You added some new sub packages to pyplusplus but forgot
> to add them to the setup script (decl_wrappers, module_builder). So
> anybody who's installing the package will miss those sub package and
> then, of course, pyplusplus will raise an exception right at import time
> (btw, obviously the cyclic thing I mentioned in the mailing list was not
> the source of the exception)

1. I think I found and kill the bug. When I switched to decl_wrapper's
classes I
started to return wrong "alias"es, so this bug has been fixed

2. I updated setup file before release, when I have something like
"feature freeze" period.
    I suppose, that every one who use CVS will be able to use it
without setup. I could be
    wrong, but right now I prefer to concentrate my attention on something =
else

> And the things that broke my script were:
>
> 1. call_policies.py was moved to a different location (so my import and
> usage was failing).

I can not change the history, I learned the lesson. Next time we will need =
to
do some changes I hope you will be not surprised. Sorry.

> 2. The recursive flag was removed from the module_creator.creator_t()
> constructor which broke my instantiation of that class.

:-(

> I fixed those in my script so it wasn't a big deal, especially when
> considering that pyplusplus is still under devlopment (but it sort of
> confirmed my previous concerns... ;)

Guilty :-)

> But even after fixing all of the above, there still seems to be a
> genuine bug which still prevents me from generating the bindings. When
> executing my script, I get the following traceback:
>
>    File "pyplusplus/module_creator/creator.py", line 54, in __init__
>      self.__decls =3D self._filter_decls( self._reorder_decls(
> self._prepare_decls( decls ) ) )
>    File "pyplusplus/module_creator/creator.py", line 68, in _prepare_decl=
s
>      decls =3D filter( lambda x: not x.ignore, decls )
>    File "pyplusplus/module_creator/creator.py", line 68, in <lambda>
>      decls =3D filter( lambda x: not x.ignore, decls )
> AttributeError: 'class_t' object has no attribute 'ignore'

I hope that until end of this week we will have some stable interface.
At least I
will try to work hard to do it.

> > Also I would like to explain them:
> > We decided on next concepts, correct me if I wrong:
> >
> > declarations tree is used to configure code creators
>
> Exactly.

This change had broken your scripts.

> >> And while I'm all for consistent coding guidelines I currently find it
> >> quite confusing to have several different classes with the same name (=
I
> >> spotted at least three different classes called "class_t"). For me, th=
is
> >> doesn't really help understanding the code.
> >
> > I know, this should be fixed. Can you propose naming scheme for
> > pyplusplus code creators
> > and declarations wrappers/decorators ?
>
> Well, you could fill in some extra words (such as class_wrapper_t) but
> as Python already organizes code in a hierarchy I guess it would be a
> better idea to use decl_wrapper.class_t explicitly instead of importing
> the classes into the namespace of another module. Then it's also clear
> to the reader which class is being referred to.

So, basically you would like to stay with name I proposed,  but user
is forced to
use "fully qualified" names, am I right?


>
> > Matthias, I think you should stop to use latest CVS in production envir=
onment,
> > until we stabilize interface. That's said.
>
> That's ok. I mean, I do want to be able to take advantage of the latest
> features, so it's ok that this might lead to some inconveniences first.
> And the sooner someone notices bugs, the sooner they can be fixed. :)
>
> - Matthias -
>


--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Updated status

[Allen B. <al...@vr...>]

A apologize in advance, this e-mail is going to be a little random 
because it is based on my notes from this afternoon.  I started trying 
to move the pypp_api interface over the the new codebase and these are 
the things I have run into right away.

- Calling parser changed a bit.

There seems to be a new interface to calling the project parser.  I 
think I just need to add a decl wrapper factory but I am not for sure.  
Roman?

- How am I supposed to use the decl_wrappers?

I am trying to parse with the project reader and then I was expecting 
that the returned decls would actually be the wrappers/decorators for 
the decls.  Is this how it is supposed to work?

- Code complexity: The new decorators does look like a LOT of additional 
code for what could be done with some grafted flags.  Not really a 
question, more of a comment.  I really think the codebase could be 
simplified a lot.  For me at least all the property usage seems like a 
lot of extra code to read through and maintain for very little payoff.

- What is module_builder_t for?  Do I have to use it?  It looks like 
Roman's stab at the higher-level interface but what do I need to copy 
from there into pypp_api.

- Why did call policies move to the wrapper module?  Those still seem 
like code_creators.

- Decl wrappers don't seem to support everything I needed/added with the 
decorators.

Specifically, how can I add custom creators?  In my flag-based version I 
could add a list of code creators that would automatically be added on 
when the creator class build the code creators for a decl. 

- What are all the options that have been added to the wrappers? (see 
documentation below)

- I think Logger should probably use info() for most of the current 
output.  Why put time and level name in log output?

My original goal with this output was to have verbose output for users 
not to provide a debug log to a log file on errors. Any objections to 
making the output look cleaner by making the info() filter just output 
the text instead of all the header information?

- Comments: We *really* need comments and documentation for the classes 
and methods in pygccxml and pyplusplus.  I think Matthias and I could 
help out by pointing to areas of the code and saying in our very nicest 
voice "Roman, please please document this".

Toward this goal (and following up on Matthias's comments from last 
week), I started looking at what it would take to generate documentation 
using epydoc and restructuredtext.  I added a script in 
"experimental/docs/generate_all_docs.py" that will call epydoc with what 
I think are the correct parameters to generate html documentation.  I 
also started converting pypp_api over to use restructured text for 
documentation. The only problem is that it looks like the tags/fields I 
am using are not being recognized correctly (ex: params, args, etc).  
According to the docs I am doing things correctly but it is just not 
working.  If either of you have experience with this tool please take a 
look and let me know if I am doing something wrong.

Does this seem like a good direction for documentation?  I know we can 
document classes and methods using this, do we want to document the 
usage of the modules like this as well?  (it is supported but do we want 
that type of documentation in the code?)

That is about all I have for now.  Until I find out more about the 
decl_wrapper's and their use I am at a stand still. :(

Thanks,
Allen

Re: [pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> Those decl wrappers should be used as high level interface for user to configure
> code creators. It does not really make the different how it implemented inside.
> In our versions it is possible to say
> 
> mb = module_builder_t(...)
> mb.namespace( ... ).ignore()
> 
> module_builder_t I created it is just proof of concept: I wanted to
> see that what you
> ( you and Allen ), could be implemented and that it really simplifies
> user interface.

Just out of curiosity, have there been any problems in getting our API 
versions to run? (as the above is just what Allen's version provides, 
just with different names) I noticed myself only later, that my version 
depends on a directory called "tmp" being in the current directory....

> In short I want you and Allen concentrate your efforts on user interface:
> class module_builder_t and multi decl wrapper and I will serve you as programmer
> who provide you all functionality you need. Do you agree?

ok

However, apart from that I was implementing a new cache class as 
mentioned in another mail. It's not yet finished because of that "dict 
vs list" bug in the handling of the "included_files" parameter. This 
first needs to be resolved before I can finish the class (so is 
source_reader_t.__parse_gccxml_created_file() supposed to return the 
files as a dictionary or as a list?)

>> currently breaks my "driver" script for creating my bindings (that's
>> what I meant the other day on the c++-sig list when I was against
>> exposing the entire internal API as public API because the chances a
>> future pyplusplus version will break a user's script are much higher
>> then. And here we are... ;)
> 
> :-(. I tested my changes. They should work as is without any changes
> from user code.
> It seems that I introduce a bug.

It seems that for your tests you're using pyplusplus right from the 
source tree (instead of building/installing them using distutils), is 
this correct? You added some new sub packages to pyplusplus but forgot 
to add them to the setup script (decl_wrappers, module_builder). So 
anybody who's installing the package will miss those sub package and 
then, of course, pyplusplus will raise an exception right at import time 
(btw, obviously the cyclic thing I mentioned in the mailing list was not 
the source of the exception)

And the things that broke my script were:

1. call_policies.py was moved to a different location (so my import and 
usage was failing).

2. The recursive flag was removed from the module_creator.creator_t() 
constructor which broke my instantiation of that class.

I fixed those in my script so it wasn't a big deal, especially when 
considering that pyplusplus is still under devlopment (but it sort of 
confirmed my previous concerns... ;)

But even after fixing all of the above, there still seems to be a 
genuine bug which still prevents me from generating the bindings. When 
executing my script, I get the following traceback:

   File "pyplusplus/module_creator/creator.py", line 54, in __init__
     self.__decls = self._filter_decls( self._reorder_decls( 
self._prepare_decls( decls ) ) )
   File "pyplusplus/module_creator/creator.py", line 68, in _prepare_decls
     decls = filter( lambda x: not x.ignore, decls )
   File "pyplusplus/module_creator/creator.py", line 68, in <lambda>
     decls = filter( lambda x: not x.ignore, decls )
AttributeError: 'class_t' object has no attribute 'ignore'

> Also I would like to explain them:
> We decided on next concepts, correct me if I wrong:
> 
> declarations tree is used to configure code creators

Exactly.

> code creators tree will be used to configure file writers

Probably. Sort of. I'm not sure. I haven't put that much thought into 
this area yet as I have a configuration that actually works for now. 
It's not ideal, but as it does work, improving it is only of second 
priority for me right now.

>> And while I'm all for consistent coding guidelines I currently find it
>> quite confusing to have several different classes with the same name (I
>> spotted at least three different classes called "class_t"). For me, this
>> doesn't really help understanding the code.
> 
> I know, this should be fixed. Can you propose naming scheme for
> pyplusplus code creators
> and declarations wrappers/decorators ?

Well, you could fill in some extra words (such as class_wrapper_t) but 
as Python already organizes code in a hierarchy I guess it would be a 
better idea to use decl_wrapper.class_t explicitly instead of importing 
the classes into the namespace of another module. Then it's also clear 
to the reader which class is being referred to.

> [...] 
> 
> class regex_matcher_t:
>     def __init__( self, regex, function ):
>         self.regex = re.compile( regex )
>         self.function = function
> 
>     def __call__( self, decl ):
>         text = self.function( decl )
>         return bool( self.regex.match( text ) )
> 
> If user want to match not only name but something else by regular expression,
> he should not derive from the class, but rather to write function that
> takes as argument
> declaration and returns some text. I found this approach easier then
> starting to find out class hierarchy.

I cannot comment on that right now, first I need to have a closer look 
on how to use that stuff...

> Matthias, I think you should stop to use latest CVS in production environment,
> until we stabilize interface. That's said.

That's ok. I mean, I do want to be able to take advantage of the latest 
features, so it's ok that this might lead to some inconveniences first. 
And the sooner someone notices bugs, the sooner they can be fixed. :)

- Matthias -

[pygccxml-development] Re: Patches and complete pyste replacement prototype for pyplusplus

[Matthias B. <ba...@ir...>]

Allen Bierbaum wrote:
[global "abbreviation" functions]
> Agreed.  How about we get the object-based interface finalized first
> and then we come back and introduce an optional way to use global
> methods later.

Agreed.

>> code in each method). I also didn't base the interface on the number of
>> contained declarations because I thought whenever I call a method on a
>> MultiDecl object I could just as well iterate over the contained
>> declarations and call that method on each of them individually. And
>> that's basically what I'm doing, relieving the user from having to write
>> that loop himself.
> 
> There were some methods that I found I didn't think this was a good
> thing to do.  For example, add_method and rename.  I didn't think it
> made much sense to rename multiple declarations to the same name 

How do you rename a set of overloaded methods?

> or to add the same method to multiple found classes in a namespace.

I admit that this is probably a rare case (not so much because it might 
not be useful but rather because you could only specify one C/C++ 
function that implements the method whereas you would have to pass 
different functions (because the type of the "self" argument differs)).

In the Maya SDK there are a couple of related classes that basically 
have the same interface (e.g. vector, float vector, point, float point, 
color, then the same thing for array versions etc). When decorating 
those classes I can treat every class of such a group the same and apply 
the same operations. This is where being able to select stuff from 
several classes at once can be quite handy.

>>> ns = mod.Namespace("test_ns")
>>> class1 = ns.Class("class1")
>>> class1_method1 = class1.Method("method1")
>>> class2 = ns.Class("class2")
>>> class2_method1 = class2.Method("method1")
>>>
>>> In Matthias's API I believe you could do something where you could ask
>>> for all methods named "method1" across the entire decl tree.
>> Right, you *could*, but you don't *have to*. The above code would work
>> in my version just as well with the same semantics, i.e. class1_method
>> would only contain the method of class1 and not the one from class2
>> because you called Method() on a previous selection of exactly one
>> class. Only if you would call Method() on the main namespace (which by
>> default also contains all children nodes) or on a class selection that
>> references both classes would you get the "method1" methods from both
>> classes.
> 
> This "all children nodes" or include children interface in your API
> was one of the biggest things that confused me.  Can you describe how
> it is meant to work 

I was regarding the decoration process as a 2-step process: First, a 
selection is done which is based on matching some filters. The result is 
a set of declarations. Then, a decoration operation is applied to this 
selection which means the operation is done on each individual item.

The result of doing a selection was an IDecl object which can be seen as 
a container for declarations. But what declarations are actually 
"contained"? Strictly applying the above scheme would mean that only 
those declarations are considered "contained" that matched the filter 
functions (by the way, I always regarded a declaration as sort of one 
"atomic" thing and not as a node in a tree that may have children). This 
would allow something like

Foo = Class("Foo")
Foo.rename("Spam")

which would only rename the class, and not its methods. But I couldn't 
go on and write

method1 = Foo.Method("method1")

because I wanted a selection on a previous selection only to return 
elements from that previous selection and in the above case Foo only 
contains 1 class declaration, so any further queries for methods would 
always return an empty set. That's when I thought that the user should 
have the possibility to specify whether he wants to include children of 
a matched declaration in the resulting set as well (that's the 
include_children flag) even though those children might not match the 
filter functions. So far, the idea was to consider everything as being 
"contained" in the set if it either matched the filter functions 
directly or if it had a parent that matched the filter functions (if 
include_children was set to True). But this was problematic as well 
because then the lines

Foo = Class("Foo")
Foo.rename("Spam")

would assign the new name not only to the class but to all of its 
methods as well.
To solve this, I made a distinction between "contained" declarations and 
"matched" declarations. The "contained" declarations are all that are 
considered to be part of the set whereas the "matched" declarations are 
only those that matched the filter functions directly (the "matched" set 
is a subset of the "contained" set and when include_children is False 
both sets are always identical). In the above case, the class 
declaration for "Foo" would be the only "matched" declaration whereas 
all the methods are still "contained". By default, I applied all 
declaration operations (and refining filter operations) to the set of 
"contained" methods, but some operations, like rename(), are only 
applied to the matched methods because applying them to children as well 
will hardly ever make any sense. Now the above lines actually did what I 
wanted them to do:

Foo = Class("Foo")
Foo.rename("Spam")  # only renames the class
method1 = Foo.Method("method1") # but the methods are still there to be 
selected


I admit that it is somewhat complicated to explain but once you got it, 
I found it consistent and straightforward to predict the results of 
doing selection/decoration operations. And in practice it turned out 
that I never had to explicitly specify the flag anyway as the default 
always did the "right" thing (because something like rename() is already 
restricted to "matched" declarations).

I'm not arguing that this way of seeing it is superior to the 
"recursive" flag, it's just one way I was trying out and it did work for 
me rather well. But I admit that the "matched" vs "contained" thing and 
operations like rename() being slightly different than other operations 
adds to the complexity. So I think it's still worthwhile to try other 
approaches as well.

> or perhaps another similar question: do you think
> that using a recursive flag as suggested below could allow the same
> functionality without having to say include_children?  Instead we
> would just be telling the search methods to recursive  deeply into all
> children which I think gives the same functionality if I understand
> your goal here.

I suppose it can be made so that it's equivalent to what I described 
above (but I'm not sure). It depends on the details:

- Is the result of a selection conceptually a flat list (as it was in my 
case) or is it a tree or a list of trees?

- When there's a "match" at some point (i.e. a particular declaration 
matches the filters) and recursive is set to True, will all children 
make it into the selection or only those that also match the filters?

- On which declarations is a decoration operation (rename, expose) 
applied to? (i.e. which declarations are considered "contained"?)


And once these questions are answered I will go on with the following 
question: Will the following sequence of statements have the effects as 
stated in the comments? If not, what would I have to change?

   # Obtain a "handle" to a class
   Foo = Class("Foo")
   # Expose the entire class (assuming that by default everything
   # is ignored, otherwise replace expose() by ignore())
   Foo.expose()
   # Rename the class
   Foo.rename("Spam")
   # Select a method for further processing
   method1 = Foo.Method("method1")


(That's how I proceeded in creating my version of the API. I already had 
in mind how I wanted an "optimal" decoration sequence to look like and 
then I was trying to come up with something that allowed me write it 
like that and that also had a consistent set of rules that make it easy 
for the user to predict the outcome of his code. Of course, you might 
argue that the above sequence is not the optimal way of expressing these 
things. Then we should take a step back and discuss how the sequence can 
be improved and what the "optimality criterion" actually is.)


By the way, another thing to think about when dealing with the recursive 
flag is if it would be better to add the flag to the decoration 
operations instead of the selection operations...


> Okay.  I have made some modifications to my version.  I would like to
> get it into a public VCS as soon as possible (see below) so we can
> refine further.

Right, could you please check it in into the "pyplusplus/experimental" 
folder that Roman has created?


- Matthias -

[pygccxml-development] Friend function that implement operators for a class don't get wrapped

[Roman Y. <rom...@gm...>]

Matthias can you create some simple test case that reproduce the problem?

Thank you very much!

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] pyplusplus development

[Roman Y. <rom...@gm...>]

Hi. I feel the need to setup for next few weeks who is working on what.

I think that you Matthias and Allen should work on module builder interface=
 and
multi decl wrapper. I think that:

    both of you have great ideas
    you already worked together and get good results
    you have much better vision of what should interface look like


On the other side I will concentrate my efforts on:

    providing you with enough functionality
        I think that module builder should be a facade to existing function=
ality

    unit testing

    core functionality
        you ( Matthias and Allen ) has some complaints on generated
code, I will fix this

What do you think?

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Roman Y. <rom...@gm...>]

On 2/28/06, Allen Bierbaum <al...@vr...> wrote:
> >>Not accepted. I created declaration wrapper for every declaration. See
> >>pyplusplus\decl_wrappers, But decorators is much better name. Should
> >>we rename them?
> >>
> >>
> This seems like a lot of code to accomplish something that really
> amounts to having a 4-5 flags per declaration.  I can live with the
> implementation but it seems like overkill IMHO.

I explain my self in an other post.


> I don't understand what you are saying here.  How will we finalize now?
> How will the user be warned if things can't be finalized?

:-). I am planning to remove finalize from pyplusplus.code_creators.
If you don't want pyplusplus create wrapper for some declaration - you will=
 use
that declaration to say this to pyplusplus:

decl.finalize()

I think this is better. What do you think? Also from finalize method I
have access

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Roman Y. <rom...@gm...>]

On 3/1/06, Matthias Baas <ba...@ir...> wrote:
> Allen Bierbaum wrote:
> >>> Accepted, but in future we will replace print statemnet with logging
> >>> module
> >>>
> > Sounds good.
>
> Sounds good to me as well.

Done, will check in shortly.

> >>> - Decl Decorator
> >>>
> >>> Not accepted. I created declaration wrapper for every declaration. Se=
e
> >>> pyplusplus\decl_wrappers, But decorators is much better name. Should
> >>> we rename them?
> >>>
> > This seems like a lot of code to accomplish something that really
> > amounts to having a 4-5 flags per declaration.  I can live with the
> > implementation but it seems like overkill IMHO.
>

I think that I did mistake - I did not explain what fundamental changes
I am going to do/did. I am sorry. ( I did not have internet connection
and did not
want to sit doing nothing. )

> I agree with Allen, this looks fairly complex. Is this supposed to be
> the standardized low level API that Allen's version should use
> internally or is this already supposed to be the high level API
> replacing our previous versions?

Those decl wrappers should be used as high level interface for user to conf=
igure
code creators. It does not really make the different how it implemented ins=
ide.
In our versions it is possible to say

mb =3D module_builder_t(...)
mb.namespace( ... ).ignore()

module_builder_t I created it is just proof of concept: I wanted to
see that what you
( you and Allen ), could be implemented and that it really simplifies
user interface.

Also as you can see I did not implemented multi decl wrapper - I don't
know how to
implement it.

In short I want you and Allen concentrate your efforts on user interface:
class module_builder_t and multi decl wrapper and I will serve you as progr=
ammer
who provide you all functionality you need. Do you agree?

> I agree with Allen, this looks fairly complex.

But here are the list of futures that this will allow us to have:
1. Analysis. Using new model I can add code that will analyze declaration f=
or:
    is exportable and if not why
    is finalizable and if not why

2. Better interface - it is easy for user to see what he can customize.

3. It allowed me, not to introduce new concept - declaration type
flags, but rather
    to use isinstance of Python.

> >>> New classes that derives from declaration_t class.
> >>> Every derived decl_wrapper has relevant properties for customizing co=
de
> >>> creators\module_creator.
> >>>
> >>> All code creators properties it is just an redirection to decl_wrappe=
r
> >>> properties.
>
> The latest changes confused me quite a bit. It seems pyplusplus
> currently is under heavy restructuring (source code layout and already
> existing interfaces) which isn't easy to follow for me and which
> currently breaks my "driver" script for creating my bindings (that's
> what I meant the other day on the c++-sig list when I was against
> exposing the entire internal API as public API because the chances a
> future pyplusplus version will break a user's script are much higher
> then. And here we are... ;)

:-(. I tested my changes. They should work as is without any changes
from user code.
It seems that I introduce a bug.

Also I would like to explain them:
We decided on next concepts, correct me if I wrong:

declarations tree is used to configure code creators

code creators tree will be used to configure file writers

That is exactly, what I did. In order to introduce, as minimum changes as I=
 can,
I just redirected code_creators properties to declaration properties.

> And while I'm all for consistent coding guidelines I currently find it
> quite confusing to have several different classes with the same name (I
> spotted at least three different classes called "class_t"). For me, this
> doesn't really help understanding the code.

I know, this should be fixed. Can you propose naming scheme for
pyplusplus code creators
and declarations wrappers/decorators ?

Thanks

> >>> pygccxml:
> >>> For every declaration class I am going to add *_matcher_t classes.
> >>> This will help
> >>> user to find declaration by type using some predefined criterias.
> >>>
> > Can you give some example of this?
>
> Yes, what will the interface look like and what do those matchers return?
>
> (Maybe it would be a good idea to add a fictitious FAQ to the wiki that
> contains some tasks and how this should be done with an "ideal" API.
> This could then serve as a "checklist" to test new API proposals against)

Here what I thought:

Finding declaration by some criteria is very useful functionality for pygcc=
xml.
I adopted Allen idea, also I changed few implementation details.
All code that shows the usage of *matcher could be found in relevant
unit testers.
( variable_matcher_tester.py, namespace_matcher_tester.py, filters_tester.p=
y,
  calldef_tester.py )

Example:

        criteria =3D declarations.variable_matcher_t( name=3D'x',
type=3D'unsigned int' )
        x =3D declarations.matcher.get_single( criteria, self.declarations =
)

        criteria =3D declarations.variable_matcher_t(
            name=3D'::bit_fields::fields_t::x'
            , type=3Ddeclarations.unsigned_int_t()
            , value=3Dx.value
            , header_dir=3Dos.path.dirname(x.location.file_name)
            , header_file=3Dx.location.file_name)

Main differences:
1. Better user interface/implementation:

declaration_matcher_t.__call__ is smart enough to find out whether
user wants to match
by full name or name only.

type could be or string or cpptypes.type_t instance.

As I already said, there is own place where user can go and find out
what criteria
he can use.

Take a look on calldef_matcher_t class. The main difference in implementati=
on is
that I allow user to specify something like this:

void f( int, double );

matcher =3D calldef_matcher_t( arg_types=3D[None, 'double'] )
This means that I want to select all function that have 2 arguments
and their second
type is double.

An other change is in regular expression matcher.
Here is the class:

class regex_matcher_t:
    def __init__( self, regex, function ):
        self.regex =3D re.compile( regex )
        self.function =3D function

    def __call__( self, decl ):
        text =3D self.function( decl )
        return bool( self.regex.match( text ) )

If user want to match not only name but something else by regular expressio=
n,
he should not derive from the class, but rather to write function that
takes as argument
declaration and returns some text. I found this approach easier then
starting to find out
class hierarchy.


> By the way, I've started adding some stuff to the discussion page in the
> wiki. As I've never used a wiki for this kind of collaborative software
> developement yet, are there any guidelines or rules about how the issues
> should be layed out or how and what information is to be added?

I think, we don't have to start with rules, but common sense.
Rules will come later.

> I also still have to reply on some stuff in the c++-sig list, I just
> wondered if I should add those replies right to the wiki or do so by
> mail (and I need some more time to do so. Replying to all the stuff and
> getting the current stuff to work has become rather time-consuming
> lately...)

Matthias, I think you should stop to use latest CVS in production environme=
nt,
until we stabilize interface. That's said.

> - Matthias -
>


--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Matthias B. <ba...@ir...>]

Allen Bierbaum wrote:
>>> Accepted, but in future we will replace print statemnet with logging 
>>> module
>>>   
> Sounds good.

Sounds good to me as well.

>>> - Decl Decorator
>>>
>>> Not accepted. I created declaration wrapper for every declaration. See
>>> pyplusplus\decl_wrappers, But decorators is much better name. Should
>>> we rename them?
>>>   
> This seems like a lot of code to accomplish something that really 
> amounts to having a 4-5 flags per declaration.  I can live with the 
> implementation but it seems like overkill IMHO.

I agree with Allen, this looks fairly complex. Is this supposed to be 
the standardized low level API that Allen's version should use 
internally or is this already supposed to be the high level API 
replacing our previous versions?

>>> New classes that derives from declaration_t class.
>>> Every derived decl_wrapper has relevant properties for customizing code
>>> creators\module_creator.
>>>
>>> All code creators properties it is just an redirection to decl_wrapper
>>> properties.

The latest changes confused me quite a bit. It seems pyplusplus 
currently is under heavy restructuring (source code layout and already 
existing interfaces) which isn't easy to follow for me and which 
currently breaks my "driver" script for creating my bindings (that's 
what I meant the other day on the c++-sig list when I was against 
exposing the entire internal API as public API because the chances a 
future pyplusplus version will break a user's script are much higher 
then. And here we are... ;)
And while I'm all for consistent coding guidelines I currently find it 
quite confusing to have several different classes with the same name (I 
spotted at least three different classes called "class_t"). For me, this 
doesn't really help understanding the code.

>>> pygccxml:
>>> For every declaration class I am going to add *_matcher_t classes.
>>> This will help
>>> user to find declaration by type using some predefined criterias.
>>>   
> Can you give some example of this?

Yes, what will the interface look like and what do those matchers return?

(Maybe it would be a good idea to add a fictitious FAQ to the wiki that 
contains some tasks and how this should be done with an "ideal" API. 
This could then serve as a "checklist" to test new API proposals against)

By the way, I've started adding some stuff to the discussion page in the 
wiki. As I've never used a wiki for this kind of collaborative software 
developement yet, are there any guidelines or rules about how the issues 
should be layed out or how and what information is to be added?
I also still have to reply on some stuff in the c++-sig list, I just 
wondered if I should add those replies right to the wiki or do so by 
mail (and I need some more time to do so. Replying to all the stuff and 
getting the current stuff to work has become rather time-consuming 
lately...)


- Matthias -

[pygccxml-development] Re: recent changes to pygccxml\pyplusplus

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

>I send this mail to mailing list, but forgot to send it to you.
>
>Please, subscribe to the mailing list and replay to mailing list
>
>Thank you
>
>On 2/28/06, Roman Yakovenko <rom...@gm...> wrote:
>  
>
>>Hi. In order to create more user friendly API I had to make few
>>changes to both projects.
>>Here is a list of changes:
>>
>>Allen changes:
>>=============
>>- Added verbose flag for additional output
>>   - pygccxml: parser (through parser config), project_reader,
>>               cache
>>      - Added timing information to output
>>      - Outputs gccxml command line
>>
>>Accepted, but in future we will replace print statemnet with logging module
>>    
>>
Sounds good.

>>=============
>>- Cache refactoring
>>
>>Accepted
>>
>>=============
>>Detection of gccxml errors
>>
>>Not accepted, it does not work with my version of gccxml, may be we
>>need to define
>>with what version of gcc xml we are working?
>>    
>>
okay.  I will check at home.  I believe I have gccxml from cvs 1 week 
old.  I am running it on Linux (FC4).

>>=============
>>- Decl Decorator
>>
>>Not accepted. I created declaration wrapper for every declaration. See
>>pyplusplus\decl_wrappers, But decorators is much better name. Should
>>we rename them?
>>    
>>
This seems like a lot of code to accomplish something that really 
amounts to having a 4-5 flags per declaration.  I can live with the 
implementation but it seems like overkill IMHO.

>>=============
>>- Additional information when finalize fails
>>
>>Not accepted. The reason is simple, I think that I don't have to
>>provide this functionality
>>on code creators at all. I will remove it. If you don't want to create
>>a wrapper you will
>>say it using standard way - customizing declarations.
>>    
>>
I don't understand what you are saying here.  How will we finalize now?  
How will the user be warned if things can't be finalized?

>>=============
>>- Extended multiple file writer
>>
>>Accepted as is.
>>
>>=============
>>Spelling fixes
>>
>>Accepted. Please fix as much as you can :-)
>>
>>=============
>>The latest versions of gccxml mark all classes and structs as artificial.
>>
>>Accepted
>>=============
>>Added/extended comments in some places
>>
>>Accepted
>>=============
>>
>>
>>My changes:
>>
>>pyplusplus:
>>
>>New classes that derives from declaration_t class.
>>Every derived decl_wrapper has relevant properties for customizing code
>>creators\module_creator.
>>
>>All code creators properties it is just an redirection to decl_wrapper
>>properties.
>>
>>pygccxml:
>>For every declaration class I am going to add *_matcher_t classes.
>>This will help
>>user to find declaration by type using some predefined criterias.
>>    
>>
Can you give some example of this?

-Allen

>>Also we should add to pygccxml all code that Allen wrote in filter.py module.
>>But it should be changed a little to use pygccxml coding convention +
>>instead of using
>>flags, I think, it should use isinstance function.
>>
>>I also added module_builder module. Please take a look on it.
>>
>>Thanks
>>
>>
>>--
>>Roman Yakovenko
>>C++ Python language binding
>>http://www.language-binding.net/
>>
>>    
>>
>
>
>--
>Roman Yakovenko
>C++ Python language binding
>http://www.language-binding.net/
>
>  
>

[pygccxml-development] ImportError on current cvs

[Matthias B. <ba...@ir...>]

Hi,

I just updated from cvs and now there's a cyclic import in the code. 
When I just import pyplusplus in a Python shell, I get an import error:

 >>> import pyplusplus
...
   File ".../pyplusplus/code_creators/array_1_registrator.py", line 10, in ?
     from pyplusplus.decl_wrappers import call_policies
ImportError: No module named decl_wrappers


The main initialization module contains the following imports:

import code_creators
import file_writers
import module_creator
import gui
import code_repository
import utils
import decl_wrappers
import module_builder

The first import already triggers code in array_1_registrator that tries 
to import from "pyplusplus.decl_wrappers" but decl_wrappers only gets 
imported later so it isn't available at that time and the above 
exception occurs.

- Matthias -

[pygccxml-development] recent changes to pygccxml\pyplusplus

[Roman Y. <rom...@gm...>]

Hi. In order to create more user friendly API I had to make few
changes to both projects.
Here is a list of changes:

Allen changes:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- Added verbose flag for additional output
   - pygccxml: parser (through parser config), project_reader,
               cache
      - Added timing information to output
      - Outputs gccxml command line

Accepted, but in future we will replace print statemnet with logging module

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- Cache refactoring

Accepted

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Detection of gccxml errors

Not accepted, it does not work with my version of gccxml, may be we
need to define
with what version of gcc xml we are working?

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- Decl Decorator

Not accepted. I created declaration wrapper for every declaration. See
pyplusplus\decl_wrappers, But decorators is much better name. Should
we rename them?

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- Additional information when finalize fails

Not accepted. The reason is simple, I think that I don't have to
provide this functionality
on code creators at all. I will remove it. If you don't want to create
a wrapper you will
say it using standard way - customizing declarations.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
- Extended multiple file writer

Accepted as is.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Spelling fixes

Accepted. Please fix as much as you can :-)

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
The latest versions of gccxml mark all classes and structs as artificial.

Accepted
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Added/extended comments in some places

Accepted
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D


My changes:

pyplusplus:

New classes that derives from declaration_t class.
Every derived decl_wrapper has relevant properties for customizing code
creators\module_creator.

All code creators properties it is just an redirection to decl_wrapper
properties.

pygccxml:
For every declaration class I am going to add *_matcher_t classes.
This will help
user to find declaration by type using some predefined criterias.

Also we should add to pygccxml all code that Allen wrote in filter.py modul=
e.
But it should be changed a little to use pygccxml coding convention +
instead of using
flags, I think, it should use isinstance function.

I also added module_builder module. Please take a look on it.

Thanks


--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Potential bug in source_reader.py

[Matthias B. <ba...@ir...>]

Hi,

I had a closer look at pygccxml to see how caching works. As I still 
have a couple of performance issues with the current file cache I'm 
planning on implementing a new cache class that splits its data among 
several files instead of one big monolithic file.

While inspecting the sources I have found some kind of bug. The update() 
method of the cache class has an argument "included_files" which I 
believe should contain the names of the header files that were included 
in the actual header file "source_file". There's no comment or something 
that says what type "included_files" should be, but from looking at the 
code it seems the cache class is treating this like a list of header 
names. However, what it actually receives is a dictionary that has the 
file names as value. With the current code, the cache class only 
inspects the keys of that dictionary (which just contains things like 
"f0", "f1", ...) and ignores the file names. This leads to a signature 
list that only contains None values and modifications to the header 
files will never be noticed.

I suppose the actual bug is in the __parse_gccxml_created_file() method 
(in the source_reader_t class) as this method returns the files as 
dictionary whereas it should probably rather return 
"list(files.itervalues())" instead of just "files".
As I have concentrated on the caching stuff I don't know if any other 
parts of the code use the result of __parse_gccxml_created_file(). So 
the question now is just if included_files should actually be a dict or 
a list?

- Matthias -

[pygccxml-development] pygccxml\pyplusplus changes

[Roman Y. <rom...@gm...>]

Hi. pygccxml\pyplusplus has mailing list now.

Here is a url where you can subscribe to the list:
http://lists.sourceforge.net/lists/listinfo/pygccxml-development

We are going to try to implement more user friendly API.
Everyone is welcome to help or comment.

Also, for some time you will be not able to use CVS version of pyplusplus.
So I created 2 zip files, pygccxml and pyplusplus, that you can
download and use.
They contain a lot of bug fixes, if you compare to previous released versio=
n.

Thanks.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] pygccxml\pyplusplus mailing list

[Roman Y. <rom...@gm...>]

I hope, that with help of community, I and other developers of
pygccxml\pyplusplus
will be able to create really useful and valuable product.

Let's do it.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/