[pygccxml-development] Re: Roadmap?

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
>> - There's no single module from which the user can import all the
>> required stuff but instead he has to know where everything is located.
> 
> I did not want to polish my implementation. I think it is very easy to
> add aliases
> to relevant classes in module_builder/__init__.py file. I think it
> should take 1 hour.
> Am I wrong?

I even think it only takes a couple of minutes. But I think this would 
have been an important thing as it affects the user's script.

In my view there are three "layers" of software involved in the process 
of creating bindings. Here's some ASCII art that shows the layers (make 
sure to use a fixed pitch font for viewing):

+------------------------------+
|  The user's "driver" script  |
+---------------------------+  |
|  High level API           |  |
+---------------------------+--+
|  pygccxml/pyplusplus "core"  |
+------------------------------+

On top there's the script written by the user that controls pyplusplus 
and creates the final bindings. It should be "easy" (whatever that 
means) for the user to write and maintain this script. The script will 
most likely stick to the high level API but it can also use the "core" 
directly if this is of any advantage.
The second layer is the high level API that handles all the internals of 
pyplusplus and provides the main interface for the driver script so that 
the user's script can concentrate on tailoring the bindings to the 
user's needs. Internally, the high level API uses the "core" just as if 
it was another user script.
The bottom layer is the actual "core" of pyplusplus. It does all the 
work and contains everything that is required to write the final C++ 
source code. It also provides an API but at a lower level (basically 
this is what pyplusplus has already been a month ago).

A few noteworthy points:

- The purpose of the high level API is to make the user's life easier 
and his script more "manageable". The API is not vital for the results. 
The script could produce the very same extension module by using the 
core directly.

- The core is entirely independent of the high level API. From the 
core's point of view the high level API is just another "user script". 
As a consequence, for every piece of code in pyplusplus it should be 
clear whether this code conceptually belongs to the core or to the high 
level API.

- The high level API may still contain "logic" and functionality that is 
not part of the core if this functionality is not strictly required for 
writing extension source code.

Now, in my opinion, when we talk about the interface our main focus 
should be the topmost layer, that is, how should the user's script look 
like? Whereas it seems to me that this thread currently rather focuses 
on the implementation of the middle and bottom layer (which is not that 
relevant at this point).

Above you say that sticking the relevant stuff into one module is just 
"polishing" the implementation. Here I disagree! This "polishing" 
directly affects the user's script on the very top of those layers and 
as such, I consider it to be a very important issue. Whereas if you 
modify something in the pyplusplus core or within the implementation of 
the high level API, but this modification is not "visible" in the user's 
script then there's actually nothing that needs discussion anyway.


>> - The module builder still requires the config object (which I would
>> rather regard as an internal pyplusplus object).
> 
> There are few ways to go:
> 1. You can pass all arguments needed for parsing to module_builder_t.__init__
>     method.
> 
>     I prefer this method
> 
> 2. You can create some class that will aggregate all needed settings and user
>     will pass it to module_builder_t.__init__
> 
>    If you like this approach, we will implement it.

I prefer 1 as well (maybe in addition to some specific config methods) 
and this is already how the experimental ModuleBuilder works. Your 
version uses method 2 which is why I was complaining.

>> - The selection functions accept *any* keyword argument even when that
>> argument is not used/valid at all.
> 
> Well, all you say is that my implementation has not been polished and you are
> right. It is just a concept.

Yes, but missing documentation and missing warning/error message make it 
rather difficult to use. You can neither read how it works nor can you 
apply "trial and error" attempts.

>> - I think making the step to create code creators explicit is not
>> necessary. To me, code creators are rather an implementation detail a
>> user doesn't really have to know about (maybe this will change with more
>> advanced features...?).
> 
> Here I am semi agree with you. I understand your point, but access to powerful
> features of pyplusplus( code creators ) should be also easy. 

When the user needs it he can of course call the function, I'm only 
arguing that it is not necessary to enforce such a call when it is not 
required by the user (which I think is the common case).

> When user call this
> function the only parameter he should give is module name. Also we talk with
> you that every step after parsing should be done explicitly, don't we?

Uhm, I don't remember having said that every step should be explicit. 
Maybe this was just a misunderstanding then...

>>> We can discuss them in few mails and I think we could implement
>>> them on main branch.
>> But if you would now implement everything so that it conforms to my
>> opinion, then you would basically end up with the experimental module.
>> So what's the point of doing that?
> 
> Implementation details. If you are going to implement experimental branch
> using functionality provided by pyplusplus 

I *am* implementing the experimental module by using the functionality 
provided by pyplusplus. This has always been the case. I'm not planning 
on implementing my own code generation tool... :)

> I am sure your code will find the way into main branch in few hours.

But I never said I wanted the high level API to be part of the main 
branch *as quickly as possible*. It has always been my point in the 
previous mails that I actually like it to be independent and separate 
from the core (as long as the API is still under heavy construction).

>> We have that already.
>> Instead we should try to get the core of pyplusplus stable and flexible
>> enough (one area which I still think is a weak point in pyplusplus is
>> the ability to customize the individual output files. I still don't know
>> how/if I could add source code at the declaration level in a particular
>> output file or modify the include files from one particular file).
> 
> Until now, no one requested this functionality. 

Didn't I mention that in a mail once? (hm, maybe I wanted to mention it 
but there were more important things at that time...) Anyway, after the 
introduction of the decl_wrappers it took me a while to get the basics 
working again, so during that period of time the more advanced features 
haven't been that important. That's going to change now... ;)

>> But there are certainly features missing in the API. See for example,
>> the section "What capabilites are not supported by the current API that
>> we should have?" in the wiki.
> 
> May be I am looking in a wrong place, can you post the link? Thanks

https://realityforge.vrsource.org/view/PyOpenSG/PyppApiDiscussion

>>> Matthias, I want people to start using pyplusplus.
>> I see that you're eager to get out a new release, and this is fine with
>> me. :) But what's the problem with releasing pyplusplus and referring to
>> the experimental module which the people can try out and provide
>> feedback? This is not going to be the 1.0 release, is it? ;)
> 
> There is one problem:I think people will not start to use it, until you can say
> explicitly: only small parts of this API could be change. And we can
> not say this on experimental branch :-(

But just because a piece of code is located in another directory doesn't 
mean it is more stable, does it? Wouldn't that be kind of a "false 
security" to think that this API couldn't change anymore?

>>> code. What do you think about next plan: this and next week you are
>>> going to develop
>>> missing functionality.From the 1/04 we close experimental branch and
>>> start merging almost all functionality to main  branch.
>> I cannot guarantee that in a week or two the experimental API will be
>> finished. Actually, I'm pretty much sure that it won't be finished by
>> then as there are still a lot of areas we haven't addressed yet.
> 
> Matthias, I think that this is okay, really. You started to use pyplusplus when
> it was a pain. 

:)

> If  I would wait, while I have "perfect" API, I
> wouldn't get your and
> Allen help. So lets go for release. 

I never said you should not do a release. I'm in favor of a new release! 
I'm only arguing that a new release shouldn't mean the end of the 
experimental module.

>> For me, one of the next things I'd like to see would be automated
>> wrapper generation.... but this is an entirely new thing that I don't
>> know yet how this could be done (and if it could be done at all). It's
>> just an area that needs some experimentation... :)
> 
> Do you know what I am thinking about? GUI. Just imagine you have tree with all
> declarations at the left. On the top you have Python interpreter or an other
> window, that will help you to select declarations set. On the center you will
> have a list of properties and explanation. Nice, ha? Almost fully automated
> solution. But ...

This might be a handy tool. But it will also take quite a bit of time to 
implement that properly...

>> So how about the following instead as the next milestone for pyplusplus:
>>
>> "Get the core of pyplusplus stable and flexible so that *any* high level
>> API could be implemented on top of that without having to modify
>> pygccxml/pyplusplus"
> 
> First of all you need to define a core. Do declaration_matcher_t class hierarchy
> and decl_wrappers are core or not. ( Pay attention select interface implemented
> on decl_wrappers )

By default, I consider everything in pyplusplus as being part of the 
core and everything beneath experimental as high level. :)
The decl_wrapper classes are core classes to me and they are actually 
used already in the experimental module, even though they are only used 
as "decorators" whereas the selection interface is not used. By the way, 
that's an example where the (builtin) high level API is intermingled 
with the core which I do not like. As I said above, I would prefer a 
modular approach where these things are strictly kept separate and where 
the core is independent from the high level API.

- Matthias -