pygccxml-development

[pygccxml-development] low level vs high level API

[Matthias B. <ba...@ir...>]

Hi,

this mail is a general reply to the previous mails posted. I think 
there's still some confusion among me and Allen about the last changes 
to pyplusplus (module_builder_t, decl_wrappers, etc). So I'd just like 
to mention a few points and check if we've already agreed on them or if 
there are still different views (which may have led to the confusion):

- The final version of pyplusplus will have two "separate" APIs, a low 
level/internal API and a high level API. Is this already general consensus?

- The internal API is what pyplusplus already had when I first tried it 
out a few weeks ago. A user could just use this API and create bindings. 
In this case, his driver script may be somewhat verbose but he has full 
control over pyplusplus.

- The high level API is a mere convenience for the user to express 
things more concisely. This API refers to the internal API to implement 
its stuff. Currently, Allen's pypp_api module constitutes the high level 
API.
If I've understood Roman correctly, I believe his new module_builder_t 
and decl_wrapper classes should already be seen as high level API, 
replacing pypp_api, right? What I liked before those classes was the 
strict separation between low level API and high level API. It was clear 
whenever I was using something "low level" (a class with _t suffix) and 
something "high level" (from pypp_api or initially also from my own 
version). This distinction has become somewhat blurred and the high 
level and low level stuff have become somewhat intermingled. As there is 
no documentation yet, the only hint would be the sources themselves but 
they don't provide any information whether something is considered to be 
part of the low level or the high level API.

So what I'm missing is documentation about the low level API that Allen 
and I could rely on to experiment with various high level APIs. Maybe 
some ideas will lead to modifications to the low level part of 
pyplusplus but when I was creating my own API version I was quite 
pleased to see that the previous internal API has already allowed 
expressing almost everything I needed. So it was not that bad at all.

- Matthias -

Re: [pygccxml-development] low level vs high level API

[Allen B. <al...@vr...>]

Matthias Baas wrote:

> Hi,
>
> this mail is a general reply to the previous mails posted. I think 
> there's still some confusion among me and Allen about the last changes 
> to pyplusplus (module_builder_t, decl_wrappers, etc). So I'd just like 
> to mention a few points and check if we've already agreed on them or 
> if there are still different views (which may have led to the confusion):
>
> - The final version of pyplusplus will have two "separate" APIs, a low 
> level/internal API and a high level API. Is this already general 
> consensus?

I agree with this.

>
> - The internal API is what pyplusplus already had when I first tried 
> it out a few weeks ago. A user could just use this API and create 
> bindings. In this case, his driver script may be somewhat verbose but 
> he has full control over pyplusplus.

This is my understanding.

>
> - The high level API is a mere convenience for the user to express 
> things more concisely. This API refers to the internal API to 
> implement its stuff. Currently, Allen's pypp_api module constitutes 
> the high level API.
> If I've understood Roman correctly, I believe his new module_builder_t 
> and decl_wrapper classes should already be seen as high level API, 
> replacing pypp_api, right? 

I am confused about this as well.  It seems that these classes mix two 
ideas:
1. They provide some of the interface that my high-level API needs to 
modify they way creators are created (the injected flags patch I submitted)
2. They also seem to be attempting to draw in some (but not all) of the 
capabilities that we were talking about providing to the users in the 
high-level API.  It seems reasonable that there will be some duplication 
here because the high-level API will use some helpers in the low-level 
API but I don't think it is a good idea to keep moving things across to 
the low-level API.  I agree with Matthias's comments below about 
separation being a good thing.

> What I liked before those classes was the strict separation between 
> low level API and high level API. It was clear whenever I was using 
> something "low level" (a class with _t suffix) and something "high 
> level" (from pypp_api or initially also from my own version). This 
> distinction has become somewhat blurred and the high level and low 
> level stuff have become somewhat intermingled. As there is no 
> documentation yet, the only hint would be the sources themselves but 
> they don't provide any information whether something is considered to 
> be part of the low level or the high level API.
>
> So what I'm missing is documentation about the low level API that 
> Allen and I could rely on to experiment with various high level APIs. 
> Maybe some ideas will lead to modifications to the low level part of 
> pyplusplus but when I was creating my own API version I was quite 
> pleased to see that the previous internal API has already allowed 
> expressing almost everything I needed. So it was not that bad at all.

Agreed.  I like the idea of you and I refining and expanding the 
high-level API while we pepper Roman with ideas, requests, and 
refinements for the low-level API.  (and as my status update message 
describes I think one of the first things I would like to see is 
documentation and comments. :)

So summary, I agree with you Matthias.

-Allen

>
> - Matthias -
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> pygccxml-development mailing list
> pyg...@li...
> https://lists.sourceforge.net/lists/listinfo/pygccxml-development
>

Re: [pygccxml-development] low level vs high level API

[Roman Y. <rom...@gm...>]

On 3/6/06, Allen Bierbaum <al...@vr...> wrote:
> >
> > - The high level API is a mere convenience for the user to express
> > things more concisely. This API refers to the internal API to
> > implement its stuff. Currently, Allen's pypp_api module constitutes
> > the high level API.
> > If I've understood Roman correctly, I believe his new module_builder_t
> > and decl_wrapper classes should already be seen as high level API,
> > replacing pypp_api, right?

Allen, your pypp_api is the only high level api. I created my class
for testers only.

> I am confused about this as well.  It seems that these classes mix two
> ideas:
> 1. They provide some of the interface that my high-level API needs to
> modify they way creators are created (the injected flags patch I submitte=
d)
> 2. They also seem to be attempting to draw in some (but not all) of the
> capabilities that we were talking about providing to the users in the
> high-level API.  It seems reasonable that there will be some duplication
> here because the high-level API will use some helpers in the low-level
> API but I don't think it is a good idea to keep moving things across to
> the low-level API.  I agree with Matthias's comments below about
> separation being a good thing.
>
> > What I liked before those classes was the strict separation between
> > low level API and high level API. It was clear whenever I was using
> > something "low level" (a class with _t suffix) and something "high
> > level" (from pypp_api or initially also from my own version). This
> > distinction has become somewhat blurred and the high level and low
> > level stuff have become somewhat intermingled. As there is no
> > documentation yet, the only hint would be the sources themselves but
> > they don't provide any information whether something is considered to
> > be part of the low level or the high level API.
> >
> > So what I'm missing is documentation about the low level API that
> > Allen and I could rely on to experiment with various high level APIs.
> > Maybe some ideas will lead to modifications to the low level part of
> > pyplusplus but when I was creating my own API version I was quite
> > pleased to see that the previous internal API has already allowed
> > expressing almost everything I needed. So it was not that bad at all.
>
> Agreed.  I like the idea of you and I refining and expanding the
> high-level API while we pepper Roman with ideas, requests, and
> refinements for the low-level API.  (and as my status update message
> describes I think one of the first things I would like to see is
> documentation and comments. :)

This is exactly what I did:

1. You needed decls printer, right? You wrote it. This was greate
piece of source.
    I put it within declarations and declarations wrappers

2. You needed some search algorithm with power of customization of
search criteria, right?
   I took Matthias ideas and some code and added it to pygccxml.declaration=
s
   The first place user will look at, in order to see how can he find
some declaration, am I
    wrong?

3. You needed multi-declaration wrapper, right?
  It took me few days to find out how can we use Python, in order to
provide best
  implementation to the class.

I think that I am not able to design high level API. So please, please
go ahead and do it.


> -Allen

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] low level vs high level API

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> 2. You needed some search algorithm with power of customization of
> search criteria, right?
>    I took Matthias ideas and some code and added it to pygccxml.declarations
>    The first place user will look at, in order to see how can he find
> some declaration, am I wrong?

Are you referring to the algorithm module, the matcher module or some 
other module? I couldn't find any code that looked like I could do the 
kind of queries I was doing in my version....?

- Matthias -

Re: [pygccxml-development] low level vs high level API

[Roman Y. <rom...@gm...>]

On 3/7/06, Matthias Baas <ba...@ir...> wrote:
> Roman Yakovenko wrote:
> > 2. You needed some search algorithm with power of customization of
> > search criteria, right?
> >    I took Matthias ideas and some code and added it to pygccxml.declara=
tions
> >    The first place user will look at, in order to see how can he find
> > some declaration, am I wrong?
>
> Are you referring to the algorithm module, the matcher module or some
> other module? I couldn't find any code that looked like I could do the
> kind of queries I was doing in my version....?

I am referring to pygccxml/declarations/filters.py module.:
1. you can find any declaration by name, full name, location( file,
directory ),
   declaration type

2. you can find any function by return type, [partial] list of arguments

3. you can find variable by type, name, value

4. you can find declaration using regular expression

5. you can find declaration by access type

> - Matthias -

That's all your ideas, not my. Take a look on it please.
Also I created new module pygccxml/declarations/matcher.py. It should repla=
ce
relevant functionality found in algorithm.py. I even put some
documentation in filters.py
module :-).

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] low level vs high level API

[Allen B. <al...@vr...>]

Roman Yakovenko wrote:

>
>
>This is exactly what I did:
>
>1. You needed decls printer, right? You wrote it. This was greate
>piece of source.
>    I put it within declarations and declarations wrappers
>
>2. You needed some search algorithm with power of customization of
>search criteria, right?
>   I took Matthias ideas and some code and added it to pygccxml.declarations
>   The first place user will look at, in order to see how can he find
>some declaration, am I
>    wrong?
>  
>
I am not so sure about this one.  I think the goal of the high-level API 
should be that the first and generally only place the user looks is the 
high-level API.  That is really the key to having a domain specific 
language, you want the user to only have to use it to do everything they 
commonly need to.  Now to implement that language/interface we could and 
will use code from throughout the codebase.  At least that is the way I 
see it.  comments?

>3. You needed multi-declaration wrapper, right?
>  It took me few days to find out how can we use Python, in order to
>provide best
>  implementation to the class.
>
>  
>
This one surprised me a little because I did not know I needed it. :)

I think the fact that we didn't understand your goal for the *_wrappers 
really caused some confusion about what the goal is for everything 
else.  As it stands now even I am not sure if we really need the 
multi-decl wrapper in the pyplusplus code.  Instead we may just need 
something like it in the high-level API which is where the code will be 
for querying the declarations anyway.

-Allen

>I think that I am not able to design high level API. So please, please
>go ahead and do it.
>
>
>  
>
>>-Allen
>>    
>>
>
>--
>Roman Yakovenko
>C++ Python language binding
>http://www.language-binding.net/
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by xPML, a groundbreaking scripting language
>that extends applications into web and mobile media. Attend the live webcast
>and join the prime developer group breaking into this new coding territory!
>http://sel.as-us.falkag.net/sel?cmd=k&kid0944&bid$1720&dat1642
>_______________________________________________
>pygccxml-development mailing list
>pyg...@li...
>https://lists.sourceforge.net/lists/listinfo/pygccxml-development
>
>  
>

Re: [pygccxml-development] low level vs high level API

[Roman Y. <rom...@gm...>]

On 3/8/06, Allen Bierbaum <al...@vr...> wrote:
> >2. You needed some search algorithm with power of customization of
> >search criteria, right?
> >   I took Matthias ideas and some code and added it to pygccxml.declarat=
ions
> >   The first place user will look at, in order to see how can he find
> >some declaration, am I
> >    wrong?
> >
> >
> I am not so sure about this one.  I think the goal of the high-level API
> should be that the first and generally only place the user looks is the
> high-level API.  That is really the key to having a domain specific
> language, you want the user to only have to use it to do everything they
> commonly need to.  Now to implement that language/interface we could and
> will use code from throughout the codebase.  At least that is the way I
> see it.  comments?

I knew,  that this is the point, that confuse you and Matthias, right?
Here are my thoughts:

I think that small understanding of pygccxml/declarations package is a
must for some
advanced functionality, For example even to find all declarations by
access type he need
to go to declarations package documentation/source code, right? From
the other side
you can put all functionality you need in order to implement high
level API in in
module/package. This will get a little bit messy, an user will get
lost quickly. Example:
pypp_api.DeclWrapper.findContainedMethod do you think that user wants to se=
e the
implementation of the method? I doubt. Also filters is a new concept,
but high level
API can hide/show it. For example:

next code is taken from "for test only" version of pyplusplus

    def namespace( self, *arguments, **keywords ):
        if len( arguments ) =3D=3D 1 and not keywords and is_callable(argum=
ents[1]):
            return decls_package.matcher.get_single( arguments[0],
self.declarations )
        else:
            matcher =3D decls_package.namespace_matcher_t( *arguments,
**keywords )
            return decls_package.matcher.get_single( matcher,
self.declarations )

I think that code is really easy to understand and more important is
how user will invoke it:

mb =3D modyule_builder_t(... )
details_ns =3D mb.namespace( name=3D=3D'::details', recursive=3DFalse )

As you see you don't have to go and learn filters concept.
Also, if user needs some custom logic, he can write something like this:

details_ns =3D mb.namespace( lambda decl: ..... )

I think this is really simple approach, but I could be wrong.

> >3. You needed multi-declaration wrapper, right?
> >  It took me few days to find out how can we use Python, in order to
> >provide best
> >  implementation to the class.
> >
> >
> >
> This one surprised me a little because I did not know I needed it. :)
>
> I think the fact that we didn't understand your goal for the *_wrappers
> really caused some confusion about what the goal is for everything
> else.  As it stands now even I am not sure if we really need the
> multi-decl wrapper in the pyplusplus code.  Instead we may just need
> something like it in the high-level API which is where the code will be
> for querying the declarations anyway.

Sorry, but I don't understand last question :-(. Does it really metter
where you write the
code? If you need something like mulit-decl wrapper in high level API
why don't you use
the one I created?

> -Allen

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

[pygccxml-development] Re: low level vs high level API

[Roman Y. <rom...@gm...>]

On 3/6/06, Matthias Baas <ba...@ir...> wrote:
> Hi,
>
> this mail is a general reply to the previous mails posted. I think
> there's still some confusion among me and Allen about the last changes
> to pyplusplus (module_builder_t, decl_wrappers, etc). So I'd just like
> to mention a few points and check if we've already agreed on them or if
> there are still different views (which may have led to the confusion):

Funny, this is exactly what I did yerstaday. I will try to update Wiki
today. But I also will post my file here.

> - The final version of pyplusplus will have two "separate" APIs, a low
> level/internal API and a high level API. Is this already general consensu=
s?

Yes. I think code creators and file writers is low level, declarations
and declarations
wrappers/decorators is concept that user should be familiar with

> - The internal API is what pyplusplus already had when I first tried it
> out a few weeks ago. A user could just use this API and create bindings.
> In this case, his driver script may be somewhat verbose but he has full
> control over pyplusplus.
>
> - The high level API is a mere convenience for the user to express
> things more concisely. This API refers to the internal API to implement
> its stuff. Currently, Allen's pypp_api module constitutes the high level
> API.

Allen pypp_api is the only high level API!

> If I've understood Roman correctly, I believe his new module_builder_t
> and decl_wrapper classes should already be seen as high level API,
> replacing pypp_api, right?

No, no, no. I just hate when I my all tests fail. So I created something mi=
nimal
I can get my tests run.

> What I liked before those classes was the
> strict separation between low level API and high level API. It was clear
> whenever I was using something "low level" (a class with _t suffix) and
> something "high level" (from pypp_api or initially also from my own
> version). This distinction has become somewhat blurred and the high
> level and low level stuff have become somewhat intermingled. As there is
> no documentation yet, the only hint would be the sources themselves but
> they don't provide any information whether something is considered to be
> part of the low level or the high level API.

I don't understand

> So what I'm missing is documentation about the low level API that Allen
> and I could rely on to experiment with various high level APIs. Maybe
> some ideas will lead to modifications to the low level part of
> pyplusplus but when I was creating my own API version I was quite
> pleased to see that the previous internal API has already allowed
> expressing almost everything I needed. So it was not that bad at all.

It was confusing at least. Try to remember your self with all those
recursive flags?
More over, if you would use old API, you will need to pay for:
scanning declarations tree and code creators tree more then necessary

> - Matthias -
>
>

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/