pygccxml-development

[pygccxml-development] Updated status

[Allen B. <al...@vr...>]

A apologize in advance, this e-mail is going to be a little random 
because it is based on my notes from this afternoon.  I started trying 
to move the pypp_api interface over the the new codebase and these are 
the things I have run into right away.

- Calling parser changed a bit.

There seems to be a new interface to calling the project parser.  I 
think I just need to add a decl wrapper factory but I am not for sure.  
Roman?

- How am I supposed to use the decl_wrappers?

I am trying to parse with the project reader and then I was expecting 
that the returned decls would actually be the wrappers/decorators for 
the decls.  Is this how it is supposed to work?

- Code complexity: The new decorators does look like a LOT of additional 
code for what could be done with some grafted flags.  Not really a 
question, more of a comment.  I really think the codebase could be 
simplified a lot.  For me at least all the property usage seems like a 
lot of extra code to read through and maintain for very little payoff.

- What is module_builder_t for?  Do I have to use it?  It looks like 
Roman's stab at the higher-level interface but what do I need to copy 
from there into pypp_api.

- Why did call policies move to the wrapper module?  Those still seem 
like code_creators.

- Decl wrappers don't seem to support everything I needed/added with the 
decorators.

Specifically, how can I add custom creators?  In my flag-based version I 
could add a list of code creators that would automatically be added on 
when the creator class build the code creators for a decl. 

- What are all the options that have been added to the wrappers? (see 
documentation below)

- I think Logger should probably use info() for most of the current 
output.  Why put time and level name in log output?

My original goal with this output was to have verbose output for users 
not to provide a debug log to a log file on errors. Any objections to 
making the output look cleaner by making the info() filter just output 
the text instead of all the header information?

- Comments: We *really* need comments and documentation for the classes 
and methods in pygccxml and pyplusplus.  I think Matthias and I could 
help out by pointing to areas of the code and saying in our very nicest 
voice "Roman, please please document this".

Toward this goal (and following up on Matthias's comments from last 
week), I started looking at what it would take to generate documentation 
using epydoc and restructuredtext.  I added a script in 
"experimental/docs/generate_all_docs.py" that will call epydoc with what 
I think are the correct parameters to generate html documentation.  I 
also started converting pypp_api over to use restructured text for 
documentation. The only problem is that it looks like the tags/fields I 
am using are not being recognized correctly (ex: params, args, etc).  
According to the docs I am doing things correctly but it is just not 
working.  If either of you have experience with this tool please take a 
look and let me know if I am doing something wrong.

Does this seem like a good direction for documentation?  I know we can 
document classes and methods using this, do we want to document the 
usage of the modules like this as well?  (it is supported but do we want 
that type of documentation in the code?)

That is about all I have for now.  Until I find out more about the 
decl_wrapper's and their use I am at a stand still. :(

Thanks,
Allen

Re: [pygccxml-development] Updated status

[Matthias B. <ba...@ir...>]

Allen Bierbaum wrote:
> Toward this goal (and following up on Matthias's comments from last 
> week), I started looking at what it would take to generate documentation 
> using epydoc and restructuredtext.  I added a script in 
> "experimental/docs/generate_all_docs.py" that will call epydoc with what 
> I think are the correct parameters to generate html documentation.  I 
> also started converting pypp_api over to use restructured text for 
> documentation. The only problem is that it looks like the tags/fields I 
> am using are not being recognized correctly (ex: params, args, etc).  
> According to the docs I am doing things correctly but it is just not 
> working.  If either of you have experience with this tool please take a 
> look and let me know if I am doing something wrong.

We're using epydoc for the PyODE manual, but we're actually using it 
with epytext markup, not rest. But anyway, I'm no expert with epydoc, 
I'm more used to doxygen.

The "experimental" directory is no package, so I had to change the 
"experimental" entry in generate_all_docs.py to "experimental/pypp_api.py".
epydoc actually generates errors on pypp_api.py. Your doc strings 
usually begin with blanks, obviously epydoc things the text is indented 
and generates errors. To test epydoc I changed the doc string for 
ModuleBuilder.__init__ as follows:

       """Initialize module.

          :Parameters:
             - `workingDir`: Directory to start processing.  (default: 
current working dir)
             - `includePaths`: List of paths to tell gccxml to search 
for header files.
             - `gccxmlPath`:   path to gccxml.  If not set then attempt 
to find it in system path.
             - `defines`:  set of symbols to add as defined when compiling.
             - `undefines`:  set of symbols to add as undefined when 
compiling.
             - `cacheFile`:    name of file to use for caching parse 
data for this module.
             - `verbose`:      if true output status and command 
information as building the module.
       """

That is, I moved the first line to the beginning of the string, I 
changed the apostrophes of the list entries into those "backticks" (or 
what are they called?) and I split the docs for define/undefine into two 
entries.
With those changes, epydoc seems to recognize the stuff but the result 
looks somewhat weird. On the generated page each line describing an 
argument begins on a new line (i.e. the argument name and its 
documentation are on two *separate* lines). This looks particularly 
weird, when I also add the type information. This rather looks like a 
bug to me.
I also tested with epytext markup and then the results look normal. So 
maybe we should consider switching to that markup...?

> Does this seem like a good direction for documentation?  I know we can 
> document classes and methods using this, do we want to document the 
> usage of the modules like this as well?  (it is supported but do we want 
> that type of documentation in the code?)

Well, I guess a short usage information in the module or class doc 
string doesn't do any harm. But I guess as there is a dedicated "docs" 
directory already existing in pyplusplus this would probably be a better 
place for adding any tutorial/introduction style thing. But I leave that 
up to Roman how he wants the documentation to be organized.


- Matthias -

Re: [pygccxml-development] Updated status

[Allen B. <al...@vr...>]

Matthias Baas wrote:

> Allen Bierbaum wrote:
>
>> Toward this goal (and following up on Matthias's comments from last 
>> week), I started looking at what it would take to generate 
>> documentation using epydoc and restructuredtext.  I added a script in 
>> "experimental/docs/generate_all_docs.py" that will call epydoc with 
>> what I think are the correct parameters to generate html 
>> documentation.  I also started converting pypp_api over to use 
>> restructured text for documentation. The only problem is that it 
>> looks like the tags/fields I am using are not being recognized 
>> correctly (ex: params, args, etc).  According to the docs I am doing 
>> things correctly but it is just not working.  If either of you have 
>> experience with this tool please take a look and let me know if I am 
>> doing something wrong.
>
>
> We're using epydoc for the PyODE manual, but we're actually using it 
> with epytext markup, not rest. But anyway, I'm no expert with epydoc, 
> I'm more used to doxygen.
>
> The "experimental" directory is no package, so I had to change the 
> "experimental" entry in generate_all_docs.py to 
> "experimental/pypp_api.py".

My bad.  I forgot to check in the __init__.py file I was using. :(

> epydoc actually generates errors on pypp_api.py. Your doc strings 
> usually begin with blanks, obviously epydoc things the text is 
> indented and generates errors. To test epydoc I changed the doc string 
> for ModuleBuilder.__init__ as follows:
>
>       """Initialize module.
>
>          :Parameters:
>             - `workingDir`: Directory to start processing.  (default: 
> current working dir)
>             - `includePaths`: List of paths to tell gccxml to search 
> for header files.
>             - `gccxmlPath`:   path to gccxml.  If not set then attempt 
> to find it in system path.
>             - `defines`:  set of symbols to add as defined when 
> compiling.
>             - `undefines`:  set of symbols to add as undefined when 
> compiling.
>             - `cacheFile`:    name of file to use for caching parse 
> data for this module.
>             - `verbose`:      if true output status and command 
> information as building the module.
>       """
>
> That is, I moved the first line to the beginning of the string, I 
> changed the apostrophes of the list entries into those "backticks" (or 
> what are they called?) and I split the docs for define/undefine into 
> two entries.
> With those changes, epydoc seems to recognize the stuff but the result 
> looks somewhat weird. On the generated page each line describing an 
> argument begins on a new line (i.e. the argument name and its 
> documentation are on two *separate* lines). This looks particularly 
> weird, when I also add the type information. This rather looks like a 
> bug to me.
> I also tested with epytext markup and then the results look normal. So 
> maybe we should consider switching to that markup...?

I am fine with using epytext (or even javadoc).  I was trying to use 
rest since Roman already used it, but I agree that it looks like there 
are still some bugs processing it.  I am more interested in getting good 
documentation in place then I am in working around bugs and picky parsers.

>
>> Does this seem like a good direction for documentation?  I know we 
>> can document classes and methods using this, do we want to document 
>> the usage of the modules like this as well?  (it is supported but do 
>> we want that type of documentation in the code?)
>
>
> Well, I guess a short usage information in the module or class doc 
> string doesn't do any harm. But I guess as there is a dedicated "docs" 
> directory already existing in pyplusplus this would probably be a 
> better place for adding any tutorial/introduction style thing. But I 
> leave that up to Roman how he wants the documentation to be organized.

Agreed.  I will go in whatever direction you both think is best.

-Allen

Re: [pygccxml-development] Updated status

[Roman Y. <rom...@gm...>]

On 3/6/06, Allen Bierbaum <al...@vr...> wrote:
I started to use epydoc for pygccxml. My approach was next:

For epydoc generated documentation I used epytext. I had some problems
with using rest. For manuals, I meen arcticles, tutorials, introduction, I =
use
docutils.

If you go to pygccxml\docs or pyplusplus\docs you will find few *.rest file=
s.
Also you will find script that builds documentation.

This is not all. source directory, root directory of both projects, contain=
s
release_builder.py file. It contains description of process how to build se=
tup.
Now it is broken, but it will take only few hours to fix it.

> >
> >> Does this seem like a good direction for documentation?  I know we
> >> can document classes and methods using this, do we want to document
> >> the usage of the modules like this as well?  (it is supported but do
> >> we want that type of documentation in the code?)
> >
> >
> > Well, I guess a short usage information in the module or class doc
> > string doesn't do any harm. But I guess as there is a dedicated "docs"
> > directory already existing in pyplusplus this would probably be a
> > better place for adding any tutorial/introduction style thing. But I
> > leave that up to Roman how he wants the documentation to be organized.

As much as I hate automatically generated documentation I think that I
don't have choice.
To many classes that should be described, that actually is a set of propert=
ies.
I think we will continue to use this mix, but I am open for suggestions.
I don't have strong opinion on this.

> Agreed.  I will go in whatever direction you both think is best.

Just take a look on docs folders.

> -Allen
>
>


--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Updated status

[Matthias B. <ba...@ir...>]

Roman Yakovenko wrote:
> On 3/6/06, Allen Bierbaum <al...@vr...> wrote:
> I started to use epydoc for pygccxml. My approach was next:
> 
> For epydoc generated documentation I used epytext. I had some problems
> with using rest. For manuals, I meen arcticles, tutorials, introduction, I use
> docutils.

That looks good to me.

> If you go to pygccxml\docs or pyplusplus\docs you will find few *.rest files.
> Also you will find script that builds documentation.
> 
> [...]
> As much as I hate automatically generated documentation [...]

Oops, really? Why is that? (at least this explains the lack of doc 
strings... ;)
Personally, I think the auto-generated docs are useful as a reference 
manual for developers and advanced users (provided that the source code 
also contains appropriate documentation which is collected by the doc 
generation tool).
I mean, what's the alternative? If you don't provide the API 
documentation right with the implementation you have to put it somewhere 
else and make sure it doesn't get out of sync...

>> > - Comments: We *really* need comments and documentation for the classes
>> > and methods in pygccxml and pyplusplus.  I think Matthias and I could
>> > help out by pointing to areas of the code and saying in our very nicest
>> > voice "Roman, please please document this".
> 
> Huston, you've got a problem :-). For me it is a huge effort to write
> documentation.
> I can not explain my self clearly. If you don't mind, I prefer to
> answer 1000 question from
> you and Matthias, but you will write documentation. Please, please, please :-)

That's a deal. But can I ask for for one thing, once there are more doc 
strings and you modify some part of the code, can you then please check 
the corresponding doc string and see if it is still valid (and adjust it 
if necessary)? All the effort would be in vain if the doc strings would 
get out of sync with the implementation after a while.
Also, when you create *new* methods/functions, could you at least 
provide some basic information such as the types of the input arguments 
and the return type (unless they are really obvious) or any precondition 
that has to be met, etc (in general, it is impossible to extract these 
things from the source code). This would already simplify the job of 
understanding the code and we might get away with a little less than 
"1000 questions"... ;-)

I've started creating a script generate_reference_manual.py in 
pygccxml/docs (based on Allen's generation script for the pyplusplus 
package). I haven't incorporated the stuff right into generate_docs.py 
because it also makes sense to run it in isolation but of course, it 
would be no problem to run that script from generate_docs.py as well.
I already converted the doc strings in my directory_cache class to 
epytext markup and also modified a few doc strings at some other places 
in the pygccxml package (so far mainly to remove the warnings/errors). 
Roman, is it ok when I commit these things?

I assume you two are also subscribed to the commit list, right? Then it 
would be great if you, Roman, would check if the doc strings we provide 
really describe the semantics you've originally intended. And Allen, as 
I suppose you are a native English speaker (even though your surname 
kind of looks as if you were German) you could have a short glimpse over 
the doc strings to see that the grammar is not too far off. ;)

- Matthias -

Re: [pygccxml-development] Updated status

[Allen B. <al...@vr...>]

Matthias Baas wrote:

> Roman Yakovenko wrote:
>
>> On 3/6/06, Allen Bierbaum <al...@vr...> wrote:
>> I started to use epydoc for pygccxml. My approach was next:
>>
>> For epydoc generated documentation I used epytext. I had some problems
>> with using rest. For manuals, I meen arcticles, tutorials, 
>> introduction, I use
>> docutils.
>
>
> That looks good to me.
>
>> If you go to pygccxml\docs or pyplusplus\docs you will find few 
>> *.rest files.
>> Also you will find script that builds documentation.
>>
>> [...]
>> As much as I hate automatically generated documentation [...]
>
>
> Oops, really? Why is that? (at least this explains the lack of doc 
> strings... ;)
> Personally, I think the auto-generated docs are useful as a reference 
> manual for developers and advanced users (provided that the source 
> code also contains appropriate documentation which is collected by the 
> doc generation tool).
> I mean, what's the alternative? If you don't provide the API 
> documentation right with the implementation you have to put it 
> somewhere else and make sure it doesn't get out of sync...

Agreed.  docstrings are good for reference documentation and are also a 
good way for the developer to communicate their implementation ideas to 
other developers (and themselves when they look at the code much later. :)

>
>>> > - Comments: We *really* need comments and documentation for the 
>>> classes
>>> > and methods in pygccxml and pyplusplus.  I think Matthias and I could
>>> > help out by pointing to areas of the code and saying in our very 
>>> nicest
>>> > voice "Roman, please please document this".
>>
>>
>> Huston, you've got a problem :-). For me it is a huge effort to write
>> documentation.
>> I can not explain my self clearly. If you don't mind, I prefer to
>> answer 1000 question from
>> you and Matthias, but you will write documentation. Please, please, 
>> please :-)
>
>
> That's a deal. But can I ask for for one thing, once there are more 
> doc strings and you modify some part of the code, can you then please 
> check the corresponding doc string and see if it is still valid (and 
> adjust it if necessary)? All the effort would be in vain if the doc 
> strings would get out of sync with the implementation after a while.
> Also, when you create *new* methods/functions, could you at least 
> provide some basic information such as the types of the input 
> arguments and the return type (unless they are really obvious) or any 
> precondition that has to be met, etc (in general, it is impossible to 
> extract these things from the source code). This would already 
> simplify the job of understanding the code and we might get away with 
> a little less than "1000 questions"... ;-)

Agreed. 

Roman: Even if the documentation you write is not 100% clear it will 
help out.  Then other people like Matthias and I can come along and 
refine and extend the documentation.

>
> I've started creating a script generate_reference_manual.py in 
> pygccxml/docs (based on Allen's generation script for the pyplusplus 
> package). I haven't incorporated the stuff right into generate_docs.py 
> because it also makes sense to run it in isolation but of course, it 
> would be no problem to run that script from generate_docs.py as well.
> I already converted the doc strings in my directory_cache class to 
> epytext markup and also modified a few doc strings at some other 
> places in the pygccxml package (so far mainly to remove the 
> warnings/errors). Roman, is it ok when I commit these things?
>
Does epydoc auto-link between related classes?  If so, shouldn't we 
generate the reference documentation from both pygccxml and pyplusplus 
at the same time so we can follow links from pygccxml types used in 
pyplusplus?

> I assume you two are also subscribed to the commit list, right? Then 
> it would be great if you, Roman, would check if the doc strings we 
> provide really describe the semantics you've originally intended. And 
> Allen, as I suppose you are a native English speaker (even though your 
> surname kind of looks as if you were German) you could have a short 
> glimpse over the doc strings to see that the grammar is not too far 
> off. ;)

You have pegged me correctly.  I am a native speaker.  I don't always 
speak it well, but I do speak it. :)

-Allen

>
> - Matthias -
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting 
> language
> that extends applications into web and mobile media. Attend the live 
> webcast
> and join the prime developer group breaking into this new coding 
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> pygccxml-development mailing list
> pyg...@li...
> https://lists.sourceforge.net/lists/listinfo/pygccxml-development
>

Re: [pygccxml-development] Updated status

[Roman Y. <rom...@gm...>]

On 3/7/06, Matthias Baas <ba...@ir...> wrote:
> > As much as I hate automatically generated documentation [...]
>
> Oops, really? Why is that? (at least this explains the lack of doc
> strings... ;)
> Personally, I think the auto-generated docs are useful as a reference
> manual for developers and advanced users (provided that the source code
> also contains appropriate documentation which is collected by the doc
> generation tool).

Because it is very difficult to find something, also it does not makes
code to look better :-)

> I mean, what's the alternative? If you don't provide the API
> documentation right with the implementation you have to put it somewhere
> else and make sure it doesn't get out of sync...

I don't see alternative, :-(

> >> > - Comments: We *really* need comments and documentation for the clas=
ses
> >> > and methods in pygccxml and pyplusplus.  I think Matthias and I coul=
d
> >> > help out by pointing to areas of the code and saying in our very nic=
est
> >> > voice "Roman, please please document this".
> >
> > Huston, you've got a problem :-). For me it is a huge effort to write
> > documentation.
> > I can not explain my self clearly. If you don't mind, I prefer to
> > answer 1000 question from
> > you and Matthias, but you will write documentation. Please, please, ple=
ase :-)
>
> That's a deal. But can I ask for for one thing, once there are more doc
> strings and you modify some part of the code, can you then please check
> the corresponding doc string and see if it is still valid (and adjust it
> if necessary)?

Yes of course.

> All the effort would be in vain if the doc strings would
> get out of sync with the implementation after a while.
> Also, when you create *new* methods/functions, could you at least
> provide some basic information such as the types of the input arguments
> and the return type (unless they are really obvious) or any precondition
> that has to be met, etc (in general, it is impossible to extract these
> things from the source code). This would already simplify the job of
> understanding the code and we might get away with a little less than
> "1000 questions"... ;-)

I will take it as a rule

> I've started creating a script generate_reference_manual.py in
> pygccxml/docs (based on Allen's generation script for the pyplusplus
> package). I haven't incorporated the stuff right into generate_docs.py
> because it also makes sense to run it in isolation but of course, it
> would be no problem to run that script from generate_docs.py as well.
> I already converted the doc strings in my directory_cache class to
> epytext markup and also modified a few doc strings at some other places
> in the pygccxml package (so far mainly to remove the warnings/errors).
> Roman, is it ok when I commit these things?

Yes, please go ahead.

> I assume you two are also subscribed to the commit list, right? Then it
> would be great if you, Roman, would check if the doc strings we provide
> really describe the semantics you've originally intended. And Allen, as
> I suppose you are a native English speaker (even though your surname
> kind of looks as if you were German) you could have a short glimpse over
> the doc strings to see that the grammar is not too far off. ;)

After first release of pyplusplus I definitely propose to talk a
little about our self :-).

When I was young, I did not think that this could be possible. I was
born in Soviet Union,
Ukraine. I think this is great.

> - Matthias -

Roman

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

Re: [pygccxml-development] Updated status

[Matthias B. <ba...@ir...>]

Allen Bierbaum wrote:
>> I've started creating a script generate_reference_manual.py in 
>> pygccxml/docs (based on Allen's generation script for the pyplusplus 
>> package). I haven't incorporated the stuff right into generate_docs.py 
>> because it also makes sense to run it in isolation but of course, it 
>> would be no problem to run that script from generate_docs.py as well.
>> I already converted the doc strings in my directory_cache class to 
>> epytext markup and also modified a few doc strings at some other 
>> places in the pygccxml package (so far mainly to remove the 
>> warnings/errors). Roman, is it ok when I commit these things?
>>
> Does epydoc auto-link between related classes?  

It doesn't do that automatically (or is there an option to enable 
that?). Instead, you have to add explicit markup: L{class_t} or even 
L{class_t<declarations.class_t>} (see 
http://epydoc.sourceforge.net/epytext.html, section 3.3).

> If so, shouldn't we 
> generate the reference documentation from both pygccxml and pyplusplus 
> at the same time so we can follow links from pygccxml types used in 
> pyplusplus?

Well, Roman wants pygccxml to be independent from pyplusplus so I 
thought it makes sense to add the script to pygccxml/docs.
If it proves to be useful the manual for pyplusplus could still include 
the manual for pygccxml so that there can be cross references. This only 
requires a few additional lines in the pyplusplus doc script and doesn't 
otherwise affect the doc strings in pygccxml.

- Matthias -

Re: [pygccxml-development] Updated status

[Roman Y. <rom...@gm...>]

On 3/8/06, Matthias Baas <ba...@ir...> wrote:
> Allen Bierbaum wrote:
> >> I've started creating a script generate_reference_manual.py in
> >> pygccxml/docs (based on Allen's generation script for the pyplusplus
> >> package). I haven't incorporated the stuff right into generate_docs.py
> >> because it also makes sense to run it in isolation but of course, it
> >> would be no problem to run that script from generate_docs.py as well.
> >> I already converted the doc strings in my directory_cache class to
> >> epytext markup and also modified a few doc strings at some other
> >> places in the pygccxml package (so far mainly to remove the
> >> warnings/errors). Roman, is it ok when I commit these things?
> >>
> > Does epydoc auto-link between related classes?
>
> It doesn't do that automatically (or is there an option to enable
> that?). Instead, you have to add explicit markup: L{class_t} or even
> L{class_t<declarations.class_t>} (see
> http://epydoc.sourceforge.net/epytext.html, section 3.3).
>
> > If so, shouldn't we
> > generate the reference documentation from both pygccxml and pyplusplus
> > at the same time so we can follow links from pygccxml types used in
> > pyplusplus?
>
> Well, Roman wants pygccxml to be independent from pyplusplus so I
> thought it makes sense to add the script to pygccxml/docs.
> If it proves to be useful the manual for pyplusplus could still include
> the manual for pygccxml so that there can be cross references. This only
> requires a few additional lines in the pyplusplus doc script and doesn't
> otherwise affect the doc strings in pygccxml.

Right, I plan to create setup file, that will install pygccxml and pypluspl=
us.
Also as a last screen it can give user an idea where to find the documentat=
ion.

Also pygccxml project is stand alone, I think you can add
documentation to it that is
related to pyplusplus, because it will give an user a hint how to use
this or that feature.

Don't you think so?

> - Matthias -

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/