epydoc-devel

[Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi!

Today, I tried epydoc (3.0beta1) on one of my projects, and found several 
things that I miss / find strange:

a) When parsing a signature that is in the first line of a function's 
docstring, epydoc fails to recognize signatures with default arguments (cf. 
test5 in the attached test file).  I later noticed that I could strip them 
from the docstrings since epydoc will extract the default values from the 
function signature nevertheless, but I still see this as a shortcoming.

b) When documenting a method, I like to write an object name in front of the 
signature in order to be able to reference the object by name in the 
documentation.  E.g. "someFoo.testMeth2(a, b)" in the attached file.  
Interestingly, epydoc *does* parse the object name, but it does not display 
it properly but introduce a new parameter of that name.  That looks like a 
bug to me, even if you could argue that it is the name of the object passed 
as "self" / first argument.  However, IMO the signature documents the *usage* 
of a function and there you would not call "Foo.testMeth2(someFoo, a, b)" but 
someFoo.testMeth2(a, b).  Comparing the documentations of testMeth1 and 
testMeth2 is also irritating, since both take the same number of arguments, 
but it looks different.

c) (enhancement proposal) Finally, I would like to be able to specify several 
signatures for the same function.  Sometimes, it is possible to call a 
function with different types of objects / different keyword arguments / ..., 
and I would like to document two different kinds of use cases.  (Yes, this is 
a little bit like function overloading, not really pythonic and not very 
important to me either.)

d) It would be great it epydoc could extract signatures for boost::python 
exported functions, too (I am using boost 1.33.1 here, but I was told that 
1.34.0 makes no difference here). Alas, I don't know how that is possible - 
the only way I know of displaying the signature / getting the info is to 
actually *call* the functions (argh!) with a dummy argument and parse the 
resulting error message. :-(  One can define a dummy class and rely on the 
fact that no function will ever do something with e.g. 42 arguments of that 
type, but that procedure looks really brittle to me, although we use that for 
tab-completion in an interactive python shell (but there, only single 
functions are called on demand, so this is less dangerous).

Otherwise, epydoc really looks impressive to me (rst support, nice 
output, ...)! :-)

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Daniele V. <pi...@de...>]

Hello Hans,

> Today, I tried epydoc (3.0beta1) on one of my projects, and found several
> things that I miss / find strange:
>
> a) When parsing a signature that is in the first line of a function's
> docstring, epydoc fails to recognize signatures with default arguments (cf.
> test5 in the attached test file).  I later noticed that I could strip them
> from the docstrings since epydoc will extract the default values from the
> function signature nevertheless, but I still see this as a shortcoming.

there was a bug indeed in the signature parsing: the regexpr used was fooled
by the spaces around the '=' symbol (actually by the one before: the one after
was correctly recognized). This wasn't widely tested because the function
signature is conventionally expressed without spaces around the '=' symbol -
see PEP 8; nevertheless i have enforced the signature parsing in the trunk
rev1589.

Also notice that the PEP 275 discourages to duplicate the function signature
in the docstring for pure Python modules: it can be retrieved by
introspection.

> b) When documenting a method, I like to write an object name in front of the
> signature in order to be able to reference the object by name in the
> documentation.  E.g. "someFoo.testMeth2(a, b)" in the attached file.
> Interestingly, epydoc *does* parse the object name, but it does not display
> it properly but introduce a new parameter of that name.  That looks like a
> bug to me, even if you could argue that it is the name of the object passed
> as "self" / first argument.  However, IMO the signature documents the
> *usage*
> of a function and there you would not call "Foo.testMeth2(someFoo, a, b)"
> but
> someFoo.testMeth2(a, b).  Comparing the documentations of testMeth1 and
> testMeth2 is also irritating, since both take the same number of arguments,
> but it looks different.

This is something the author has deliberately decided to do. Anyway it bit
me too once: this "feature" also kicks in when documenting a module
function. IIRC the psycopg signature for the "connect()" function was
expressed as "psycopg.connect()" and it appeared as "connect(psycopg)". I
was maintaining psycopg documentation, not epydoc yet, so i preferred to
change the function signature.

Edward, what is the rationale behind the "selfparam" in the function
signature parsing? It seems to me that dropping it would produce better
results in most cases (even for a bound method expressed whose signature in
the docstring is expressed as

   inst.func(a, b)

reporting "func(a, b)" seems kinda more correct than "func(inst, a, b)",
even if the self argument is missing)

> c) (enhancement proposal) Finally, I would like to be able to specify
> several
> signatures for the same function.  Sometimes, it is possible to call a
> function with different types of objects / different keyword arguments /
> ...,
> and I would like to document two different kinds of use cases.  (Yes, this
> is
> a little bit like function overloading, not really pythonic and not very
> important to me either.)

mmm... this is definitely to be documented in the docstring body. Python
doesn't support methods overloading: i don't think Epydoc should document a
function "as it were implemented in C++".

> d) It would be great it epydoc could extract signatures for boost::python
> exported functions, too (I am using boost 1.33.1 here, but I was told that
> 1.34.0 makes no difference here). Alas, I don't know how that is possible -
> the only way I know of displaying the signature / getting the info is to
> actually *call* the functions (argh!) with a dummy argument and parse the
> resulting error message. :-(  One can define a dummy class and rely on the
> fact that no function will ever do something with e.g. 42 arguments of that
> type, but that procedure looks really brittle to me, although we use that
> for
> tab-completion in an interactive python shell (but there, only single
> functions are called on demand, so this is less dangerous).

I don't know much about boost::python bindings. Shooting random parameters
to an unknown signature doesn't seem fair to me :) sip generated bindings
share a similar characteristic (actually sip doesn't generate any docstring
at all, i don't know about boost). If you point me at an OS project built
with boost i may play a bit with such functions.

> Otherwise, epydoc really looks impressive to me (rst support, nice
> output, ...)! :-)

Thank you ;)

Ciao ciao

-- 
Daniele Varrazzo - Develer S.r.l.
http://www.develer.com

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Am Donnerstag, 10. Mai 2007 20:57:17 schrieb Daniele Varrazzo:
> Also notice that the PEP 275 discourages to duplicate the function
> signature in the docstring for pure Python modules: it can be retrieved by
> introspection.

Do you really mean "PEP 275: Switching on Multiple Values"?

..Ah, now I found it, it is: 257 - Docstring Conventions!
  http://www.python.org/dev/peps/pep-0257/

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Edward L. <ed...@gr...>]

On May 10, 2007, at 2:57 PM, Daniele Varrazzo wrote:
> Edward, what is the rationale behind the "selfparam" in the function
> signature parsing? It seems to me that dropping it would produce  
> better
> results in most cases (even for a bound method expressed whose  
> signature in
> the docstring is expressed as
>
>    inst.func(a, b)
>
> reporting "func(a, b)" seems kinda more correct than "func(inst, a,  
> b)",
> even if the self argument is missing)

I took the lead here from the python builtins:

     >>> print list.append.__doc__
     L.append(object) -- append object to end

It made most sense to me to render the docs for this as "append(L,  
object)" to be consistent with how non-builtin object methods are  
represented.  I'm not strongly attached to the decision, though, if  
there's support for changing this.  But I'd like the generated docs  
for the builtin objects to still make sense.

>> d) It would be great it epydoc could extract signatures for  
>> boost::python
>> exported functions, too [...]
>
> I don't know much about boost::python bindings. Shooting random  
> parameters
> to an unknown signature doesn't seem fair to me :) sip generated  
> bindings
> share a similar characteristic (actually sip doesn't generate any  
> docstring
> at all, i don't know about boost). If you point me at an OS project  
> built
> with boost i may play a bit with such functions.

Perhaps boost and/or sip could be encouraged to include some  
introspectable value that can be used to find the signature?  Either  
by automatically including a signature at the beginning of the  
__doc__ attribute, or by adding a new attribute to the object?  If  
they did that, I'd be happy to add support to epydoc for reading it.

-Edward

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi again!

Am Freitag, 11. Mai 2007 00:38:02 schrieb Edward Loper:
> Perhaps boost and/or sip could be encouraged to include some
> introspectable value that can be used to find the signature?  Either
> by automatically including a signature at the beginning of the
> __doc__ attribute, or by adding a new attribute to the object?  If
> they did that, I'd be happy to add support to epydoc for reading it.
I posted a request for that on the c++-sig list (boost::python development 
list), but so far, nobody volunteered:
http://mail.python.org/pipermail/c++-sig/2007-May/012355.html

(Yes, I know the signature-in-docstring sentence is wrong, since they are not 
discouraged in extension modules AFAICS.)

Would you be willing to add support for the extended docstrings with C++ 
signature?  Now that I built the new boost::python 1.34.0beta, I am swamped 
in docutils warnings:

+---------------------------------------------------------------------------------------------------------------
| File /home/meine/local-SuSE-10.2/lib/python2.4/hourglassmodule.so, in 
hourglass.DynamicCostQueue.empty
|   Warning: Unexpected indentation.
|
+---------------------------------------------------------------------------------------------------------------
| File /home/meine/local-SuSE-10.2/lib/python2.4/hourglassmodule.so, in 
hourglass.DynamicCostQueue.insert
|   Warning: Unexpected indentation.
|
...

This is because the docstrings now look like this one:
> Return True iff pop() can deliver another pair.
> C++ signature:
>     empty(vigra::DynamicCostQueue {lvalue}) -> bool

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi!

(Resending, first try went to Edward only.)

> On May 10, 2007, at 2:57 PM, Daniele Varrazzo wrote:
> > Edward, what is the rationale behind the "selfparam" in the function
> > signature parsing? It seems to me that dropping it would produce
> > better results in most cases (even for a bound method expressed whose
> > signature in the docstring is expressed as
> >
> >    inst.func(a, b)
> >
> > reporting "func(a, b)" seems kinda more correct than "func(inst, a,
> > b)", even if the self argument is missing)

Let me repeat myself here: There is a use for having the name in the docs, 
namely that you can *reference* the name in the docs.  I.e. 

"""myFoo.append(obj)

This method appends obj to the end of myFoo."""

is slightly clearer than 

"""append(obj)

This method appends obj to the end of this Foo object."""

Actually, this looks like a weak argument since I chose such a simple example, 
wait a moment... look at this:

            """someFace.holeCount() -> int
            
            Returns the number of holes.  This is equal to
            len(list(someFace.holeContours())), but faster."""

OK, I could rephrase it as 

            """Returns the number of holes.  someFace.holeCount() is equal to
            len(list(someFace.holeContours())), but faster."""

if you don't want to introduce the "objName." prefix in the epydoc output, 
which I could understand because it could look a little bit inconsistent 
(however, I would only expect that in the detailed docs anyways).

> I took the lead here from the python builtins:
>      >>> print list.append.__doc__
>
>      L.append(object) -- append object to end
>
> It made most sense to me to render the docs for this as "append(L,
> object)" to be consistent with how non-builtin object methods are
> represented.

I was about to write that I don't understand this, but now that I wrote the 
above, I can understand you.  (This does not have anything to do with builtin 
vs. non-builtin though, does it?)

> I'm not strongly attached to the decision, though, if 
> there's support for changing this.  But I'd like the generated docs
> for the builtin objects to still make sense.

As I said, I would prefer simply having the "L.append(object)" signature in 
the detailed HTML (not the class overview).  list.append.__doc__ is a bad 
example though, since it does not even reference 'L' in the rest of the 
docstring; I would expect sth. along
  "L.append(object) -- append object to L's end"

> >> d) It would be great it epydoc could extract signatures for
> >> boost::python
> >> exported functions, too [...]
> >
> > I don't know much about boost::python bindings. Shooting random
> > parameters
> > to an unknown signature doesn't seem fair to me :)

Same here.. ;-)

> Perhaps boost and/or sip could be encouraged to include some
> introspectable value that can be used to find the signature?  Either
> by automatically including a signature at the beginning of the
> __doc__ attribute, or by adding a new attribute to the object?  If
> they did that, I'd be happy to add support to epydoc for reading it.

Actually, I remember having read that the upcoming boost 1.34.0 has such a 
feature, that is why I asked a student to try that out, but he told me that 
it hadn't.  Still, a quick search brings up this:
   http://lists.boost.org/Archives/boost/2007/04/120397.php
..which tells us that "Boost.Python now automatically appends C++ signatures 
to docstrings.".  (Note that "C++ signatures" include full C++ type specs 
before the argument names, and may not even contain the names if they were 
not specified..)

I will try that out myself now and report back.

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Am Freitag, 11. Mai 2007 13:34:01 schrieb Hans Meine:
> Actually, I remember having read that the upcoming boost 1.34.0 has such a
> feature, that is why I asked a student to try that out, but he told me that
> it hadn't.  Still, a quick search brings up this:
>    http://lists.boost.org/Archives/boost/2007/04/120397.php
> ..which tells us that "Boost.Python now automatically appends C++
> signatures to docstrings.".  (Note that "C++ signatures" include full C++
> type specs before the argument names, and may not even contain the names if
> they were not specified..)
>
> I will try that out myself now and report back.

OK, after some struggling with the changed build system, which is incompatible
with our university's NFS layout, I found that it works indeed (no clue why I
was told that it does not work).  Here are some examples:

Function exported without any doc markup:
In [2]: applyFourierFilter.__doc__
Out[2]: 'C++ signature:\n    applyFourierFilter(vigra::PythonFixedBandsImage<float>, vigra::PythonFixedBandsImage<float>) -> vigra::PythonFixedBandsImage<vigra::TinyVector<float, 2> >'

The same, but a method of the class Diff2D:
In [4]: Diff2D.norm.__doc__
Out[4]: 'C++ signature:\n    norm(vigra::Diff2D {lvalue}) -> double'

Function exported with argument names, but no docstring:
In [5]: oversamplingGradient.__doc__
Out[5]: 'C++ signature:\n    oversamplingGradient(vigra::PythonSingleBandImage image, double sigma, bool withOffset=False) -> vigra::PythonFixedBandsImage<vigra::TinyVector<float, 2> >'

Function exported with both docstring (incl. signature) and argument names:
In [6]: susanDiscWindow.__doc__
Out[6]: 'susanDiscWindow(image, threshold, radius = 3.4, similarityExponent = 6.0)\nthreshold: grayvalue similarity threshold\nradius: radius of the circular window (default 3.4 -> 37 pixels)\nsimilarityExponent: exponent of the Gaussian similarity measure\nC++ signature:\n    susanDiscWindow(vigra::PythonImage image, double threshold, double radius=3.3999999999999999, double similarityExponent=6.0) -> vigra::PythonFixedBandsImage<float>'

That is, boost::python indeed *appends* the C++ signature to the docstring,
prepending a unique prefix as you can see.

There is support for switching the signatures off, and one can even disable
all docstrings, but the format seems to be fixed, hard-coded, and parsable:

> docstring_options(bool show_all=true);
>
> Effects: Constructs a docstring_options object which controls the
> appearance of function and member-function docstrings defined in the
> code that follows. If show_all is true, both the user-defined
> docstrings and the automatically generated C++ signatures are
> shown. If show_all is false the __doc__ attributes are None.
>
> docstring_options(bool show_user_defined, bool show_signatures);
>
> Effects: Constructs a docstring_options object which controls the
> appearance of function and member-function docstrings defined in the
> code that follows. Iff show_user_defined is true, the user-defined
> docstrings are shown. Iff show_signatures is true, C++ signatures are
> automatically added. If both show_user_defined and show_signatures are
> false, the __doc__ attributes are None.

Hope that helps.

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi again!

Am Donnerstag, 10. Mai 2007 20:57:17 schrieben Sie:
> Also notice that the PEP 275 discourages to duplicate the function
> signature in the docstring for pure Python modules: it can be retrieved by
> introspection.

That is interesting, I did not know that it was discouraged, but that makes 
sense indeed.

> > c) (enhancement proposal) Finally, I would like to be able to specify
> > several
> > signatures for the same function.  Sometimes, it is possible to call a
> > function with different types of objects / different keyword arguments /
> > ...,
> > and I would like to document two different kinds of use cases.  (Yes,
> > this is
> > a little bit like function overloading, not really pythonic and not very
> > important to me either.)
>
> mmm... this is definitely to be documented in the docstring body. Python
> doesn't support methods overloading: i don't think Epydoc should document a
> function "as it were implemented in C++".

I agree.  While I wrote the above, the idea looked less and less like a good 
one to me. ;-)  However, I now changed this docstring for example:

    """simplifyMapEdges(map, perpendicularDistEpsilon)
    simplifyMapEdges(map, perpendicularDistEpsilon, maxStep)

    Changes ..."""

like this to mark the last argument as optional:

    """simplifyMapEdges(map, perpendicularDistEpsilon[, maxStep])
    ..."""

epydoc recognizes this (cool!), but displays it with an ellipsis:

   simplifyMapEdges(map, perpendicularDistEpsilon, maxStep=...)

This is not really what I want; can epydoc be modified to display the optional 
argument in square brackets like in the docstring?

In case this is relevant, the Python definition is:

def simplifyMapEdges(map, *args):
    """simplifyMapEdges(map, perpendicularDistEpsilon[, maxStep])

    Simplifies all edges in-place ..."""
    pass

Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Daniele V. <pi...@de...>]

> I agree.  While I wrote the above, the idea looked less and less like a good
> one to me. ;-)  However, I now changed this docstring for example:
>
>     """simplifyMapEdges(map, perpendicularDistEpsilon)
>     simplifyMapEdges(map, perpendicularDistEpsilon, maxStep)
>
>     Changes ..."""
>
> like this to mark the last argument as optional:
>
>     """simplifyMapEdges(map, perpendicularDistEpsilon[, maxStep])
>     ..."""
>
> epydoc recognizes this (cool!), but displays it with an ellipsis:
>
>    simplifyMapEdges(map, perpendicularDistEpsilon, maxStep=...)
>
> This is not really what I want; can epydoc be modified to display the
> optional
> argument in square brackets like in the docstring?
>
> In case this is relevant, the Python definition is:
>
> def simplifyMapEdges(map, *args):
>     """simplifyMapEdges(map, perpendicularDistEpsilon[, maxStep])

Hello,

you should consider that not everybody has the same conventions to write the
signature. While a certain quantity of heuristics can be applied, i am sure
there will always be a funky example baffling the parser, unless a precise
grammar is specified.

The situation is worse if the signature description is actually different
from what can be introspected: in your latter example a function accepting a
list of positional argument is disguised behind a signature taking two
_named_ parameters, the second of which optional.

A more pythonic way to write your function would be, in my opinion::

    def simplifyMapEdges(map, perpendicularDistEpsilon, maxStep=None):
        """Simplifies all edges in-place.

        :Parameters:
          - `map`: the object to modify
          - `perpendicularDistEpsilon`: you know what it is.
          - `maxStep`: max number of steps. If omitted will be guessed.
        """

There is both a docstring conventions issue and a "pythonic" way to write a
function signature. You may have a good implementation reason to say "*arg"
instead of what i propose: if so, there's noting to be told. But i'd be more
"sincere" in this case and avoid the description of names which are in fact
not used: instead i'd describe in the docstring what the function really
takes as its input and how the caller should invoke. Or else somebody could
try to call your function as "simplifyMapEdges(mymap,
perpendicularDistEpsilon=42)", that will raise an exception.

If forced to keep your signature, i'd write a docstring such::

    def simplifyMapEdges(map, *args):
        """Simplifies all edges in-place.

        :Parameters:
          - `map`: the object to be updated
          - `args`: further positional arguments. At least the first is
            mandatory:

            1. the Perpendicular Distance Epsilon
            2. the max number of steps. If omitted will be guessed.

            Furhter parameters are ignored.
        """

I know that "pythonic conventions" are not the only ones existing, but they
are the most probable in a Python project, so Epydoc is tweaked to do its
best if they are applied and do something in between "what it can" and
"tries to survive" in all the other ones :)

I think you will obtain the best results avoiding to repeat yourself (adding
a signature where it can be introspected instead) and adding descriptions
really matching the "real situation" (which is also useful to detect
errors).

About a point you have raised somewhere else: when a docstring is more than
a paragraph long, only the first is printed in the summary table, while the
complete docstring content is put in a box below. You may found this, and
other useful hints, in the fine manual (tm) ;)

Regards :)

-- 
Daniele Varrazzo - Develer S.r.l.
http://www.develer.com

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Am Freitag, 11. Mai 2007 15:52:27 schrieb Daniele Varrazzo:
> About a point you have raised somewhere else: when a docstring is more than
> a paragraph long, only the first is printed in the summary table, while the
> complete docstring content is put in a box below. You may found this, and
> other useful hints, in the fine manual (tm) ;)

I know that (also from e.g. javadoc, Doxygen), and like it.

What I wanted to say is that I have found several examples where the latter 
(the complete docstring) is missing.  I.e. in the summary the first sentence 
is given, but it is *not* a link, and there is nothing to be linked to.  In 
the last mail I attached a simple test case that fails with epydoc-3.0beta1, 
and I just found another one:

def parseGeometry(geometryString):
	"""Convenience function for parsing geometry strings of various
	formats::

	  >>> fig.parseGeometry("320,240-640,480")
	  fig.Rect(320,240,640,480)
	  >>> fig.parseGeometry("50,50+50,50")
	  fig.Rect(50,50,100,100)
	  >>> fig.parseGeometry("40,40,320,240")
	  fig.Rect(40,40,320,240)
	  >>> r = fig.Rect(0,0,320,240)
	  >>> fig.parseGeometry(str(r)) == r
	  True
	"""
	ma = _re_geometry.match(geometryString)

These docs are not listed in the "function details" section either. :-(

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Daniele V. <pi...@de...>]

> and I just found another one:

> 

> def parseGeometry(geometryString):

> 	"""Convenience function for parsing geometry strings of various

> 	formats::

> 

> 	  >>> fig.parseGeometry("320,240-640,480")

> 	  fig.Rect(320,240,640,480)

> 	  >>> fig.parseGeometry("50,50+50,50")

> 	  fig.Rect(50,50,100,100)

> 	  >>> fig.parseGeometry("40,40,320,240")

> 	  fig.Rect(40,40,320,240)

> 	  >>> r = fig.Rect(0,0,320,240)

> 	  >>> fig.parseGeometry(str(r)) == r

> 	  True

> 	"""

> 	ma = _re_geometry.match(geometryString)



Ok: this seems a bug. After the first paragraph there is not another paragraph but a block: this is probably creating some confusion. The following works fine instead::



    def parseGeometry(geometryString):

        """Convenience function for parsing geometry strings of various

         formats. Examples are::

             ...



I'll investigate. Thank you.



-- 

Daniele Varrazzo - Develer S.r.l.

http://www.develer.com

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi Daniele!

Am Freitag, 11. Mai 2007 16:20:35 schrieb Daniele Varrazzo:
> Ok: this seems a bug. After the first paragraph there is not another
> paragraph but a block: this is probably creating some confusion.
Yeah, I already figured it was sth. like that.

(It is really amazing how much one has to adapt documentation to a specific 
documentation tool, or - to put it the other way round - how many possible 
ways of writing non-conventional documentation there are.  I already 
experienced the same with C++/Doxygen.)

BTW: I just ran epydoc on my fig.py module and put the documentation online, 
in case you are interested in another case study: 
http://kogs.informatik.uni-hamburg.de/~meine/software/figpy/

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi!

Am Dienstag, 15. Mai 2007 13:45:52 schrieb Daniele Varrazzo:
> To overcome this problem, i suggest you to add a module variable
> "__docformat__ = 'restructuredtext'" in your modules and to run epydoc with
> the "--docformat=plaintext" option. This way docstrings in boost-generated
> modules will not be parsed as reST.

Hehe, but I *want* to parse reST in the extension module.
I will use the attached patch for now.  That could be the right position to 
add further BPL (boost::python library) support, like the following:

   sig_re = _SIGNATURE_RE

    boost_signature_pos = doc_source.docstring.find("C++ signature:\n    ")
    if boost_signature_pos >= 0:
        boost_signature = doc_source.docstring[boost_signature_pos+19:]
        doc_source.docstring = doc_source.docstring[:boost_signature_pos]
        sig_re = _BOOST_SIGNATURE_RE # _SIGNATURE_RE + type support

   m = sig_re.match(doc_source.docstring)
   ...

Alas, there is one important case which I did not think about yet: Overloaded 
functions get multiple signatures appended:

'norm(vector)\nmagnitude(vector)\n\nCalculate the L2 norm / length of a vector 
(or scalar).\nC++ signature:\n    norm(vigra::PythonVectorView<float> 
vector) -> float\nC++ signature:\n    norm(double) -> double\nC++ signature:
\n    norm(vigra::TinyVector<double, 2>) -> double\nC++ signature:\n    
norm(vigra::TinyVector<double, 3>) -> double\nC++ signature:\n    
norm(vigra::TinyVector<double, 4>) -> double'

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi Edward!

Am Freitag, 11. Mai 2007 00:38:02 schrieb Edward Loper:
> Perhaps boost and/or sip could be encouraged to include some
> introspectable value that can be used to find the signature?  Either
> by automatically including a signature at the beginning of the
> __doc__ attribute, or by adding a new attribute to the object?  If
> they did that, I'd be happy to add support to epydoc for reading it.

I wrote a parser for the C++ signatures, using pyparsing.[*]
Even if the BPL people would not implement other support, that could bring 
users a working solution now, with boost 1.34.0 (CVS or beta, widely used).

I attach the code for you consideration; the epydoc_*.diff for 
docstringparser.py is far from finished, but hopefully shows what I mean.

[*] Of course, pyparsing would be a new dependency, but C++ signatures cannot 
be parsed with regular expressions at all, IMO.  I tried to make pyparsing be 
only loaded if epydoc is ran on code with C++ signatures embedded, so there 
could be a "pyparsing missing" warning just in that case.

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi again!

Am Freitag, 18. Mai 2007 16:28:43 schrieb Hans Meine:
> Am Freitag, 11. Mai 2007 00:38:02 schrieb Edward Loper:
> > Perhaps boost and/or sip could be encouraged to include some
> > introspectable value that can be used to find the signature?  Either
> > by automatically including a signature at the beginning of the
> > __doc__ attribute, or by adding a new attribute to the object?  If
> > they did that, I'd be happy to add support to epydoc for reading it.
>
> I wrote a parser for the C++ signatures, using pyparsing.[*]

I knew that the parser was not perfect when I wrote this, but I wanted to have 
some feedback.

Now I realized there is a specific flaw: In order to let my test suite run 
(parsing 206 random C++ signatures from a real-life extension module), I 
extended the parser to grok e.g. python list literals, which is needed for 
the default arguments.  However, I should have separated parsers for C++ 
literals (template arguments in the type specs) and python literals (default 
values). :-(

Alas, the only existing C++ parser I could find is part of 
http://synopsis.fresco.org/, and I would still need pyparsing for the special 
C++ signature format with mixed C++/python expressions.

BTW: How does epydoc handle default arguments with commas, e.g.

"""test(a, b, c = [1,2,3], d = None)

Foo bar baz..."""

AFAICS, that cannot be done with regular expressions very easily?! (OK, I 
chose numbers as list contents, which are no valid identifiers.  I could try 
harder though. ;-) )

-- 
Ciao, /  /
     /--/
    /  / ANS

Re: [Epydoc-devel] Docstring parsing shortcomings

[Hans M. <me...@in...>]

Hi Edward!

Am Freitag, 11. Mai 2007 16:10:46 schrieb Edward Loper:
> On May 11, 2007, at 10:05 AM, Hans Meine wrote:
> > What I wanted to say is that I have found several examples where
> > the latter (the complete docstring) is missing.
>
> Can you please file a bug report w/ these two examples?

Voila:
http://sourceforge.net/tracker/index.php?func=detail&aid=1732610&group_id=32455&atid=405618

-- 
Ciao, /  /
     /--/
    /  / ANS