Re: [Docutils-users] Programming reference documentation

[Beni C. <cb...@te...>]

On 2003-01-18, David Goodger wrote:

> I have begun work on a Python Source Reader component which will
> accomplish this.
>
Forgot to announce it; since the source reader is not ready for use (is
it?), I think somebody might benefit from my solution.  I've written code
that was documented with rST docstrings but couldn't use them (pydoc
ignores the rST, docutils barfs at the python code).  So I took the
plain-text formatter of pydoc and hacked it until the result was a
document with chapters instead of the "| " nesting, etc., so that it could
be processed by the plain rST reader of docutils.

It wasn't ideal, most notably interpretted text was just ignored instead
of becoming references (that could be quick-fixed, I think, for 80%
percent of the cases). In fact only references in the class hierarchy
(converted to a nested list) were explicitly turned into links and classes
were prepended with link targets.  There were lots of empty lines and the
text looked much less readable than the original pydoc's output.  After
processing with docutils this was hidden but the result didn't look much
better (if at all) than pydoc's html.  The most important benefit was that
I was able to validate my docstrings with docutils for being correct rST
(actually sometimes it was wrong because of the dumb conversion and I had
to change my docs to work around).

If somebody wants my code, I'll upload it later today.  Please don't read
it ;-), it was very quick and dirty - I just mutated the text formatter
instead of copying as a new formatter, I changed it only until it worked
on my docstrings so it could break on yours, etc.

> Ian Bicking wrote:
> > One thing I *don't* like is reference documentation in the style of
> > pydoc.To me it's too automatic, and generally too hard to follow.
> > I find reference documentation with some framework of text (lead-in
> > paragraphs, categorization, etc) to be much better.
>
Like GNU info files tend to be?  libc, elisp, make, they have the best
hierarchically organised documentation that I've ever read.  Puts
everything in the right place in your head after first reading but also
allows you to quickly read just what you need.

I'm now writing a library that I'd like to document very well, and info
files are my standard of excellence ;-).  So I want to figure out a way
for creating such documentation with docutils, too.

Now that I think of it, is anyone else interested in a GNU info writer
for docutils?  If yes, I'll append it to my task queue (i.e. don't hold
your breath).

> That can be tricky.One idea I had (documented in the spec/notes
> file) is to recognize JavaDoc-style documentation comments almost
> anywhere in the code.This would allow a more "literate programming"
> approach than what pydoc provides.
>
I'm not sure comments/strings in the code have anything to do with this.
It's about writing extra overview/design documentation tat doesn't
necessarily follow the code structure (and would be of higher quality if
the programmer is not tempted to bind them together for saving a few lines
of typing).

> > My vision for a docstring-based reference documentation would
> > probably look like:
> >
> > """
> > Module
> > ======
> >
> > This module does X, Y, Z.Most work will be done through the class:
> >
> > .. docstring:: MainClass
> >
> > In certain cases you may wish to access the functionality
> > separately, yada yada yada, and may wish to use:
> >
> > .. docstring:: some_function
> > .. docstring:: some_other_function
> > """
> >
> > class MainClass:
> >   """
> >   Recursively we continue, now with methods:
> >
> >   .. docstring:: MainClass.some_method
> >   """
>
Hmm, consider Leo outlines - I've never used it but it seems to be able to
do this kind of thing...

> Ienvision something more along the lines of this::
>
>   """
>   A general description of the highlights of the module, with
>   references (interpreted text) to the details.  We don't need
>   a "Module" title because it will be derived automatically.
>
>   This module does X, Y, Z.  Most work will be done through the
>   `MainClass` class.
>
>   In certain cases you may wish to access the functionality
>   separately, yada yada yada, and may wish to use `some_function`
>   or `some_other_function`
>   """
>
>   class MainClass:
>       """
>       Description of this class, with references to specific
>       methods like `some_method`.  We don't need to fully qualify
>       (no "MainClass." prefix) because "some_method" is in the
>       local namespace.
>       """
>
This are closest to what I was doing.  The module & class docstrings will
contain overview information, perhaphs even in many chapters, that would
link to relevant places in the low-level per-method documentation.

The problem with this approach is that it relies too much on hyperlinks
for a good order of reading.  The great thing about GNU info files is that
they are also organised in an order that is almost optimal for one-pass
front-to-back reading.  Ian Bicking's idea is better because it allows you
to specify the order of the documentation entries differently from the
order of the code.

>       ##
>       # My idea from JavaDoc is to allow document comments
>       # interspersed in the code in some way, which will
>       # interleave with the code objects and docstrings.
>       # These would include section titles, allowing a
>       # subdivision of methods by category.  For example:
>
>       ##
>       # Magic Methods
>       # =============
>
>       def __str__(self):
>           """String representation of this object (pseudo-XML)."""
>           ...
>
I think I see your idea.  But in many some cases this is short of what's
needed.  Consider (invented example) an application where many classes
contain special `gui_repr` methods that defines the user interface for
this object.  They are completely irrelevant to the algorithms described
in other methods, so a good manual will probably separate the
documentation of all GUI methods into a separate chapter discussing them
all at once.

So I think there should be (optional) ways to specify reordering /
reorganisation of nodes.  I'm not sure how to do it in a convenient yet
flexible way.  In the above gui case, for example, I would want to grab
all these methods without listing each one and risking omission.
Probably Python scripting to generate the reorganisation info would do the
trick but would not be equally applicable to non-python-source uses of
rST.

Also, for big documentation, it would make sense to hold the user-only
parts in separate files, leaving only what's relevant to the source's
design in the source.  The current include directive might solve this task
too, if the reorganisation is applied later.

See `(standards) GNU Manuals`__ for more thougths in this direction and
`(findutils) Top`__ for an example of a good manual that doesn't follow
the code and even describes several different programs together.

__ http://www.delorie.com/gnu/docs/GNU/standards_28.html
__ http://www.delorie.com/gnu/docs/findutils/find_toc.html

-- 
Beni Cherniavsky <cb...@tx...>

There is an Excel spreadsheet here.  Do you want to open it? y
There was a grid bug in the spreadsheet.  The grid bug bites.