docutils-develop — For developer discussions of the implementation.

Re: [Docutils-develop] Interpreted text for terms?

[Benja F. <b.f...@gm...>]

Hi Mark,

Mark Nodine wrote:
> In this example, the term is actually defined in the text
> of the document (rather than a separate glossary).  It is
> typeset as italics, but, and this is the key, it also creates
> an implicit target so that I can refer to this term from
> elsewhere in the document (and especially from the index).
> 
> Is there a way to make interpreted text (perhaps with specific
> role types) create implicit targets?  Or is there some other
> way I should be going about this?

Hey, if you don't need italics, how about using explicit targets? :) 
Like this:

     An _`inline target` looks almost like a hyperlink.

Doesn't look more wordy than any interpreted text, to me ;)

- Benja

Re: [Docutils-develop] Interpreted text for terms?

[Mark N. <no...@so...>]

David Goodger wrote:
> 
> Benja Fallenstein wrote:
> > I'm currently writing two articles using ReST, and regarding the recent
> > interpreted text roles, I have noticed one role I could use: 'terms.'
> > For example, I'm writing about something called 'nomadicity.' In the
> > printed/HTML version, I'd italicize this, yet it's not really an
> > emphasis; at least in plain text, I wouldn't write *nomadicity* (maybe
> > I'd use single quotes as above).
> >
> > Any chance this could be an interpreted text role? It would probably
> > have to have an unobstrusive alias, two chars at most, for me to use
> > this... at least I think so.
> 
> I think that depends on its intended use.  In `The Chicago Manual of Style`,
> 14th ed., sections 6.71-74 deal with "special terminology".  If all that is
> desired is the typographical effect, I don't see enough of a distinction
> between a "term" and an "emphasized word or phrase" to warrant a separate
> element.  One of the traditional uses of italics as emphasis is for this
> kind of term.  I don't want to let in all kinds of inline elements with
> marginal uses.  I would first need to see solid evidence (use cases,
> references, etc.) and see that there's a widespread need.  It'll go on the
> "to-do?" list for now.
> 
> On the other hand, if the term is meant to be defined in a glossary, I think
> it would be a useful addition: a "glossary term".  However, there *would*
> have to be a glossary for the term to refer to; the lack of one would be
> flagged as an error.  The term in the text would link to the entry in the
> glossary proper.  A "glossary-entry" directive could allow entries to be
> defined anywhere in the document, and collected (via a "glossary" directive
> perhaps) at one point.

If the item is defined in a glossary, you can simply
make the term a link to the glossary entry.  What I'm
wondering how to do in RST is what was represented
between tildes in structure-enhanced text.  It is 
obviously closely related to interpreted text, and
in fact, I always interpreted it as terms to be included
in the index.  For example, ::

  A ~behavioral bus monitor~ is a module written in the same 
  language as the design (such as Verilog) to
  monitor the signals on the bus and produce a warning if the 
  conditions they check are violated.

In this example, the term is actually defined in the text
of the document (rather than a separate glossary).  It is
typeset as italics, but, and this is the key, it also creates
an implicit target so that I can refer to this term from
elsewhere in the document (and especially from the index).

Is there a way to make interpreted text (perhaps with specific
role types) create implicit targets?  Or is there some other
way I should be going about this?

	--Mark

Re: [Docutils-develop] Re: [Docutils-users] Programming reference documentation

[Ian B. <ia...@co...>]

On Sun, 2003-01-19 at 06:00, Beni Cherniavsky wrote:
> Forgot to announce it; since the source reader is not ready for use (is
> it?),

It seems to work okay for me -- I think David might have some features
he'd like, but what's there seems sufficient to me.

>  I think somebody might benefit from my solution.  I've written code
> that was documented with rST docstrings but couldn't use them (pydoc
> ignores the rST, docutils barfs at the python code).  So I took the
> plain-text formatter of pydoc and hacked it until the result was a
> document with chapters instead of the "| " nesting, etc., so that it could
> be processed by the plain rST reader of docutils.

Ack... sounds like a mess.

> > Ian Bicking wrote:
> > > One thing I *don't* like is reference documentation in the style of
> > > pydoc.To me it's too automatic, and generally too hard to follow.
> > > I find reference documentation with some framework of text (lead-in
> > > paragraphs, categorization, etc) to be much better.
> >
> Like GNU info files tend to be?  libc, elisp, make, they have the best
> hierarchically organised documentation that I've ever read.  Puts
> everything in the right place in your head after first reading but also
> allows you to quickly read just what you need.
> 
> I'm now writing a library that I'd like to document very well, and info
> files are my standard of excellence ;-).  So I want to figure out a way
> for creating such documentation with docutils, too.

Documenting a library is different then a program, in my mind.  Or at
least there should be some entirely thorough but not necessarily
intuitively structured documentation.  In Webware I'm also making
documents that are task and instruction oriented.  But they don't make
up for the reference documentation that you want when you're already
familiar with the library, or at least the concept behind the library,
and you just want to know how to spell the method name or something.

This is how I use the Python reference documentation -- which on the
whole is quite good -- and is how I'd expect to use a docstring
extractor.  I just write the other documents in a separate file.

> Now that I think of it, is anyone else interested in a GNU info writer
> for docutils?  If yes, I'll append it to my task queue (i.e. don't hold
> your breath).

Info documents are often well organized, but the actual reader I've
never been impressed with.  I get lost too easily.  But web pages
certainly aren't ideal either.

> >
> >   """
> >   A general description of the highlights of the module, with
> >   references (interpreted text) to the details.  We don't need
> >   a "Module" title because it will be derived automatically.
> >
> >   This module does X, Y, Z.  Most work will be done through the
> >   `MainClass` class.
> >
> >   In certain cases you may wish to access the functionality
> >   separately, yada yada yada, and may wish to use `some_function`
> >   or `some_other_function`
> >   """
> >
> >   class MainClass:
> >       """
> >       Description of this class, with references to specific
> >       methods like `some_method`.  We don't need to fully qualify
> >       (no "MainClass." prefix) because "some_method" is in the
> >       local namespace.
> >       """
> >
> This are closest to what I was doing.  The module & class docstrings will
> contain overview information, perhaphs even in many chapters, that would
> link to relevant places in the low-level per-method documentation.
> 
> The problem with this approach is that it relies too much on hyperlinks
> for a good order of reading.  The great thing about GNU info files is that
> they are also organised in an order that is almost optimal for one-pass
> front-to-back reading.  Ian Bicking's idea is better because it allows you
> to specify the order of the documentation entries differently from the
> order of the code.

I'm realizing that reordering the methods might be best anyway --
usually they don't match up with the description because they were just
written in a certain order.

The whole hyperlinking thing I'm still unsure about.  I don't even know
if there's a way to indicate that `references` should be turned into
some sort of link.

> >       ##
> >       # My idea from JavaDoc is to allow document comments
> >       # interspersed in the code in some way, which will
> >       # interleave with the code objects and docstrings.
> >       # These would include section titles, allowing a
> >       # subdivision of methods by category.  For example:
> >
> >       ##
> >       # Magic Methods
> >       # =============
> >
> >       def __str__(self):
> >           """String representation of this object (pseudo-XML)."""
> >           ...
> >
> I think I see your idea.  But in many some cases this is short of what's
> needed.  Consider (invented example) an application where many classes
> contain special `gui_repr` methods that defines the user interface for
> this object.  They are completely irrelevant to the algorithms described
> in other methods, so a good manual will probably separate the
> documentation of all GUI methods into a separate chapter discussing them
> all at once.

When you start getting into that level or organization you have to have
an application-wide view for the document.  I'm just thinking of modules
now -- but there certainly are many modules that aren't well contained.

> So I think there should be (optional) ways to specify reordering /
> reorganisation of nodes.  I'm not sure how to do it in a convenient yet
> flexible way.  In the above gui case, for example, I would want to grab
> all these methods without listing each one and risking omission.
> Probably Python scripting to generate the reorganisation info would do the
> trick but would not be equally applicable to non-python-source uses of
> rST.

I know... I've already thought of several packages I have that I'd like
to autogenerate the documentation, but the format needs to be structured
different from how it generally would be.  For instance, in one library
I have error strings that can be overridden -- these could be extracted,
but only by a program that understood how my specific code was
structured.

> Also, for big documentation, it would make sense to hold the user-only
> parts in separate files, leaving only what's relevant to the source's
> design in the source.  The current include directive might solve this task
> too, if the reorganisation is applied later.

I'm not sure what you mean here...?   

-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

Re: [Docutils-develop] extractor.py

[Ian B. <ia...@co...>]

After documenting a larger module from Webware, I realized that the
including thing was kind of annoying.  Or, rather, I just found myself
including things in order (or wanting to reorder the definitions).

So, while including is still supported, now all methods and functions
get included automatically.  But intermediate docstrings can be used to
provide commentary, like:

    """
    SomeModule does lots of stuff
    """

    def stuff(): whatever

    """
    The following functions are dangerous!
    """

    def dangerousstuff(): whatever

You can still do .. inline: ..., and that will suppress the automatic
insertion of that function or method.  You can also .. ignore: ... the
function, or put :ignore: in the docstring.  I just realized it should
parse __all__ for hints too -- in most cases anything not in __all__ can
be ignored.  However, some class instances may only be created
indirectly, so there'd have to be a way to override that.  

Anyway, I feel like this brings the extracting into decent shape for
me.  I'll try using it again tomorrow, which will be the real test.  The
next thing I see is dealing with inheritance (since multiple
modules/classes come together to make one interface, and I'd like it
documented centrally but from docstrings that aren't centrally
located).  Then other cross-module references, which I think is still an
open issue for reST...?

Altogether, though, things seem to be coming together fairly well.

-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

[Docutils-develop] extractor.py

[Ian B. <ia...@co...>]

I wrote a document generator using the python reader, with the features
I talked about recently.  It's in /sandbox/ianb/extractor/extractor.py
-- I also updated the documentation of /sandbox/ianb/wiki/Wiki.py so it
uses those features (so it's a test case).

It works pretty well, I think.  It adds the directives:
  .. inline: function_or_class
  .. inline-all:
  .. ignore: function_or_class
and uses the attribute :ignore: to signify a method's docstring should
not be included (when using inline-all).

They aren't real directives -- I just implemented them as a
preprocessor.  That's obviously not the best way, but it was expedient,
especially since it means I didn't have to delve into the parser code
with which I am unfamiliar.

It certainly needs improvement in a number of areas (for instance,
there's no documentation of global variables or attributes), but it's
sufficient for my purposes, especially since the Webware style uses only
functions, classes, and methods publicly.  I may start using it just as
it is.

Another thing that occurred to me as a problem is new-style classes and
metaclasses.  These create classes whose interface is not represented
well through the code -- for instance, you have properties with
docstrings that are run-time created (though potentially you could parse
the property statement for most cases).  And metaclasses can do much
worse.  With the technique used in extractor.py (explicit inclusion)
it's easy enough to add documentation to the class or module for those
features that aren't represented outwardly in the source.  I do that in
Wiki.py to describe the properties.


-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

Re: [Docutils-develop] Interpreted text for terms?

[David G. <go...@py...>]

Benja Fallenstein wrote:
> What would you recomment for now? Asterisk emphasis?

Yes.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Interpreted text for terms?

[Benja F. <b.f...@gm...>]

David Goodger wrote:
> Benja Fallenstein wrote:
> 
>>I'm currently writing two articles using ReST, and regarding the recent
>>interpreted text roles, I have noticed one role I could use: 'terms.'
>>For example, I'm writing about something called 'nomadicity.' In the
>>printed/HTML version, I'd italicize this, yet it's not really an
>>emphasis; at least in plain text, I wouldn't write *nomadicity* (maybe
>>I'd use single quotes as above).
>>
>>Any chance this could be an interpreted text role? It would probably
>>have to have an unobstrusive alias, two chars at most, for me to use
>>this... at least I think so.
> 
> 
> I think that depends on its intended use.  In `The Chicago Manual of Style`,
> 14th ed., sections 6.71-74 deal with "special terminology".  If all that is
> desired is the typographical effect, I don't see enough of a distinction
> between a "term" and an "emphasized word or phrase" to warrant a separate
> element.  One of the traditional uses of italics as emphasis is for this
> kind of term.  I don't want to let in all kinds of inline elements with
> marginal uses.  I would first need to see solid evidence (use cases,
> references, etc.) and see that there's a widespread need.  It'll go on the
> "to-do?" list for now.

What would you recomment for now? Asterisk emphasis?

Thanks,
- Benja

Re: [Docutils-develop] Interpreted text for terms?

[David G. <go...@py...>]

Benja Fallenstein wrote:
> I'm currently writing two articles using ReST, and regarding the recent
> interpreted text roles, I have noticed one role I could use: 'terms.'
> For example, I'm writing about something called 'nomadicity.' In the
> printed/HTML version, I'd italicize this, yet it's not really an
> emphasis; at least in plain text, I wouldn't write *nomadicity* (maybe
> I'd use single quotes as above).
>
> Any chance this could be an interpreted text role? It would probably
> have to have an unobstrusive alias, two chars at most, for me to use
> this... at least I think so.

I think that depends on its intended use.  In `The Chicago Manual of Style`,
14th ed., sections 6.71-74 deal with "special terminology".  If all that is
desired is the typographical effect, I don't see enough of a distinction
between a "term" and an "emphasized word or phrase" to warrant a separate
element.  One of the traditional uses of italics as emphasis is for this
kind of term.  I don't want to let in all kinds of inline elements with
marginal uses.  I would first need to see solid evidence (use cases,
references, etc.) and see that there's a widespread need.  It'll go on the
"to-do?" list for now.

On the other hand, if the term is meant to be defined in a glossary, I think
it would be a useful addition: a "glossary term".  However, there *would*
have to be a glossary for the term to refer to; the lack of one would be
flagged as an error.  The term in the text would link to the entry in the
glossary proper.  A "glossary-entry" directive could allow entries to be
defined anywhere in the document, and collected (via a "glossary" directive
perhaps) at one point.

DocBook contains two related elements, "firstterm" and "glossterm".  The
"firstterm" element (which seems to be close to what is being asked for) can
link to a glossary entry, but doesn't have to.

-- David Goodger    http://starship.python.net/~goodger

[Docutils-develop] Interpreted text for terms?

[Benja F. <b.f...@gm...>]

Hi,

I'm currently writing two articles using ReST, and regarding the recent 
interpreted text roles, I have noticed one role I could use: 'terms.' 
For example, I'm writing about something called 'nomadicity.' In the 
printed/HTML version, I'd italicize this, yet it's not really an 
emphasis; at least in plain text, I wouldn't write *nomadicity* (maybe 
I'd use single quotes as above).

Any chance this could be an interpreted text role? It would probably 
have to have an unobstrusive alias, two chars at most, for me to use 
this... at least I think so.

Thanks,
- Benja

[Docutils-develop] Re: [Docutils-users] Programming reference documentation

[Beni C. <cb...@te...>]

On 2003-01-18, David Goodger wrote:

> I have begun work on a Python Source Reader component which will
> accomplish this.
>
Forgot to announce it; since the source reader is not ready for use (is
it?), I think somebody might benefit from my solution.  I've written code
that was documented with rST docstrings but couldn't use them (pydoc
ignores the rST, docutils barfs at the python code).  So I took the
plain-text formatter of pydoc and hacked it until the result was a
document with chapters instead of the "| " nesting, etc., so that it could
be processed by the plain rST reader of docutils.

It wasn't ideal, most notably interpretted text was just ignored instead
of becoming references (that could be quick-fixed, I think, for 80%
percent of the cases). In fact only references in the class hierarchy
(converted to a nested list) were explicitly turned into links and classes
were prepended with link targets.  There were lots of empty lines and the
text looked much less readable than the original pydoc's output.  After
processing with docutils this was hidden but the result didn't look much
better (if at all) than pydoc's html.  The most important benefit was that
I was able to validate my docstrings with docutils for being correct rST
(actually sometimes it was wrong because of the dumb conversion and I had
to change my docs to work around).

If somebody wants my code, I'll upload it later today.  Please don't read
it ;-), it was very quick and dirty - I just mutated the text formatter
instead of copying as a new formatter, I changed it only until it worked
on my docstrings so it could break on yours, etc.

> Ian Bicking wrote:
> > One thing I *don't* like is reference documentation in the style of
> > pydoc.To me it's too automatic, and generally too hard to follow.
> > I find reference documentation with some framework of text (lead-in
> > paragraphs, categorization, etc) to be much better.
>
Like GNU info files tend to be?  libc, elisp, make, they have the best
hierarchically organised documentation that I've ever read.  Puts
everything in the right place in your head after first reading but also
allows you to quickly read just what you need.

I'm now writing a library that I'd like to document very well, and info
files are my standard of excellence ;-).  So I want to figure out a way
for creating such documentation with docutils, too.

Now that I think of it, is anyone else interested in a GNU info writer
for docutils?  If yes, I'll append it to my task queue (i.e. don't hold
your breath).

> That can be tricky.One idea I had (documented in the spec/notes
> file) is to recognize JavaDoc-style documentation comments almost
> anywhere in the code.This would allow a more "literate programming"
> approach than what pydoc provides.
>
I'm not sure comments/strings in the code have anything to do with this.
It's about writing extra overview/design documentation tat doesn't
necessarily follow the code structure (and would be of higher quality if
the programmer is not tempted to bind them together for saving a few lines
of typing).

> > My vision for a docstring-based reference documentation would
> > probably look like:
> >
> > """
> > Module
> > ======
> >
> > This module does X, Y, Z.Most work will be done through the class:
> >
> > .. docstring:: MainClass
> >
> > In certain cases you may wish to access the functionality
> > separately, yada yada yada, and may wish to use:
> >
> > .. docstring:: some_function
> > .. docstring:: some_other_function
> > """
> >
> > class MainClass:
> >   """
> >   Recursively we continue, now with methods:
> >
> >   .. docstring:: MainClass.some_method
> >   """
>
Hmm, consider Leo outlines - I've never used it but it seems to be able to
do this kind of thing...

> Ienvision something more along the lines of this::
>
>   """
>   A general description of the highlights of the module, with
>   references (interpreted text) to the details.  We don't need
>   a "Module" title because it will be derived automatically.
>
>   This module does X, Y, Z.  Most work will be done through the
>   `MainClass` class.
>
>   In certain cases you may wish to access the functionality
>   separately, yada yada yada, and may wish to use `some_function`
>   or `some_other_function`
>   """
>
>   class MainClass:
>       """
>       Description of this class, with references to specific
>       methods like `some_method`.  We don't need to fully qualify
>       (no "MainClass." prefix) because "some_method" is in the
>       local namespace.
>       """
>
This are closest to what I was doing.  The module & class docstrings will
contain overview information, perhaphs even in many chapters, that would
link to relevant places in the low-level per-method documentation.

The problem with this approach is that it relies too much on hyperlinks
for a good order of reading.  The great thing about GNU info files is that
they are also organised in an order that is almost optimal for one-pass
front-to-back reading.  Ian Bicking's idea is better because it allows you
to specify the order of the documentation entries differently from the
order of the code.

>       ##
>       # My idea from JavaDoc is to allow document comments
>       # interspersed in the code in some way, which will
>       # interleave with the code objects and docstrings.
>       # These would include section titles, allowing a
>       # subdivision of methods by category.  For example:
>
>       ##
>       # Magic Methods
>       # =============
>
>       def __str__(self):
>           """String representation of this object (pseudo-XML)."""
>           ...
>
I think I see your idea.  But in many some cases this is short of what's
needed.  Consider (invented example) an application where many classes
contain special `gui_repr` methods that defines the user interface for
this object.  They are completely irrelevant to the algorithms described
in other methods, so a good manual will probably separate the
documentation of all GUI methods into a separate chapter discussing them
all at once.

So I think there should be (optional) ways to specify reordering /
reorganisation of nodes.  I'm not sure how to do it in a convenient yet
flexible way.  In the above gui case, for example, I would want to grab
all these methods without listing each one and risking omission.
Probably Python scripting to generate the reorganisation info would do the
trick but would not be equally applicable to non-python-source uses of
rST.

Also, for big documentation, it would make sense to hold the user-only
parts in separate files, leaving only what's relevant to the source's
design in the source.  The current include directive might solve this task
too, if the reorganisation is applied later.

See `(standards) GNU Manuals`__ for more thougths in this direction and
`(findutils) Top`__ for an example of a good manual that doesn't follow
the code and even describes several different programs together.

__ http://www.delorie.com/gnu/docs/GNU/standards_28.html
__ http://www.delorie.com/gnu/docs/findutils/find_toc.html

-- 
Beni Cherniavsky <cb...@tx...>

There is an Excel spreadsheet here.  Do you want to open it? y
There was a grid bug in the spreadsheet.  The grid bug bites.

[Docutils-develop] Re: [Docutils-users] Programming reference documentation

[Ian B. <ia...@co...>]

On Sat, 2003-01-18 at 10:34, David Goodger wrote:
> > I'm working with the Webware project (webware.sf.net), and we really
> > need some cohesive reference documentation, aimed particularly at
> > the public interfaces.  I'm wondering if anyone is using docutils
> > for this now... if so, I'd like to see how it's working.  It's not
> > entirely clear to me what docutils status is (I've used reST a lot,
> > but not much else).
> 
> Are you referring to "auto-documentation" (extracting documentation
> from source code)?  I have begun work on a Python Source Reader
> component which will accomplish this.  The code so far is in
> docutils/readers/python/moduleparser.py.  The plan is described in PEP
> 258 (<http://docutils.sf.net/spec/pep-0258.html>),
> <http://docutils.sf.net/spec/pysource.html>, and
> <http://docutils.sf.net/spec/notes.html#python-source-reader>.

Yes, I'm thinking of auto-documentation, or at least documentation with
an automated step.

> The Python Reader package doesn't extract docstrings yet.  I'm
> currently concentrating on finding a new job [#]_, which means I don't
> know how much time I'll be able to devote to this.
> 
> .. [#] Check out my resume at
>    <http://starship.python.net/~dgoodger/cv/>!  Please tell your
>    friends, relatives, neighbors, managers, and HR people!  Let me
>    know if you have any leads.  Thanks!

You're in Canada, right?  My scant connections don't go as far as
Canada, but who knows...

> > * Using internal references in those docstrings, so I could
> >   effectively include the docstrings from other portions of code
> >   (classes, methods, etc).
> 
> I hadn't thought of that approach.  From your description, it seems to
> require a lot of bookkeeping and redundancy, which I wouldn't want to
> write or require others to write.  But I encourage you to explore it.

It's about the same amount of bookkeeping that you'd need to create a
high-quality reference document (like the Python module documentation,
for instance).  It should be about as difficult to write.  The advantage
would be entirely in the maintenance, by having code and documentation
next to each other.

Some of the bookkeeping might be possible to ease -- for instance,
keeping track of methods that aren't included and giving a warning (so
you don't forget to include documentation you wrote).  Or maybe have a
directive that would just include all the documentation of the class or
module -- either ordered like in the source, or ordered alphabetically. 

> > One thing I *don't* like is reference documentation in the style of
> > pydoc.  To me it's too automatic, and generally too hard to follow.
> 
> I hope that Docutils' auto-documentation will be more flexible than
> pydoc's, although it will have the same basis.  It will certainly have
> more flexible rendering, and I hope much more readable.
> 
> > I find reference documentation with some framework of text (lead-in
> > paragraphs, categorization, etc) to be much better.
> 
> That can be tricky.  One idea I had (documented in the spec/notes
> file) is to recognize JavaDoc-style documentation comments almost
> anywhere in the code.  This would allow a more "literate programming"
> approach than what pydoc provides.

If everything is ordered like in the source, perhaps a comment (or plain
string) could be parsed out as in-between text.  Like:

"""
These functions are all dangerous:
"""

def yadayada():
    "docs for this"

And so on... they could appear in class definitions as well.  That they
are strings implies they are different than mere comments -- most of the
comments I write are intended to be read by someone reading the code,
and I have no desire to separate them into documentation; I'd be willing
to make that distinction strict in the code I work with.

But that's harder to work with than inclusion, because your narrative is
split up all over the place with lots of code in between.  

> > There are also
> > methods which, while "public" aren't actually useful, and I don't
> > think those should be documented -- they are just distracting.
> 
> How could software tell the difference?  If we can come up with an
> unobtrusive way to tell the docstring processing system to disregard
> certain methods, it could be useful.  Maybe a class attribute
> equivalent of the "__all__" module attribute?  Or perhaps the
> opposite: a "__none__" class attribute ;).

Well, there could be a directive in the docstring like:

  .. public: no

Or I guess it would be more appropriate:

  :Public: no

"Public" probably isn't the right term, but something like that. 
":Obscure: yes" might be more correct ;)

-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

[Docutils-develop] my resume URL

[David G. <go...@py...>]

Sorry, mistyped the URL.  It should be
<http://starship.python.net/~goodger/cv/>.  Thanks again!

-- David Goodger    http://starship.python.net/~goodger

[Docutils-develop] Re: [Docutils-users] Programming reference documentation

[David G. <go...@py...>]

I'm copying this to docutils-develop, a more appropriate venue for the
discussion IMO.  Subscribe at
<http://lists.sf.net/lists/listinfo/docutils-develop>.

Ian Bicking wrote (quoted in full):
> I'm working with the Webware project (webware.sf.net), and we really
> need some cohesive reference documentation, aimed particularly at
> the public interfaces.  I'm wondering if anyone is using docutils
> for this now... if so, I'd like to see how it's working.  It's not
> entirely clear to me what docutils status is (I've used reST a lot,
> but not much else).

Are you referring to "auto-documentation" (extracting documentation
from source code)?  I have begun work on a Python Source Reader
component which will accomplish this.  The code so far is in
docutils/readers/python/moduleparser.py.  The plan is described in PEP
258 (<http://docutils.sf.net/spec/pep-0258.html>),
<http://docutils.sf.net/spec/pysource.html>, and
<http://docutils.sf.net/spec/notes.html#python-source-reader>.

> Some options I've considered, in order of complexity:
> * Writing the whole thing in reST, separate from the code itself.
>   The writing wouldn't be so bad, but the maintenance would be more
>   difficult.

This can be done now of course.

> * Splitting it up into modules, and putting the entire reference
>   documentation in the module docstring.

The Python Reader package doesn't extract docstrings yet.  I'm
currently concentrating on finding a new job [#]_, which means I don't
know how much time I'll be able to devote to this.

.. [#] Check out my resume at
   <http://starship.python.net/~dgoodger/cv/>!  Please tell your
   friends, relatives, neighbors, managers, and HR people!  Let me
   know if you have any leads.  Thanks!

> * Using internal references in those docstrings, so I could
>   effectively include the docstrings from other portions of code
>   (classes, methods, etc).

I hadn't thought of that approach.  From your description, it seems to
require a lot of bookkeeping and redundancy, which I wouldn't want to
write or require others to write.  But I encourage you to explore it.

> One thing I *don't* like is reference documentation in the style of
> pydoc.  To me it's too automatic, and generally too hard to follow.

I hope that Docutils' auto-documentation will be more flexible than
pydoc's, although it will have the same basis.  It will certainly have
more flexible rendering, and I hope much more readable.

> I find reference documentation with some framework of text (lead-in
> paragraphs, categorization, etc) to be much better.

That can be tricky.  One idea I had (documented in the spec/notes
file) is to recognize JavaDoc-style documentation comments almost
anywhere in the code.  This would allow a more "literate programming"
approach than what pydoc provides.

> There are also
> methods which, while "public" aren't actually useful, and I don't
> think those should be documented -- they are just distracting.

How could software tell the difference?  If we can come up with an
unobtrusive way to tell the docstring processing system to disregard
certain methods, it could be useful.  Maybe a class attribute
equivalent of the "__all__" module attribute?  Or perhaps the
opposite: a "__none__" class attribute ;).

> So the last option is the one I like the most.  But I doubt it's in
> place.

Correct, it's not.

> I've already decided I'm not going to get the reference
> documentation done for the next release (there's other documentation
> that needs to be written, and documentation is tiring :), so I don't
> plan to use this in the short term and I could put some work into
> producing a system I like.

Glad for the help!

> My vision for a docstring-based reference documentation would
> probably look like:
> 
> """
> Module
> ======
> 
> This module does X, Y, Z.  Most work will be done through the class:
> 
> .. docstring:: MainClass
> 
> In certain cases you may wish to access the functionality
> separately, yada yada yada, and may wish to use:
> 
> .. docstring:: some_function
> .. docstring:: some_other_function
> """
> 
> class MainClass:
>     """
>     Recursively we continue, now with methods:
> 
>     .. docstring:: MainClass.some_method
>     """

I envision something more along the lines of this::

    """
    A general description of the highlights of the module, with
    references (interpreted text) to the details.  We don't need
    a "Module" title because it will be derived automatically.
    
    This module does X, Y, Z.  Most work will be done through the
    `MainClass` class.
    
    In certain cases you may wish to access the functionality
    separately, yada yada yada, and may wish to use `some_function`
    or `some_other_function`
    """
    
    class MainClass:
        """
        Description of this class, with references to specific
        methods like `some_method`.  We don't need to fully qualify
        (no "MainClass." prefix) because "some_method" is in the
        local namespace.
        """

        ##
        # My idea from JavaDoc is to allow document comments
        # interspersed in the code in some way, which will
        # interleave with the code objects and docstrings.
        # These would include section titles, allowing a
        # subdivision of methods by category.  For example:
        
        ##
        # Magic Methods
        # =============
        
        def __str__(self):
            """String representation of this object (pseudo-XML)."""
            ...

> How far away might this be?

A function of my spare time and volunteer contributions.  Hint hint.

> Are there better alternatives already in place, or planned on?

Alternative software packages?  There's HappyDoc, EpyDoc, effbot's
PythonDoc, and lots of older stuff too; see PEP 256.  None use
reStructuredText though.

> Does someone experienced in writing
> reference documentation think this is the wrong way to go about it?

ISTM that there's a built-in limit to the potential quality of
auto-documentation.  It may not lend itself to a more narrative or
free-form style.  However, I hope that Docutils will be a platform for
experimentation.  That's why I'm keeping the processing stages
separate and well-defined.  See
<http://article.gmane.org/gmane.text.docutils.devel/139> and the
thread starting with
<http://article.gmane.org/gmane.text.docutils.devel/118>.

-- 
David Goodger    http://starship.python.net/~goodger    Projects:
  * Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  * The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Proposed change to RST parser:justification in simple tables

[Mark N. <no...@so...>]

eng...@ss... wrote:
> 
> On Fri, 17 Jan 2003, Richard Jones wrote:
> 
> > On Fri, 17 Jan 2003 2:08 pm, Richard Jones wrote:
> > > On Fri, 17 Jan 2003 12:25 pm, Benja Fallenstein wrote:
> > > > David Goodger wrote:
> > > > > My first knee-jerk objection was, what about when the text begins in
> > > > > the leftmost *and* ends in the rightmost column? ::
> > > > >
> > > > >     ==========  ===
> > > > >     Name        Age
> > > > >     ==========  ===
> > > > >     David        34
> > > > >     Methuselah  969
> > > > >     ==========  ===
> 
> this one would have 34 right aligned and 969 left or should alignment be a
> columnsetting.
> 
> BTW: what about
>     =========== =======
>     Name          Val
>     =========== =======
>     me             1.65
>     you           10.60
>     =========== =======
> 
> what would Val and of course i want alignement on the decimal.

My original proposal considers alignment on a cell basis, so that
"Val" would be center justified and the numbers right justified.
I don't have a neat answer for decimal justification.

	--Mark

Re: [Docutils-develop] Proposed change to RST =?iso-8859-1?q?parser=3A justification=20in=20simple?= tables

[<eng...@ss...>]

On Fri, 17 Jan 2003, Richard Jones wrote:

> On Fri, 17 Jan 2003 2:08 pm, Richard Jones wrote:
> > On Fri, 17 Jan 2003 12:25 pm, Benja Fallenstein wrote:
> > > David Goodger wrote:
> > > > My first knee-jerk objection was, what about when the text begins in
> > > > the leftmost *and* ends in the rightmost column? ::
> > > >
> > > >     ==========  ===
> > > >     Name        Age
> > > >     ==========  ===
> > > >     David        34
> > > >     Methuselah  969
> > > >     ==========  ===

this one would have 34 right aligned and 969 left or should alignment be a
columnsetting.

BTW: what about
    =========== =======
    Name          Val
    =========== =======
    me             1.65
    you           10.60
    =========== =======

what would Val and of course i want alignement on the decimal.


> > >
> > > But the rule is precise about that: in this case, left justification as
> > > the reasonable default. If the author wants right justification, they
> > > have to add a space as you said.


> >
> > Or perhaps justification could be automatically switched off?
>
>   ... in the case where there's ambuguity.
>
>
> Stupid lunchtime beer.
>
>    Richard (trying to be helpful, but might just go have a snooze instead ;)

sch sound asleep

-- 
 BINGO: aggressively priced next-generation critical mass
 --- Engelbert Gruber -------+
 SSG Fintl,Gruber,Lassnig   /
 A6410 Telfs Untermarkt 9  /
 Tel. ++43-5262-64727 ----+

Re: [Docutils-develop] New directive proposal: shell

[Fred L. D. Jr. <fd...@ac...>]

Benja Fallenstein writes:
 > Except if you run them inside a strong bastion by default...

No, there's no way to do this in the standard library.

 > I do believe that 'python' is a better name than 'exec,' for language 
 > agnosticity, btw.

Yes, most certainly!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

Re: [Docutils-develop] Proposed change to RST =?iso-8859-1?q?parser=3A justification=20in=20simple?= tables

[Richard J. <rj...@ek...>]

On Fri, 17 Jan 2003 2:08 pm, Richard Jones wrote:
> On Fri, 17 Jan 2003 12:25 pm, Benja Fallenstein wrote:
> > David Goodger wrote:
> > > My first knee-jerk objection was, what about when the text begins in
> > > the leftmost *and* ends in the rightmost column? ::
> > >
> > >     ==========  ===
> > >     Name        Age
> > >     ==========  ===
> > >     David        34
> > >     Methuselah  969
> > >     ==========  ===
> >
> > But the rule is precise about that: in this case, left justification as
> > the reasonable default. If the author wants right justification, they
> > have to add a space as you said.
>
> Or perhaps justification could be automatically switched off?

  ... in the case where there's ambuguity.


Stupid lunchtime beer.


   Richard (trying to be helpful, but might just go have a snooze instead ;)

Re: [Docutils-develop] Proposed change to RST =?iso-8859-1?q?parser=3A justification=20in=20simple?= tables

[Richard J. <rj...@ek...>]

On Fri, 17 Jan 2003 12:25 pm, Benja Fallenstein wrote:
> David Goodger wrote:
> > Mark Nodine wrote:
> >>I propose that ... for cells in simple tables that comprise a single
> >>line, that the justification be inferred according to the following
> >>rule:
> >>  1. If the text begins at the leftmost column of the cell,
> >>then left justification, ELSE
> >>  2. If the text begins at the rightmost column of the cell,
> >>then right justification, ELSE
> >>  3. Center justification.
> >
> > My first knee-jerk objection was, what about when the text begins in
> > the leftmost *and* ends in the rightmost column? ::
> >
> >     ==========  ===
> >     Name        Age
> >     ==========  ===
> >     David        34
> >     Methuselah  969
> >     ==========  ===
>
> But the rule is precise about that: in this case, left justification as
> the reasonable default. If the author wants right justification, they
> have to add a space as you said.

Or perhaps justification could be automatically switched off?

+1 for the idea in general though :)


    Richard

Re: [Docutils-develop] Proposed change to RST parser: justification in simple tables

[Benja F. <b.f...@gm...>]

David Goodger wrote:
> Mark Nodine wrote:
> 
>>I propose that ... for cells in simple tables that comprise a single
>>line, that the justification be inferred according to the following
>>rule:
>>  1. If the text begins at the leftmost column of the cell,
>>then left justification, ELSE
>>  2. If the text begins at the rightmost column of the cell,
>>then right justification, ELSE
>>  3. Center justification.
> 
> 
> My first knee-jerk objection was, what about when the text begins in
> the leftmost *and* ends in the rightmost column? ::
> 
>     ==========  ===
>     Name        Age
>     ==========  ===
>     David        34
>     Methuselah  969
>     ==========  ===

But the rule is precise about that: in this case, left justification as 
the reasonable default. If the author wants right justification, they 
have to add a space as you said.

- Benja

Re: [Docutils-develop] Proposed change to RST parser: justification in simple tables

[David G. <go...@py...>]

Mark Nodine wrote:
> I propose that ... for cells in simple tables that comprise a single
> line, that the justification be inferred according to the following
> rule:
>   1. If the text begins at the leftmost column of the cell,
> then left justification, ELSE
>   2. If the text begins at the rightmost column of the cell,
> then right justification, ELSE
>   3. Center justification.

My first knee-jerk objection was, what about when the text begins in
the leftmost *and* ends in the rightmost column? ::

    ==========  ===
    Name        Age
    ==========  ===
    David        34
    Methuselah  969
    ==========  ===

But I suppose the onus is on the author to make the situation
unambiguous by adding blank columns as necessary::

    ===========  ====
    Name          Age
    ===========  ====
    David          34
    Methuselah    969
    ===========  ====

Seems like a reasonable proposal.  I have no objections, although I
would like to see a parser setting to turn off
justification-recognition (normally on would be fine).  I'll add it to
the to-do.  Patches to the code and docs are welcome!

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] New directive proposal: shell

[Richard J. <rj...@ek...>]

On Fri, 17 Jan 2003 11:34 am, Benja Fallenstein wrote:
> Richard Jones wrote:
> > On Fri, 17 Jan 2003 10:43 am, David Goodger wrote:
> >>Mark Nodine wrote:
> >>>I'd like to get everyone's thoughts about a new RST directive,
> >>>which I'll tentatively call "shell", though I'm open to other
> >>>ideas for the name.
> >>
> >>It's already in the to-do list, with the directive name "system":
> >><http://docutils.sf.net/spec/notes.html#misc-system>.  There's also an
> >>"exec" directive for executing Python code.
> >>
> >>They are both just awaiting implementation ;).
> >
> > Just glancing at that, I believe that "misc.exec" and "misc.eval" need to
> > have their notes upgraded to "Definitely dangerous!".
>
> Except if you run them inside a strong bastion by default...

There appears to be no chance that a secure bastion will exist for Python in 
the near future. The Bastion and rexec code is being removed from Python 2.3 
for this reason.


> I do believe that 'python' is a better name than 'exec,' for language
> agnosticity, btw.

+1


    Richard

Re: [Docutils-develop] New directive proposal: shell

[Benja F. <b.f...@gm...>]

Richard Jones wrote:
> On Fri, 17 Jan 2003 10:43 am, David Goodger wrote:
> 
>>Mark Nodine wrote:
>>
>>>I'd like to get everyone's thoughts about a new RST directive,
>>>which I'll tentatively call "shell", though I'm open to other
>>>ideas for the name.
>>
>>It's already in the to-do list, with the directive name "system":
>><http://docutils.sf.net/spec/notes.html#misc-system>.  There's also an
>>"exec" directive for executing Python code.
>>
>>They are both just awaiting implementation ;).
> 
> 
> Just glancing at that, I believe that "misc.exec" and "misc.eval" need to have 
> their notes upgraded to "Definitely dangerous!".

Except if you run them inside a strong bastion by default...

I do believe that 'python' is a better name than 'exec,' for language 
agnosticity, btw.

Cheers,
- Benja

Re: [Docutils-develop] New directive proposal: shell

[Richard J. <rj...@ek...>]

On Fri, 17 Jan 2003 10:43 am, David Goodger wrote:
> Mark Nodine wrote:
> > I'd like to get everyone's thoughts about a new RST directive,
> > which I'll tentatively call "shell", though I'm open to other
> > ideas for the name.
>
> It's already in the to-do list, with the directive name "system":
> <http://docutils.sf.net/spec/notes.html#misc-system>.  There's also an
> "exec" directive for executing Python code.
>
> They are both just awaiting implementation ;).

Just glancing at that, I believe that "misc.exec" and "misc.eval" need to have 
their notes upgraded to "Definitely dangerous!".


     Richard

Re: [Docutils-develop] New directive proposal: shell

[David G. <go...@py...>]

Mark Nodine wrote:
> I'd like to get everyone's thoughts about a new RST directive,
> which I'll tentatively call "shell", though I'm open to other
> ideas for the name.

It's already in the to-do list, with the directive name "system":
<http://docutils.sf.net/spec/notes.html#misc-system>.  There's also an
"exec" directive for executing Python code.

They are both just awaiting implementation ;).

-- 
David Goodger  <go...@py...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[Docutils-develop] Re: is sf down?

[David G. <go...@py...>]

Anonymous CVS access is still down, but I've fixed the hourly update script
so that it no longer clobbers good snapshots with bad.  I will manually
update the snapshots as required until SF's anon CVS is back up.

-- David Goodger    go...@py...