docutils-develop — For developer discussions of the implementation.

[Docutils-develop] Docutils' HTML-writer: Replacement of accentuated characters by Char entities

[<g.s...@gm...>]

Pierre-Yves wrote:
>
> However the accentuated characters are not converted in Char entities, 
> as far as I could test the things.
>
> As REST format is aimed to be written by people not having converter
tools,
> how can I safely ensure the entities conversion together with the
DocFactory
> process ?
>
> example :
>
> <p>L'UPA a pour objet, l'étude, la protection et la promotion de la </p>

Hello Pierre-Yves,

Sorry for my late reply. I have been on vacation for the last two weeks...

Understanding that you would like accentuated characters (like "é") to be
replaced by their Char entities ("&#233;") when ReST is converted to HTML I
have to disappoint you: As far as I can see the HTML-writer of Docutils does not
replace accentuated characters by Char entities. However, it creates a line

<meta http-equiv="Content-Type" content="text/html; charset=XXXXXX" />

in the head of the HTML document (where XXXXXX is replaced with the
output-encoding of your Docutils-configuration - for example cp1252 or ISO-8859-1).
As far as I know this line tells the webbrowser which charset has to be used
when the HTML page is displayed and there is no need to replace accentuated
characters with Char entities. (At least I did not face any problems with IE5
and Netscape4.7.)

If you really want to replace accentuated characters by Char entities I
recommend to use the "raw"-directive of Docutils. 

Example::

  L'UPA a pour objet, |l'étude|, la protection et 
  la promotion de la

  .. |l'étude| raw:: html

      l'&#233;tude

(This tells Docutils' HTML-writer to replace "|l'étude|" with the raw-data
"l'&#233;tude" during the HTML-conversion.) 

I have posted a copy of this mail to doc...@li....
Maybe someone out there knows a better solution ...

Best regards,
Gunnar.


-- 
+++ GMX - Mail, Messaging & more  http://www.gmx.net +++
NEU: Mit GMX ins Internet. Rund um die Uhr für 1 ct/ Min. surfen!

[Docutils-develop] reST merged into Zope 2.7/HEAD

[Andreas J. <an...@an...>]

I just merged the reST integration branch into the Zope HEAD including 
ZReST.

Changes to ZReST:

I added "input_encoding" and "output_encoding" as properties so users
can freely choose the encodings. The default for both parameters
is iso-8859-15.

Andreas

    ---------------------------------------------------------------------
   -    Andreas Jung                     http://www.andreas-jung.com   -
  -   EMail: andreas at andreas-jung.com                              -
   -            "Life is too short to (re)write parsers"               -
    ---------------------------------------------------------------------

Re: [Docutils-develop] Headers

[Ian B. <ia...@co...>]

On Thu, 2003-01-30 at 09:52, Mark Nodine wrote:
> > I'm looking for something vaguely like the Python reference manual.  I
> > like the reference manuals for the most part, except they don't have
> > very good indexes (which is something I really look forward to with
> > docutils/reST).
> 
> We discussed this issue a couple of weeks back, but what I'm
> planning to do for my index entries is to label them as
> inline targets (or explicit markup targets tied to inline
> targets) and have a separate writer that can extract them
> into a usable index.

Oops, I meant table of contents.  But I want an index too!  I want it
all! ;)

It seems it would be very easy to start with an index made up of all the
targets in the document.  I noticed some notes on something like
`word`:index: which would also be possible, but probably more thorough
than I'd use in programming documentation.  

-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

Re: [Docutils-develop] Headers

[Mark N. <no...@so...>]

Ian Bicking wrote:
> 
> On Wednesday, January 29, 2003, at 02:16 PM, David Goodger wrote:
> >> Now, I could make each function a title, like
> >>
> >> function_name(params):
> >> **********************
> >>
> >> The docstring
> >>
> >> But I don't think I can get that to look like I want, even with
> >> fancy CSS.
> >
> > What do you want it to look like?
> 
> I'm looking for something vaguely like the Python reference manual.  I
> like the reference manuals for the most part, except they don't have
> very good indexes (which is something I really look forward to with
> docutils/reST).

We discussed this issue a couple of weeks back, but what I'm
planning to do for my index entries is to label them as
inline targets (or explicit markup targets tied to inline
targets) and have a separate writer that can extract them
into a usable index.

	--Mark

Re: [Docutils-develop] Headers

[Ian B. <ia...@co...>]

On Wednesday, January 29, 2003, at 02:16 PM, David Goodger wrote:
>> Now, I could make each function a title, like
>>
>> function_name(params):
>> **********************
>>
>> The docstring
>>
>> But I don't think I can get that to look like I want, even with
>> fancy CSS.
>
> What do you want it to look like?

I'm looking for something vaguely like the Python reference manual.  I 
like the reference manuals for the most part, except they don't have 
very good indexes (which is something I really look forward to with 
docutils/reST).

>> Not to mention that I don't know at what level (h1, h2, h3, etc)
>> the function definition will be -- it's largely orthogonal to other
>> structure.  Hmm... what I'd really like is a directive, like:
>>
>> .. header:: function_name
>>
>> That would create an invisible sort of header, where it shows up in
>> the TOC and is a link target, but isn't otherwise displayed.  Of
>> course, "function_name" always the name of the target, I've been
>> using "ClassName.function_name".  So something like:
>>
>> .. header:: function_name
>>    :anchor-name: ClassName.function_name
>>
>> Anyway, that's my initial thought.  Maybe there's a different way to
>> do this.
>
> I don't think this is a good approach.  It smells like a kludge: too
> single-purpose to be worthwhile, and very magical.  How would this
> translate into elements in the doctree?  The doctree must remain
> self-consistent and style-free (as much as possible).
>
> Although I still suspect you're letting style dictate structure, I'll
> try to help you out.  But we must recognize that there are limits to
> reStructuredText, beyond which we have to consider alternatives.
> ReStructuredText is aimed at ordinary documents; it's not suitable for
> some really creative graphic design exercises, and I don't think it
> should be.

This is, or is supposed to be, the quintessential reStructuredText 
document -- a programmer's reference.  If the doctree is nice, then 
with a little XSL or what-have-you anything is possible (though not 
necessarily easy even then).  But the preprocessor already feels like a 
kludge, and adding a postprocessor feels even worse.  But then I'm 
anti-XML, so I'd feel that way ;)  I prefer event-driven to a tree 
representation, but whatever.

And layout does really matter -- that's a big part of why I don't like 
the output of pydoc.  I want the documentation to look right both in 
the source and in the generated documentation.  It's not an 
unreasonable goal, and I don't think a difficult goal either.

> It seems that you're looking for finer-grained selectability, to be
> able to more precisely target styles.  Currently we have two extremes:
> overly broad (a <div> element surrounds every section, but different
> "types" of sections are not distinct from each other), and overly
> precise (each <div> section element has a unique "id" attribute, which
> *could* be used in the stylesheet, but it would be a manual pain).
>
> Perhaps the "topic" directive would suit your purposes.  See
> <http://docutils.sf.net/spec/rst/directives.html#topic>, and note that
> topics cannot contain subsections or subtopics.  Currently, topics
> don't show up in the table of contents, but that could be remedied
> with a new option to the "contents" directive.

Yes, topics look good.  I just tried using them, and with a little CSS 
tweaking they seem alright.  Though because of the use of <p> tags for 
the title and body, the function title and parameter line is separated 
from its docstring, which I would prefer not to have.  I haven't seen 
anything that lets you change the space between paragraphs with CSS, 
which I've wanted to do often...

> Docutils uses an optional "class" attribute on all elements, which is
> carried all the way through the HTML writer.  The class attribute can
> contain multiple values (space-separated).  A new directive could be
> added that adds a class value to the enclosing element, and this class
> could be selected in the stylesheet.  If the section *containing* the
> specialized function sections is given a class, say
> "function-container", the stylesheet could contain a style like this::
>
>     div.function-container div { ... your styles here ... }
>
> I'll add these to the to-do list.  Contributions are welcome!

Yes, that would work nicely, because then I wouldn't have to monopolize 
topic:: for function definitions.  I still haven't looked at the 
internals of the parsing, but I'll try to look into it more -- it might 
give me some insight into how the extractor should work as well.

   Ian

Re: [Docutils-develop] Headers

[David G. <go...@py...>]

[David Goodger]
>> Stop fighting the markup.  ReStructuredText doesn't use indentation
>> for large-scale structure, only for local constructs (lists etc.).
>> If it sounds like a duck and acts like a duck, it's a duck.  Use
>> section structure, and the table of contents is easy,
>> cross-references are easy.
>> 
>> If you want the output to have indentation for each section level,
>> you can do that in the stylesheet.  Don't let style dictate
>> structure.

[Ian Bicking]
> Okay, so now I'm doing that, without indentation.  Except for
> functions/methods, where the actual docstring is indented, as in:
> 
> `function_name(params)`:
>     The docstring.
> 
> Which probably won't cause any problem.  But now I'm trying to get
> the functions in the table of contents.

That construct is a definition list.  If that's what you want, fine,
but there's no support for adding it to the TOC.

Also, did you intend literals (``) instead of interpreted text (`)?

> Now, I could make each function a title, like
> 
> function_name(params):
> **********************
> 
> The docstring
> 
> But I don't think I can get that to look like I want, even with
> fancy CSS.

What do you want it to look like?

> Not to mention that I don't know at what level (h1, h2, h3, etc)
> the function definition will be -- it's largely orthogonal to other
> structure.  Hmm... what I'd really like is a directive, like:
> 
> .. header:: function_name
> 
> That would create an invisible sort of header, where it shows up in
> the TOC and is a link target, but isn't otherwise displayed.  Of
> course, "function_name" always the name of the target, I've been
> using "ClassName.function_name".  So something like:
> 
> .. header:: function_name
>    :anchor-name: ClassName.function_name
>
> Anyway, that's my initial thought.  Maybe there's a different way to
> do this.

I don't think this is a good approach.  It smells like a kludge: too
single-purpose to be worthwhile, and very magical.  How would this
translate into elements in the doctree?  The doctree must remain
self-consistent and style-free (as much as possible).

Although I still suspect you're letting style dictate structure, I'll
try to help you out.  But we must recognize that there are limits to
reStructuredText, beyond which we have to consider alternatives.
ReStructuredText is aimed at ordinary documents; it's not suitable for
some really creative graphic design exercises, and I don't think it
should be.

It seems that you're looking for finer-grained selectability, to be
able to more precisely target styles.  Currently we have two extremes:
overly broad (a <div> element surrounds every section, but different
"types" of sections are not distinct from each other), and overly
precise (each <div> section element has a unique "id" attribute, which
*could* be used in the stylesheet, but it would be a manual pain).

Perhaps the "topic" directive would suit your purposes.  See
<http://docutils.sf.net/spec/rst/directives.html#topic>, and note that
topics cannot contain subsections or subtopics.  Currently, topics
don't show up in the table of contents, but that could be remedied
with a new option to the "contents" directive.

As for "anchor-name", that can be a simple internal hyperlink target.
Together, it would look something like this::

    .. contents::
       :topics:

    .. _ClassName.function.name:

    .. topic:: 

Docutils uses an optional "class" attribute on all elements, which is
carried all the way through the HTML writer.  The class attribute can
contain multiple values (space-separated).  A new directive could be
added that adds a class value to the enclosing element, and this class
could be selected in the stylesheet.  If the section *containing* the
specialized function sections is given a class, say
"function-container", the stylesheet could contain a style like this::

    div.function-container div { ... your styles here ... }

I'll add these to the to-do list.  Contributions are welcome!

> Also, would there be a way to create a smaller TOC for a section?

Yes, you can use the "contents" directive with the ":local:" option:

    Generate a local table of contents.  Entries will only include
    subsections of the section in which the directive is given.  If no
    explicit title is given, the table of contents will not be titled.

The ":depth:" option may also be useful:

    The number of section levels that are collected in the table of
    contents.  The default is unlimited depth.

See <http://docutils.sf.net/spec/rst/directives.html#table-of-contents>
for details, and <http://docutils.sf.net/spec/doctree.html> for
examples of both options.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Headers

[Ian B. <ia...@co...>]

On Sat, 2003-01-25 at 10:10, David Goodger wrote:
> Stop fighting the markup.  ReStructuredText doesn't use indentation for
> large-scale structure, only for local constructs (lists etc.).  If it sounds
> like a duck and acts like a duck, it's a duck.  Use section structure, and
> the table of contents is easy, cross-references are easy.
> 
> If you want the output to have indentation for each section level, you can
> do that in the stylesheet.  Don't let style dictate structure.

Okay, so now I'm doing that, without indentation.  Except for
functions/methods, where the actual docstring is indented, as in:

`function_name(params)`:
    The docstring.

Which probably won't cause any problem.  But now I'm trying to get the
functions in the table of contents.  Now, I could make each function a
title, like

function_name(params):
**********************

The docstring

But I don't think I can get that to look like I want, even with fancy
CSS.  Not to mention that I don't know at what level (h1, h2, h3, etc)
the function definition will be -- it's largely orthogonal to other
structure.  Hmm... what I'd really like is a directive, like:

.. header:: function_name

That would create an invisible sort of header, where it shows up in the
TOC and is a link target, but isn't otherwise displayed.  Of course,
"function_name" always the name of the target, I've been using
"ClassName.function_name".  So something like:

.. header:: function_name
   :anchor-name: ClassName.function_name

Anyway, that's my initial thought.  Maybe there's a different way to do
this.

Also, would there be a way to create a smaller TOC for a section?  I'm
thinking about putting several modules in one document, to make
interlinking easier, to make the public interface easier to navigate
(since the interface is made up of the methods from several classes that
inherit from each other).  It would be nice to have a TOC for each
class, for instance.

Thanks,
  Ian

RE: [Docutils-develop] Interpreted text for terms?

[Moore, P. <Pau...@at...>]

From: Benja Fallenstein [mailto:b.f...@gm...]
> As I said, I'm fine with this, but I just noticed a case I
> hadn't realized before and thought I'd pass it on. From one
> article-in-writing:
>
>   UML (the Unified Modeling Language) ... has become the *de
>   facto* industry-standard language for specifying, visualizing,
>   constructing, and documenting the products in software
>   engineering.
>
> It is pretty common to set this kind of term in italics in print
> publications, yet when asterisk-emphasing it, the meaning changes
> for me:

This raises an interesting issue. Asterisks mean *emphasis*, and
as such should not be used automatically where a printed rendering
of italic is intended. (The fact that the markup is often used in
naive plain-text markup to imply "bold" tends to exaggerate the
distinction). On the other hand, that's going to strengthen the case
for having a variety of specialist interpreted text roles...

Maybe it would be useful to have some form of general directive which
maps a role to specific markup (obviously writer-dependent). Something
like

    .. role:: term
       :html: <em>$text</em>

No idea how this would translate to other output formats - markup
like TeX or Docbook should be OK, but things like PDF could be
impossible...

Just a thought,
Paul.

Re: [Docutils-develop] Interpreted text for terms?

[Benja F. <b.f...@gm...>]

David Goodger wrote:
> Benja Fallenstein wrote:
> 
>>I'm currently writing two articles using ReST, and regarding the recent
>>interpreted text roles, I have noticed one role I could use: 'terms.'
>>For example, I'm writing about something called 'nomadicity.' In the
>>printed/HTML version, I'd italicize this, yet it's not really an
>>emphasis; at least in plain text, I wouldn't write *nomadicity* (maybe
>>I'd use single quotes as above).
>>
>>Any chance this could be an interpreted text role? It would probably
>>have to have an unobstrusive alias, two chars at most, for me to use
>>this... at least I think so.
> 
> 
> I think that depends on its intended use.  In `The Chicago Manual of Style`,
> 14th ed., sections 6.71-74 deal with "special terminology".  If all that is
> desired is the typographical effect, I don't see enough of a distinction
> between a "term" and an "emphasized word or phrase" to warrant a separate
> element.  One of the traditional uses of italics as emphasis is for this
> kind of term.  I don't want to let in all kinds of inline elements with
> marginal uses.  I would first need to see solid evidence (use cases,
> references, etc.) and see that there's a widespread need.  It'll go on the
> "to-do?" list for now.

As I said, I'm fine with this, but I just noticed a case I hadn't 
realized before and thought I'd pass it on. From one article-in-writing:

   UML (the Unified Modeling Language) ... has become the *de facto*
   industry-standard language for specifying, visualizing, constructing,
   and documenting the products in software engineering.

It is pretty common to set this kind of term in italics in print 
publications, yet when asterisk-emphasing it, the meaning changes for 
me: The sentence seems to emphasize that this is the *de facto*, but 
*not* the real standard in SE (and I don't think any special emphasis on 
this point is intended).

As I said, just thought I'd share the observation.

- Benja

Re: [Docutils-develop] Upgrade problem/ set_reader() problem

[Andreas J. <an...@an...>]

--On Samstag, 25. Januar 2003 11:01 -0500 David Goodger 
<go...@py...> wrote:

> Andreas Jung wrote:
>> I tried upgrading the docutils package for the Zope 2.7 release but
>> the API seems to have changed since November.
>
> Are you surprised? ;-)

Not really. Thanks for your explanations. reST is now working again
in Zope.

-aj

Re: [Docutils-develop] Headers

[David G. <go...@py...>]

Ian Bicking wrote:
> Okay, I'm messing around with the document extractor again, and trying
> to document a specific class.  The class has a lot of methods, and I'd
> like to categorize them.  I've also been using blockquotes to represent
> the nesting of the methods in the class and the class in the module.

Why use blockquotes?

> Now, I'd like **Accessors** to actually be a header, like:
> 
>    Accessors
>    ---------
>
> But I don't believe reST allows a header in a blockquote.  **Accessors**
> isn't too bad, and I can still put in a link target, but the thing I
> *really* want is a table of contents.

So make it a header.  Don't use blockquotes.

> Though, as I realize it, I *really* want a table of contents that
> includes the function definitions, so it'd be like:
> 
> HTTPRequest
>     Accessors
>         transaction
>         setTransaction
>         time
> 
> So that's even further from what reST is doing.  What might I do to
> achieve that kind of formating?  A table of contents like that would be
> quite important.

Stop fighting the markup.  ReStructuredText doesn't use indentation for
large-scale structure, only for local constructs (lists etc.).  If it sounds
like a duck and acts like a duck, it's a duck.  Use section structure, and
the table of contents is easy, cross-references are easy.

If you want the output to have indentation for each section level, you can
do that in the stylesheet.  Don't let style dictate structure.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Upgrade problem/ set_reader() problem

[David G. <go...@py...>]

Andreas Jung wrote:
> I tried upgrading the docutils package for the Zope 2.7 release but
> the API seems to have changed since November.

Are you surprised? ;-)

> The following code does no longer work and I can figure out why:
...
>    pub.set_reader('restructuredtext', None, 'restructuredtext')
...
> ImportError: No module named restructuredtext

On November 8th I "removed bogus aliases" from docutils/readers/__init__.py
(version 1.12).  There is no such thing as a "restructuredtext" reader; that
was a mistake ("restructuredtext" is the parser).  Use "standalone"
instead::

    pub.set_reader('standalone', None, 'restructuredtext')

It looks like your code was copied from Richard Jones' ZReST writer module.
I fixed his code when I made the changes, but didn't know about yours.

I also changed the IO classes' API a bit.  They'll still work the old way,
but raise a deprecation warning.  As of Docutils 0.3, the old way will stop
working.  Change this:

>    pub.source = docutils.io.StringInput(pub.settings)
>    pub.source.source = src

To this:

        pub.source = docutils.io.StringInput(
            source=self.source, encoding=pub.settings.input_encoding)

And this:

>    pub.destination = docutils.io.StringOutput(pub.settings)

To this:

        pub.destination = docutils.io.StringOutput(
            encoding=pub.settings.output_encoding)

You should be able to simplify your code a lot if you use the
docutils.core.publish_string convenience function.  It might even insulate
your code against future changes -- not that I'm planning any, but one never
knows.  Docutils *is* still experimental.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

[Docutils-develop] Upgrade problem/ set_reader() problem

[Andreas J. <an...@an...>]

I tried upgrading the docutils package for the Zope 2.7 release but
the API seems to have changed since November. The following code
does no longer work and I can figure out why:

import docutils.core, docutils.io

class Warnings:
    def __init__(self):
        self.messages = []

    def write(self, message):
        self.messages.append(message)


def HTML(src, writer='html4zope', report_level=1):
    """ render HTML from a reStructuredText string

        - 'src'  -- string containing a valid reST document

        - 'writer' -- docutils writer

        - 'report_level' - verbosity of reST parser
    """

    pub = docutils.core.Publisher()
    pub.set_reader('restructuredtext', None, 'restructuredtext')
    pub.set_writer(writer)

    # go with the defaults
    pub.get_settings()

    pub.settings.stylesheet = "default.css"

    # this is needed, but doesn't seem to do anything
    pub.settings._destination = ''

    # set the reporting level to something sane
    pub.settings.report_level = report_level

    # don't break if we get errors
    pub.settings.halt_level = 6

    # remember warnings
    pub.settings.warning_stream = Warnings()

    # input
    pub.source = docutils.io.StringInput(pub.settings)
    pub.source.source = src

    # output - not that it's needed
    pub.destination = docutils.io.StringOutput(pub.settings)

    # parse!
    document = pub.reader.read(pub.source, pub.parser, pub.settings)
    warnings = ''.join(pub.settings.warning_stream.messages)

    # do the format
    return pub.writer.write(document, pub.destination)

The Zope traceback is:

Traceback (innermost last):

    * Module ZPublisher.Publish, line 98, in publish
    * Module ZPublisher.mapply, line 88, in mapply
    * Module ZPublisher.Publish, line 39, in call_object
    * Module OFS.DTMLMethod, line 126, in __call__
    * Module DocumentTemplate.DT_String, line 474, in __call__
    * Module DocumentTemplate.DT_Var, line 274, in render
      __traceback_info__: ('p1', 
'+------------+------------+-----------+\n| Header 1 | Header 2 | Header 3 
|\n+============+============+===========+\n| body row 1 | column 2 | 
column 3 |\n+------------+------------+-----------+\n| body row 2 | Cells 
may span columns.|\n+------------+------------+-----------+\n| body row 3 | 
Cells may | - Cells |\n+------------+ span rows. | - contain |\n| body row 
4 | | - blocks. |\n+------------+------------+-----------+\n', {'': 'p1', 
'fmt': 'restructured-text'})
    * Module DocumentTemplate.DT_Var, line 408, in restructured_text
    * Module reStructuredText, line 39, in HTML
    * Module docutils.core, line 66, in set_reader
    * Module docutils.readers, line 87, in get_reader_class

ImportError: No module named restructuredtext

Any ideas?

Andreas

[Docutils-develop] Headers

[Ian B. <ia...@co...>]

Okay, I'm messing around with the document extractor again, and trying
to document a specific class.  The class has a lot of methods, and I'd
like to categorize them.  I've also been using blockquotes to represent
the nesting of the methods in the class and the class in the module.  So
the text looks like:

HTTPRequest
===========

    class `HTTPRequest(Request)`:
        The well-named `HTTPRequest` represents a single HTTP
        request.  It is created by the `Application`, and
        belongs to a `Transaction`.
        
        It is created by `Transaction`.
        
        
        **Accessors**
        
            `transaction()`:
                The `Transaction` this request belongs to
                
            `setTransaction(trans)`:
                This method should be invoked after the transaction is 
                created for this request.
                
            `time()`:
                Returns the time that the request was received,
                a timestamp (seconds from the epoch).



Now, I'd like **Accessors** to actually be a header, like:

        Accessors
        ---------

But I don't believe reST allows a header in a blockquote.  **Accessors**
isn't too bad, and I can still put in a link target, but the thing I
*really* want is a table of contents.

Though, as I realize it, I *really* want a table of contents that
includes the function definitions, so it'd be like:

HTTPRequest
    Accessors
        transaction
        setTransaction
        time

So that's even further from what reST is doing.  What might I do to
achieve that kind of formating?  A table of contents like that would be
quite important.

I'm also thinking about putting groups of modules together into a single
document (mostly modules that contain classes that inherit from each
other).  As a reader I find this easier to navigate generally, and it
makes references easier to handle.  So the table of contents would
become much more important in that case.


-- 
Ian Bicking           Colorstudy Web Development
ia...@co...   http://www.colorstudy.com
PGP: gpg --keyserver pgp.mit.edu --recv-keys 0x9B9E28B7
4869 N Talman Ave, Chicago, IL 60625 / (773) 275-7241

[Docutils-develop] Re: Building the Docutils community

[David A. <da...@bo...>]

David Goodger <go...@py...> writes:

> Docutils *is* a complex system.  

Also a great idea and a useful tool.

> Having limited time, my approach has been to write docs only when
> *I* have felt a need, internally or externally.

Somehow it seems to me that a project like DocUtils, in particular,
should be a model for quality-of-documentation.  ReST
literate-programming anyone?

-- 
                       David Abrahams
   da...@bo... * http://www.boost-consulting.com
Boost support, enhancements, training, and commercial distribution

[Docutils-develop] Re: call for existing interpreted text art

[Aahz <aa...@py...>]

> Can you detail what you expect from the "list", "figure", and "chapter"
> roles?  Perhaps a minimal self-contained (including targets) input example?
> This may be related to the object references described in
> 
> http://docutils.sf.net/spec/notes.html#object-numbering-and-object-references

Probably related, yes.  Note that I disagree with the use of boilerplate
text in addition to the numbers, because of the singular/plural problem
that you'll see below.

Here's what I'm intending.  First, I've got a ToC file that will be
included in the chapters::

    .. chapter-list::
        Way
        Objects
        Data
        Control
        Func
        Module
        OOP
        Text
        Iterator
        Newstyle
        Apps

This creates auto-number chapter references.  Here's the sample I sent
you earlier, now including the targets; it'd be in the Objects/
directory under ToC, and the file itself is Objects::

    .. include:: ../ToC

    To understand better how mutable and immutable objects work, see
    code listings :list:`mutable.py` and :list:`immutable.py`, as well
    as figures :figure:`mutable.eps` and :figure:`immutable.eps`.
    You'll also want to read chapter :chapter:`Data`.

    .. include-code:: mutable.py

    .. figure:: mutable.eps

    .. include-code:: immutable.py

    .. figure:: immutable.eps

For text output, this would generate::

    To understand better how mutable and immutable objects work, see code
    listings 2.1 and 2.2, as well as figures 2.1 and 2.2.  You'll also
    want to read chapter 3.

    Listing 2.1: mutable.py

        a = [1]
        b = a
        a.append('spam')
        print b

    Figure 2.1: mutable.eps

    .....
-- 
Aahz (aa...@py...)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach

Re: [Docutils-develop] Re: [Docutils-users] Programming reference documentation

[Beni C. <cb...@te...>]

On 2003-01-20, Ian Bicking wrote:

> On Sun, 2003-01-19 at 06:00, Beni Cherniavsky wrote:
> > Forgot to announce it; since the source reader is not ready for use (is
> > it?),
>
> It seems to work okay for me -- I think David might have some features
> he'd like, but what's there seems sufficient to me.
>
Oops, just now found rjhack.py.  Works great for me.

> >I think somebody might benefit from my solution.  I've written code
> > that was documented with rST docstrings but couldn't use them (pydoc
> > ignores the rST, docutils barfs at the python code).So I took the
> > plain-text formatter of pydoc and hacked it until the result was a
> > document with chapters instead of the "| " nesting, etc., so that it could
> > be processed by the plain rST reader of docutils.
>
> Ack... sounds like a mess.
>
Indeed :).

> > > Ian Bicking wrote:
> > > > One thing I *don't*like is reference documentation in the style of
> > > > pydoc.To me it's too automatic, and generally too hard to follow.
> > > > I find reference documentation with some framework of text (lead-in
> > > > paragraphs, categorization, etc) to be much better.
> > >
> > Like GNU info files tend to be?libc, elisp, make, they have the best
> > hierarchically organised documentation that I've ever read.Puts
> > everything in the right place in your head after first reading but also
> > allows you to quickly read just what you need.
> >
> > I'm now writing a library that I'd like to document very well, and info
> > files are my standard of excellence ;-).So I want to figure out a way
> > for creating such documentation with docutils, too.
>
> Documenting a library is different then a program, in my mind.  Or at
> least there should be some entirely thorough but not necessarily
> intuitively structured documentation.

True.

> In Webware I'm also making documents that are task and instruction
> oriented.  But they don't make up for the reference documentation that
> you want when you're already familiar with the library, or at least the
> concept behind the library, and you just want to know how to spell the
> method name or something.
>
> This is how I use the Python reference documentation -- which on the
> whole is quite good -- and is how I'd expect to use a docstring
> extractor.  I just write the other documents in a separate file.
>
Sure, if I write intorductory / how-to style docs, I'd want them out of my
source.  But it can be nice to place them together in one output file and
at least in info files, I never felt that these parts obstruct reference
reading in any way.

> > Now that I think of it, is anyone else interested in a GNU info writer
> > for docutils?  If yes, I'll append it to my task queue (i.e. don't
> > hold your breath).
>
> Info documents are often well organized, but the actual reader I've
> never been impressed with.  I get lost too easily.  But web pages
> certainly aren't ideal either.
>
It's much more impressive when you are restricted to a text terminal :-).
Besides, info is very handy in emacs.

> > This are closest to what I was doing.The module & class docstrings will
> > contain overview information, perhaphs even in many chapters, that would
> > link to relevant places in the low-level per-method documentation.
> >
> > The problem with this approach is that it relies too much on hyperlinks
> > for a good order of reading.The great thing about GNU info files is that
> > they are also organised in an order that is almost optimal for one-pass
> > front-to-back reading.Ian Bicking's idea is better because it allows you
> > to specify the order of the documentation entries differently from the
> > order of the code.
>
> I'm realizing that reordering the methods might be best anyway --
> usually they don't match up with the description because they were just
> written in a certain order.
>
> The whole hyperlinking thing I'm still unsure about.  I don't even know
> if there's a way to indicate that `references` should be turned into
> some sort of link.
>
The question is how files will be combined, so that a link in one file can
refence a section of another?  Perhaps in large projects, some sort of
top-level config file saying which files are processed and how are they
combined is needed.  Then, even when processing single files, the context
will be known to enable links processing.

Or maybe the "linking" model is better: individual files can be generated;
then, if you wish, you perform a linking stage to resolve links between
them.  Currently this implies buidling a single tree in memory of all the
docs.  Perhaps something like LaTeX's mechanisms for crossrefs should be
done: when a single file is processed, it emits an index file the link
targets it contains and resolves external links from other files.  You run
it several times to resolve all links.  OTOH, this sounds like overkill.

> > I think I see your idea.  But in many some cases this is short of
> > what's needed.  Consider (invented example) an application where many
> > classes contain special `gui_repr` methods that defines the user
> > interface for this object.  They are completely irrelevant to the
> > algorithms described in other methods, so a good manual will probably
> > separate the documentation of all GUI methods into a separate chapter
> > discussing them all at once.
>
> When you start getting into that level or organization you have to have
> an application-wide view for the document.  I'm just thinking of modules
> now -- but there certainly are many modules that aren't well contained.
>
If the code is organized well enough, you can put top-level glue docs in
the __init__.py of a packages.  If not, you need reorganisation again...

> > So I think there should be (optional) ways to specify reordering /
> > reorganisation of nodes.  I'm not sure how to do it in a convenient
> > yet flexible way.  Probably Python scripting to generate the
> > reorganisation info would do the trick but would not be equally
> > applicable to non-python-source uses of rST.
>
> I know... I've already thought of several packages I have that I'd like
> to autogenerate the documentation, but the format needs to be structured
> different from how it generally would be.  For instance, in one library
> I have error strings that can be overridden -- these could be extracted,
> but only by a program that understood how my specific code was
> structured.
>
> > Also, for big documentation, it would make sense to hold the user-only
> > parts in separate files, leaving only what's relevant to the source's
> > design in the source.  The current include directive might solve this
> > task too, if the reorganisation is applied later.
>
> I'm not sure what you mean here...?
>
Like what you said, that the introductory/task-oriented parts are better
stored in separate files.

-- 
Beni Cherniavsky <cb...@tx...>

Re: [Docutils-develop] Re: [Docutils-users] Programming reference documentation

[Beni C. <cb...@te...>]

On 2003-01-20, David Goodger wrote:

> [dropping docutils-users from the cc: list]
>
> [Beni Cherniavsky]
> >> Forgot to announce it; since the source reader is not ready for use (is
> >> it?),
>
> No, not for a while.  When it is ready, I'll announce it, never fear :).
>
No, I meant that I forgot to announce my hack when I wrote it :).

> >> I think somebody might benefit from my solution.
>
> Send it, please!
>
Sorry for the delay, here__ is it.  Example command lines to use it are
here__.

__ http://www.technion.ac.il/~cben/python/rstpydoc.py
__ http://www.technion.ac.il/~cben/python/rst

-- 
Beni Cherniavsky <cb...@tx...>

There is an Excel spreadsheet here.  Do you want to open it? y
There was a grid bug in the spreadsheet.  The grid bug bites.

Re: [Docutils-develop] Interpreted text for terms?

[fantasai <fan...@es...>]

Mark Nodine wrote:
>
>   A ~behavioral bus monitor~ is a module written in the same 
>   language as the design (such as Verilog) to
>   monitor the signals on the bus and produce a warning if the 
>   conditions they check are violated.
> 
> In this example, the term is actually defined in the text
> of the document (rather than a separate glossary).  It is
> typeset as italics, but, and this is the key, it also creates
> an implicit target so that I can refer to this term from
> elsewhere in the document (and especially from the index).

Just wanted to note that in HTML this corresponds to <dfn>.

   http://www.w3.org/TR/REC-html40/struct/text.html#h-9.2.1

It is rendered in italics in MSIE and Gecko-based browsers,
possibly others as well.

If you're not worried about NS4 and other older browsers,
you can use the 'id' attribute to make it a target::

   <dfn id="term">term</dfn>

Otherwise, you'd need to use <a name="term"><dfn>term</dfn></a>.

~fantasai

Re: [Docutils-develop] Interpreted text for terms?

[David G. <go...@py...>]

[David Goodger]
>> I wasn't aware of this feature of Setext.  I thought ~tildes~ simply
>> indicated italics...

[Mark Nodine]
> Yes, the typotags documentation simply says that tildes
> produce italics.  However, in private correspondence I had
> with Ian Feldman, the originator of structure-enhanced text::

Thanks for the reference.

>>   An _`inline target` does the trick
>> 
>> And you can make it italic with a stylesheet entry.  The above is rendered
>> as ``<a class="target" ...>``.
> 
> I could live with this if I could tell the difference between an
> inline target and other targets; otherwise if everything is
> class="target", I can't sufficiently target the index entries.

Only inline targets contain text.  So a style making ``<a
class="target"...>`` italic should have the desired effect.

> What about the possibility of extending the inline target syntax
> to allow a role::
> 
>    An _`inline target`:index: would do the trick
> 
> since the parser could keep the role attribute and the writer could
> use the role attribute to determine the class.

Correction: the writer doesn't know anything about interpreted text.
Interpreted text is simply a syntax construct, not an end in itself (there's
no "interpreted" element in the DTD anymore; I ripped it out).  For more
background, see my Jan 5 post "Re: Handling interpreted text".

As for the proposal, it's unnecessary.  The "index" role will make its text
into an inline target if that is part of its semantics (which it probably
is).  No need to complicate the markup.

>> If you want a solution that doesn't resort to stylesheet trickery, then an
>> interpreted text role would be appropriate.  The role could simply wrap the
>> text with both emphasis and target elements.
> 
> This wrapping would be done by the writer?

No, by the parser, or by a transform if the parser doesn't have enough
information.

> If so, does it need to insert the <system_message> if there's a
> duplicate target?

The hyperlink resolution transform would, yes.

> I'd rather keep the name registration stuff in the parser if
> possible.

In the Docutils model, the parser registers most names, and the
tranformer/transforms handle the rest of the bookkeeping.  This is
because all of the names need to be known to form links.  If you
haven't yet, please read PEP 258, which outlines the Docutils model.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

[Fwd: Re: [Docutils-develop] Interpreted text for terms?]

[Mark N. <no...@so...>]

Sorry, I meant to send this to the list.

	--Mark

-------- Original Message --------
Subject: Re: [Docutils-develop] Interpreted text for terms?
Date: Mon, 20 Jan 2003 15:04:22 -0600
From: Mark Nodine <no...@ib...>
Reply-To: Mar...@mo...
To: David Goodger <go...@py...>
References: <BA51BCA1.2F16D%go...@py...>

David Goodger wrote:
> 
> Mark Nodine wrote:
> > What I'm wondering how to do in RST is what was represented
> > between tildes in structure-enhanced text.  It is
> > obviously closely related to interpreted text, and
> > in fact, I always interpreted it as terms to be included
> > in the index.  For example, ::
> >
> >    A ~behavioral bus monitor~ is a module written in the same
> ...
> >
> > In this example, the term is actually defined in the text
> > of the document (rather than a separate glossary).  It is
> > typeset as italics, but, and this is the key, it also creates
> > an implicit target so that I can refer to this term from
> > elsewhere in the document (and especially from the index).
> 
> I wasn't aware of this feature of Setext.  I thought ~tildes~ simply
> indicated italics; see the files in
> <http://docutils.sourceforge.net/mirror/setext/>, especially typotags.txt.
> Are you using a different Setext description, perhaps derived later?

Yes, the typotags documentation simply says that tildes 
produce italics.  However, in private correspondence I had
with Ian Feldman, the originator of structure-enhanced text::

  From: ia...@ra... (Ian Feldman)
  Date: Sun, 12 Jun 94 17:40:42 +0200

  That's right, they are the "natural" elements to enhance in this
  fashion.  In the setext2HTML tool I've also specified (though yet
  to be implemented) that all by definition single-word ~italic~
  elements be turned into <A NAME="italic"><i>italic</i></A> anchors
  and, that all (yet unnamed but soon to be named ref-tt) elements
  answering to the reg-exp of

    ^\[[A-Za-z0-9]+([\s][A-Za-z0-9]*\])\s

  such as, say, a fairly-frequent way to express exientific
  references in plaintext:

    [Feldman 1994] ......

  be turned into

    <A NAME="Feldman_1994">[Feldman 1994]</A> ....

  (which may be linked to via #Feldman_1994_ etc).

I implemented this enhancement and then used it for index
entries.  I actually used it in three contexts: (1) inline
text as my previous example, (2) around the term of a
definition list (*very* useful), and (3) in a comment to
define an index entry that links to the section header.
I'm not saying that's the best solution, but that's how
I did things like::

  Articles
  --------

  .. ~article, definite~

  Welsh has no indefinite article, only the ~definite article~ "the".

In rST, it would be better to have "article, definite" be an indirect
target to "definite article".

> > Is there a way to make interpreted text (perhaps with specific
> > role types) create implicit targets?  Or is there some other
> > way I should be going about this?
> 
> As Benja pointed out,
> 
>     An _`inline target` does the trick
> 
> And you can make it italic with a stylesheet entry.  The above is rendered
> as ``<a class="target" ...>``.

I could live with this if I could tell the difference between an
inline target and other targets; otherwise if everything is 
class="target", I can't sufficiently target the index entries.

What about the possibility of extending the inline target syntax
to allow a role::

   An _`inline target`:index: would do the trick

since the parser could keep the role attribute and the writer could
use the role attribute to determine the class.
 
> If you want a solution that doesn't resort to stylesheet trickery, then an
> interpreted text role would be appropriate.  The role could simply wrap the
> text with both emphasis and target elements.

This wrapping would be done by the writer?  If so, does it need
to insert the <system_message> if there's a duplicate target?
I'd rather keep the name registration stuff in the parser if
possible.

	--Mark

Re: [Docutils-develop] extractor.py

[David G. <go...@py...>]

Ian Bicking wrote:
> So, while including is still supported, now all methods and
> functions get included automatically.  But intermediate docstrings
> can be used to provide commentary, like:
> 
>     """
>     SomeModule does lots of stuff
>     """
> 
>     def stuff(): whatever
> 
>     """
>     The following functions are dangerous!
>     """
> 
>     def dangerousstuff(): whatever

This has come up before.  From the notes file::

    * Should standalone strings also become (module/class) docstrings?
      Under what conditions?  We want to prevent arbitrary strings
      from becomming docstrings of prior attribute assignments etc.
      Assume that there must be no blank lines between attributes and
      attribute docstrings?  (Use lineno of NEWLINE token.)

> Then other cross-module references, which I think is
> still an open issue for reST...?

One of many, yes.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Re: [Docutils-users] Programming reference documentation

[David G. <go...@py...>]

[dropping docutils-users from the cc: list]

[Beni Cherniavsky]
>> Forgot to announce it; since the source reader is not ready for use (is
>> it?),

No, not for a while.  When it is ready, I'll announce it, never fear :).

>> I think somebody might benefit from my solution.

Send it, please!

[Ian Bicking]
> It seems to work okay for me -- I think David might have some features
> he'd like, but what's there seems sufficient to me.

I'll try to take a look at your code soon, but no guarantees.

> I'm realizing that reordering the methods might be best anyway --
> usually they don't match up with the description because they were just
> written in a certain order.

Personally, I'm not worried about the literate programming aspects at this
point.  Once the basic infrastructure is in place, I hope it will be easy to
try different approaches.  So I'm not going to get into this discussion
(plenty on my plate as it is), but I encourage anyone interested to explore
it.

> The whole hyperlinking thing I'm still unsure about.  I don't even know
> if there's a way to indicate that `references` should be turned into
> some sort of link.

The intention is that when using the Python Source Reader component, the
default role for interpreted text will be something like
"python-identifier".  Each identifier will be resolved in the current
namespace (i.e., `self.x` is a method, or an attribute) and hyperlinked if
possible.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Interpreted text for terms?

[David G. <go...@py...>]

Mark Nodine wrote:
> What I'm wondering how to do in RST is what was represented
> between tildes in structure-enhanced text.  It is
> obviously closely related to interpreted text, and
> in fact, I always interpreted it as terms to be included
> in the index.  For example, ::
> 
>    A ~behavioral bus monitor~ is a module written in the same
...
> 
> In this example, the term is actually defined in the text
> of the document (rather than a separate glossary).  It is
> typeset as italics, but, and this is the key, it also creates
> an implicit target so that I can refer to this term from
> elsewhere in the document (and especially from the index).

I wasn't aware of this feature of Setext.  I thought ~tildes~ simply
indicated italics; see the files in
<http://docutils.sourceforge.net/mirror/setext/>, especially typotags.txt.
Are you using a different Setext description, perhaps derived later?

> Is there a way to make interpreted text (perhaps with specific
> role types) create implicit targets?  Or is there some other
> way I should be going about this?

As Benja pointed out,

    An _`inline target` does the trick

And you can make it italic with a stylesheet entry.  The above is rendered
as ``<a class="target" ...>``.

If you want a solution that doesn't resort to stylesheet trickery, then an
interpreted text role would be appropriate.  The role could simply wrap the
text with both emphasis and target elements.

-- David Goodger    http://starship.python.net/~goodger

Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv

Re: [Docutils-develop] Interpreted text for terms?

[Mark N. <no...@so...>]

Benja Fallenstein wrote:
> 
> Hi Mark,
> 
> Mark Nodine wrote:
> > In this example, the term is actually defined in the text
> > of the document (rather than a separate glossary).  It is
> > typeset as italics, but, and this is the key, it also creates
> > an implicit target so that I can refer to this term from
> > elsewhere in the document (and especially from the index).
> >
> > Is there a way to make interpreted text (perhaps with specific
> > role types) create implicit targets?  Or is there some other
> > way I should be going about this?
> 
> Hey, if you don't need italics, how about using explicit targets? :)
> Like this:
> 
>      An _`inline target` looks almost like a hyperlink.
> 
> Doesn't look more wordy than any interpreted text, to me ;)

But I _do_ want italics to draw the reader's eye to the
text after following the link (or point his/her attention
to the term I'm defining).

	--Mark