docutils-develop — For developer discussions of the implementation.

Re: [Docutils-develop] split output (was Re: sandbox/tibs/pysource/notes thoughts.txt)

[Richard J. <rj...@ek...>]

On Tue, 15 Oct 2002 1:17 am, David Goodger wrote:
> One thing to take out of this, is to compare how different tools
> generate their collections of files.  Epydoc makes one directory with
> a slew of files, some with *very* long names, like
> "docutils.parsers.rst.states.EnumeratedList.html".  I'd much prefer to
> use the filesystem to organize files hierarchically, as in
> "docutils/parsers/rst/states/EnumeratedList.html".  This was one of
> the reasons I got interested in auto-documentation in the first place:
> pythondoc did the long name thing, and MacOS classic couldn't handle
> it with its 38-char filename limit.

Note that OSX still has that limit in places (tar, of all things, unless it's 
been fixed in a recent patch release).


> So here's a requirement: when splitting files, each split level should
> generate a new directory rather than another segment of a compound file
> name.

Seems reasonable enough to me, as long as there's a good TOC :)


   Richard

Re: [Docutils-develop] split output (was Re: sandbox/tibs/pysource/notes thoughts.txt)

[David G. <go...@us...>]

[Richard]
>> Haven't looked at javadoc for a while - what's this mean?

[Tibs]
> As David guessed, it means the sort of thing that javadoc does -
> specifically, it produces a directory structure, containing (in
> general) an HTML file for each class, a directory for each package,
> index and contents pages, maybe a summary page for each package, and
> (I think) other bumf as well.

One thing to take out of this, is to compare how different tools
generate their collections of files.  Epydoc makes one directory with
a slew of files, some with *very* long names, like
"docutils.parsers.rst.states.EnumeratedList.html".  I'd much prefer to
use the filesystem to organize files hierarchically, as in
"docutils/parsers/rst/states/EnumeratedList.html".  This was one of
the reasons I got interested in auto-documentation in the first place:
pythondoc did the long name thing, and MacOS classic couldn't handle
it with its 38-char filename limit.

So here's a requirement: when splitting files, each split level should
generate a new directory rather than another segment of a compound file
name.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] docutils api docs

[David G. <go...@us...>]

> David Goodger wrote:
>> I recently moved the bibliographic fields that were in all the
>> module docstrings into simple comments.  This gets rid of ":Author:
>> David Goodger :Contact: goodger@users." from the API docs of almost
>> EVERY SINGLE MODULE ENTRY!

Tony J Ibbs (Tibs) wrote:
> I'd wondered why you'd done that - it seems like docstring-relevant
> stuff to me. Is it just embarassment over having your name everywhere? I
> mean, if you wrote it...

It wasn't in anticipation of Richard producing API docs -- those were an
unexpected surprise.  It was in anticipation of eventually producing API
docs; it seemed overly verbose to include that info in every module's
docstring.  The epydoc API docs merely confirmed what I had suspected.  I
began moving them in modules I was editing for some other reason, and then
one day I did a mass edit, made easy thanks to Emacs' directory-edit-mode
and query-replace tools.

Yes, I wrote much of the code, but a comment is good enough.  The developer
looking for API clues doesn't need to see my name front & center.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

RE: [Docutils-develop] docutils api docs

[Tony J I. (Tibs) <to...@ls...>]

David Goodger wrote:
> I recently moved the bibliographic fields that were in all the
> module docstrings into simple comments.  This gets rid of ":Author:
> David Goodger :Contact: goodger@users." from the API docs of almost
> EVERY SINGLE MODULE ENTRY!

I'd wondered why you'd done that - it seems like docstring-relevant
stuff to me. Is it just embarassment over having your name everywhere? I
mean, if you wrote it...

Tibs
--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.co.uk/
Give a pedant an inch and they'll take 25.4mm
(once they've established you're talking a post-1959 inch, of course)
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)

RE: [Docutils-develop] split output (was Re: sandbox/tibs/pysource/notes thoughts.txt)

[Tony J I. (Tibs) <to...@ls...>]

Gosh - I come in Monday morning and there's email to reply to...
I'll try to amalgamate several replies in this thread...

Early on, Richard Jones wrote:
>> One specific thing to be decided, particularly for HTML, is
>> whether one is outputting a "cluster" of files (e.g., as javadoc
>> does).
>
> Haven't looked at javadoc for a while - what's this mean?

As David guessed, it means the sort of thing that javadoc does -
specifically, it produces a directory structure, containing (in general)
an HTML file for each class, a directory for each package, index and
contents pages, maybe a summary page for each package, and (I think)
other bumf as well. One single Java source file can, all by itself,
produce a whole set of files.

  ( So, Richard, is that resolved now? :)

Now, sometimes I want all of my documentation in one document (it's a
bugbear of mine about the current Python development docs that you only
get this form published when they're released, not when they're being
developed - chapters in lots of little bits, yuck), and sometimes I want
it split up for fast loading, etc. I imagine we *will* want to provide
these options for pysource.

On the other hand, this seems to me like a minor issue compared to
getting the *information* right - that is, sorting out the data mining,
the joining up of `interpreted-Python` links, etc.

On the other other hand, early adoption probably *does* depend on
getting nice presentation. Life's fun that way.

Richard Jones wrote:
> It's going to have to be many generic doctrees though. That is, the
"stylist
> transforms" are likely to produce multiple output doctrees (an index,
a
> package page, multiple module pages, ...)

David Goodger replied:
> I don't know how this will work yet.  Perhaps the Writer and/or I/O
object
> will handle forests (multiple trees).  Perhaps the doctree will remain
> monolithic, and a transform will insert "split here" indicators.  It's
all
> up in the air.

I can certainly see other document-producers wanting to split output
(the obvious case being people doing "book" production, where individual
chapter output can be quite useful). I suspect, however, that pysource
would be a good environment to test out ideas, since it probably has the
most extreme requirements (or perhaps that should be "wish list").

Richard Jones then wrote:
> The "split here" part worries me - that then implies the HTML Writer
knows how
> to output multiple files, which I don't believe it should. It should
be the
> job of the pysource "framework" (the bit that drives the parse ->
transform
> -> writer mechanisms) to decide what files need to be written and how
they
> should be written.

to which David responded:
> Problem is, PySource is a *Reader*, which doesn't know and shouldn't
care
> about final output.

Well, no, pysource is *either* a module providing facilities callable by
other software, *or* the default application using said module. And in
either case, it will be at least a Reader and a set of transformers for
the output of said reader (after all, it has to transform stuff from its
"special" docutils tree into the generic tree, and there will be options
on how to do that).

Anyway, if, as Richard wants (and I had assumed) the default way of
working is to read in "everything we care about" (gosh, memory) so that
one can get overview contents and index pages, and then output it as
single or multiple documents, then I think that David is right - one of
the transform phases needs to identify where to split to produce a
forest from the original tree (since we're assuming a standard Writer,
we need to split things up into separate document trees before it gets
fed things - i.e., it's easiest to imagine that a Writer only writes one
file, at least for the moment).

That splitting can either be on explicit break-points (which makes for a
very simple splitting utility), or might be on designated locations -
e.g., for each class. That latter would assume that more "fix up" work
would be needed.

In short, I see the process as something like (and my terminology is
going to be all astray again - sorry::

    PySourceReader
      :produces:
    <PySource doctree>
      :through:
    PySourceTransformer
      :produces:
    <Standard doctree>
      :through:
     StandardWriter

*or*::

                PySourceReader
                  :produces:
               <PySource doctree>
                  :through:
               PySourceTransformer
                  :produces:
    <Standard doctree1> <Standard doctree2> ...
      :through:           :through:
     StandardWriter      StandardWriter     ...

If/when we get a "generic" split transform, we might shift that to::

                PySourceReader
                  :produces:
               <PySource doctree>
                  :through:
               PySourceTransformer
                  :produces:
              <Standard doctree1>
                  :through:
              <SplitTransformer>
                  :produces:
     <Standard doctreeA>  <Standard doctreeB>
      :through:            :through:
     StandardWriter       StandardWriter     ...

However, whilst this is all fun, I think it's the technically easy bit
(the *design* of this is the hard bit), and for pysource itself, output
is still the least of our worries (having said that, this is all
volunteer work, and if someone thinks they'd rather fix this lacuna
first, then that's their business).

Tibs, who must stop wittering and go back to work.

--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.co.uk/
.. "equal" really means "in some sense the same, but maybe not
.. the sense you were hoping for", or, more succinctly, "is
.. confused with". (Gordon McMillan, Python list, Apr 1998)
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)

[Docutils-develop] split output (was Re: sandbox/tibs/pysource/notes thoughts.txt)

[David G. <go...@us...>]

[David]
>>>> I don't know how this will work yet.  Perhaps the Writer and/or I/O
>>>> object will handle forests (multiple trees).  Perhaps the doctree will
>>>> remain monolithic, and a transform will insert "split here" indicators.
>>>> It's all up in the air.

[Richard]
>>> The "split here" part worries me - that then implies the HTML Writer
>>> knows how to output multiple files, which I don't believe it should. It
>>> should be the job of the pysource "framework" (the bit that drives the
>>> parse -> transform -> writer mechanisms) to decide what files need to be
>>> written and how they should be written.

[David]
>> Problem is, PySource is a *Reader*, which doesn't know and shouldn't care
>> about final output.

[Richard]
> I don't think pysource is going to be able to get away with being *just* a
> Reader. The one-source-to-many issue kinda throws a huge spanner in the
> works.

The one-source-to-many issue is pervasive, not specific to PySource at all.
For example, I'd like to be able to split the "reStructuredText Markup
Specification" into multiple .html files, one per first-level section (and
perhaps even one per second-level section).  "The Docutils Document Tree: A
Guide to the Docutils DTD", whose HTML is now up to 208K, is becoming much
too big.  And it's far from finished!

I agree that the HTML Writer shouldn't have to know how to split & write
multiple files.  This may be a job for a specialized I/O object, like
"MultiFileOutput".  It will need some earlier support though, such as
reference adjustment (any given reference may need to point to
``href="file#id"`` instead of just ``href="#id"``).  Perhaps I/O objects
should become full-fledged components (i.e. subclasses of
``docutils.Component``, as are Readers, Parsers, and Writers now), and thus
have associated option/setting specs and transforms.

In any case, we should try to find an elegant, general solution to the
problem.  We may not get there with the first attempt, or even the second,
but that's OK.  One thing this project has taught me is to let go of code,
to be willing to sacrifice it when it has proven unfit.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Publishing to string

[David G. <go...@us...>]

Aahz wrote:
> Well, success at last!  I have now produced an OpenOffice.org
> document completely programmatically.

Congrats!

> I have to say that this is a Bad Design for Publisher.  "Ordinary
> setup" shouldn't requre specific ordering; at the very least, there
> ought to be error checking higher up in the call chain.

Noted.  Patches and **constructive criticism** welcome.

> If people want to see my code, I'm willing to stick it in a sandbox,

Are you "aahz" on SourceForge?

> but I really, really don't want comments about how ugly it is.

And here you are calling "Bad Design" on *my* code.  You can dish it
out but can't take it, hmm?  ;-)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Publishing to string

[Aahz <aa...@py...>]

On Fri, Oct 11, 2002, David Goodger wrote:
> Aahz wrote:
>> 
>>     pub = core.Publisher()
>>     options = pub.set_options()
>>     pub.source = io.FileInput(options, source_path=sys.argv[1])
>>     pub.destination = io.StringOutput(options)
>>     pub.set_reader('standalone', None, 'restructuredtext')
>>     pub.writer = OOwriter.Writer
> 
> Is "OOwriter.Writer" a class or an object?  "pub.writer" needs to be
> an object.

That I figured out on my own.  Here's what works::

    pub = core.Publisher(writer=OOwriter.Writer())
    pub.set_reader('standalone', None, 'restructuredtext')
    options = pub.set_options()
    pub.source = io.FileInput(options, source_path=sys.argv[1])
    pub.destination = io.StringOutput(options)
    content = pub.publish()

> Sorry, I forgot that you need to set up the reader & writer *before*
> calling pub.set_options().  The reader, parser, and writer
> (collectively, the "components") specify the available options along
> with their default values.

Well, success at last!  I have now produced an OpenOffice.org document
completely programmatically.  It's rough and ugly, but it will be enough
to make my publishers happy that I haven't completely screwed up.

I have to say that this is a Bad Design for Publisher.  "Ordinary setup"
shouldn't requre specific ordering; at the very least, there ought to be
error checking higher up in the call chain.

If people want to see my code, I'm willing to stick it in a sandbox, but
I really, really don't want comments about how ugly it is.  Constructive
criticism that takes less than fifteen minutes to implement is welcome,
though.
-- 
Aahz (aa...@py...)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/

Re: [Docutils-develop] Publishing to string

[Aahz <aa...@py...>]

On Fri, Oct 11, 2002, David Goodger wrote:
> Aahz wrote:
>>
>> Actually, it now looks like a snapshot bug.
> 
> What's a snapshot bug?

It's a bug that exists because someone checked in code with a bug.  I
call it a snapshot bug because they tend to be bugs that would have been
caught with an appropriate regression test.  It appears that I was
wrong, though.

> But I'm too busy to do it myself, and I don't feel the itch.  I'll add
> it to the to-do list, and await a champion.

I'm too busy myself...  ;-)  (Seriously, I hate being dependant on
pre-alpha software -- which is what I've done to myself here -- but I'm
overall still convinced I made the best choice.  Nevertheless, the real
work of writing a book has to take precedence.)
-- 
Aahz (aa...@py...)           <*>         http://www.pythoncraft.com/

Project Vote Smart: http://www.vote-smart.org/

Re: [Docutils-develop] Re: sandbox/tibs/pysource/notes thoughts.txt

[Richard J. <rj...@ek...>]

On Sat, 12 Oct 2002 11:01 am, David Goodger wrote:
> > On Sat, 12 Oct 2002 10:37 am, David Goodger wrote:
> >> I don't know how this will work yet.  Perhaps the Writer and/or I/O
> >> object will handle forests (multiple trees).  Perhaps the doctree will
> >> remain monolithic, and a transform will insert "split here" indicators. 
> >> It's all up in the air.
>
> Richard Jones wrote:
> > The "split here" part worries me - that then implies the HTML Writer
> > knows how to output multiple files, which I don't believe it should. It
> > should be the job of the pysource "framework" (the bit that drives the
> > parse -> transform -> writer mechanisms) to decide what files need to be
> > written and how they should be written.
>
> Problem is, PySource is a *Reader*, which doesn't know and shouldn't care
> about final output.  HappyDoc may have already tackled this issue;
> analyzing its code is on my to-do.  But also on my to-do are window trim,
> painting, grouting, baseboard mouldings, winterizing the garage, and on and
> on...

I don't think pysource is going to be able to get away with being *just* a 
Reader. The one-source-to-many issue kinda throws a huge spanner in the 
works.


> > Then this is definitely an issue that must be resolved now ;)
>
> I hope you're volunteering :-)

Heh, sure. It can go on my TODO along with Roundup 0.5.1, weeding and 
generally attacking the garden, finish the bedside tables, hallstand, 
bookcases, checking the roof after our recent strong winds, ... :)


   Richard

Re: [Docutils-develop] Publishing to string

[Richard J. <rj...@ek...>]

On Sat, 12 Oct 2002 10:50 am, David Goodger wrote:
> Aahz wrote:
> > Actually, it now looks like a snapshot bug.
>
> What's a snapshot bug?

A bug that's present when the snapshot is taken :)

[but then, I think you got that from the response you posted ;)]


    Richard

Re: [Docutils-develop] docutils api docs

[Richard J. <rj...@ek...>]

On Sat, 12 Oct 2002 10:45 am, David Goodger wrote:
> Richard Jones wrote:
> > This seemed like a good idea, hope I haven't overstepped the mark,
> > David.
>
> Using the competition?!?  Traitor!  How dare you?
>
> ;-)
>
> Sure, it's fine.  If it's useful to anyone, I have no objections.
> We're sure to get some good ideas from epydoc (and HappyDoc etc.).
> Eventually, of course, it will be replaced with the One True API
> Docset. ;-)

Or specifically, offer an output mode that _doesn't_ look like the (IMHO) ugly 
pydoc/happydoc page structure. Euwww :)



> One nit, though.  Could you replace it with a run over the curremt CVS
> code?  I recently moved the bibliographic fields that were in all the
> module docstrings into simple comments.  This gets rid of ":Author:
> David Goodger :Contact: goodger@users." from the API docs of almost
> EVERY SINGLE MODULE ENTRY!  Sheesh!  You'd think the guy was conceited
> or something.

Heh - sorry, I hadn't cvs up'ed for a while. Will do this weekend.


    Richard

Re: [Docutils-develop] Re: sandbox/tibs/pysource/notes thoughts.txt

[David G. <go...@us...>]

> On Sat, 12 Oct 2002 10:37 am, David Goodger wrote:
>> I don't know how this will work yet.  Perhaps the Writer and/or I/O object
>> will handle forests (multiple trees).  Perhaps the doctree will remain
>> monolithic, and a transform will insert "split here" indicators.  It's all
>> up in the air.

Richard Jones wrote:
> The "split here" part worries me - that then implies the HTML Writer knows how
> to output multiple files, which I don't believe it should. It should be the
> job of the pysource "framework" (the bit that drives the parse -> transform
> -> writer mechanisms) to decide what files need to be written and how they
> should be written.

Problem is, PySource is a *Reader*, which doesn't know and shouldn't care
about final output.  HappyDoc may have already tackled this issue; analyzing
its code is on my to-do.  But also on my to-do are window trim, painting,
grouting, baseboard mouldings, winterizing the garage, and on and on...

> Then this is definitely an issue that must be resolved now ;)

I hope you're volunteering :-)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] docutils api docs

[David G. <go...@us...>]

Richard Jones wrote:
> It has some really good ideas in the output

Agreed.

> I haven't had a chance to look into what it takes to play with the
> current docutils pysource effort to produce a similar output.

Unless you've made some radical changes to the sandbox/tibs/pysource
code (I confess that I haven't looked it over thoroughly), it still
takes a very different path to create its output.  You (and anybody
interested) should read these first:

* http://docutils.sf.net/spec/pysource.html
* http://docutils.sf.net/spec/pep-0258.html#python-source-reader
* http://docutils.sf.net/sandbox/tibs/pysource/notes/thoughts.html

The last one consists of Tibs' parting notes summing up his ideas
about where to go from here, and is especially relevant.  I see you
caught it from docutils-checkins though.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] QuickRef (was RE: Figure references)

[David G. <go...@us...>]

Tony J Ibbs (Tibs) wrote:
> David may answer this better than I can (he's *much* better at
> designing directives!).
> 
> But the idea, as I would see it, is that one doesn't write::
> 
>     +-----+--------------------+
>     | raw | the same formatted |
>     +-----+--------------------+
> 
> but instead something like::
> 
>     ..examples::
>       rst::
>          The *raw* text we want
>       rst::
>          Another such.

I see, you're using "rst::" as a delimiter between examples,
effectively generating rows in an "example table".  Since "rst" is
assumes, this could be done more easily, perhaps with indentation and
".."::

    .. examples::
           The *raw* text we want.
       ..
           Another such.

>     ..examples::
>       rst::
>          The *raw* text we want.
> 
>          Which spans two paragraphs.
>       rst::
>          And might be a title
>          ====================
>          Or subtitle
>          -----------

The latter would be tricky, since section headers can only occur in
certain contexts (e.g., not within a block quote, and certainly not
within a table).  They may have to be simulated somehow.

> Since a directive can do "anything it likes" (that *may* be a quote)
> with the stuff "inside" it, this approach seems like it should work...

Given enough effort, anything's possible. :-)

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] QuickRef (was RE: Figure references)

[David G. <go...@us...>]

Engelbert Gruber wrote:
> I donot beleive yet
> 
> +-----+--------------------+
> | raw | the same formatted |
> +-----+--------------------+
> where  do you put the "::", inside the cell ?
> does this work ? 
> i thought the lines on the left must be marked literal
> each.

In a brute-force way, like this::

    +-------+--------------------+
    | ::    |                    |
    |       |                    |
    |   raw | the same formatted |
    +-------+--------------------+

or like this::

    +---------+--------------------+
    | ``raw`` | the same formatted |
    +---------+--------------------+

Try it, it works!

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] QuickRef (was RE: Figure references)

[David G. <go...@us...>]

[Tony, re "fold-in" and "call-out" references]
> The names of the two styles certainly don't help - good naming is
> one of the hardest bits of any task, and I don't think I did
> particularly well there. On the other hand, they *are* two different
> things, and both styles are important - what *are* they called?

Yes, the proper names would help.  I have the feeling I saw the name
of the hardcopy style somewhere, but I haven't been able to find it.

> However, in that case I still think the quickref has to say "well,
> there are these two possible styles of output, and you'll get the
> one best suited to your output medium"

A note saying that would be fine.  I just think it's over-emphasized
and under-justified at present.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Publishing to string

[David G. <go...@us...>]

Aahz wrote:
> Actually, it now looks like a snapshot bug.

What's a snapshot bug?

> I've been mulling over whether I should suggest this,

Never hesitate to suggest something potentially useful!

> but it seems to me that snapshots probably ought to pass a
> regression suite before being uploaded, and perhaps only get updated
> once per day or so.  Seems to me that anyone who really cares about
> the latest files can grab them from CVS, and snapshots would have a
> bit more consistency to them, even if they're not formal releases.

The snapshots were originally put in place for people who, for
whatever reason, *don't* have access to CVS.  They know who they
are. ;-)  But I find snapshots useful too.

I used to have the snapshots updated once per day, but then I'd find
myself racing to commit files before the cutoff time, or manually
updating them if I was late.  The former is ridiculous, and the latter
a waste of time.  So I switched to hourly.

I'm all for adding more tests.  I (almost) always run
docutils/test/alltests.py before committing anything, which runs all
the unit tests under the docutils/test directory.  The test suite
doesn't have any regression or functional testing yet.  Any
volunteers?

And I'd be in favor of making passing the test suite a prerequisite to
snapshot update, but only if the process were completely automatic.
The update script (in CVS at
sandbox/davidg/infrastructure/docutils-update) could be modified to
only update the snapshots if files have actually changed in CVS
(saving some SourceForge server cycles), and run the tests before
changing the snapshot.  It could even be rewritten in Python!

But I'm too busy to do it myself, and I don't feel the itch.  I'll add
it to the to-do list, and await a champion.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Re: sandbox/tibs/pysource/notes thoughts.txt

[Richard J. <rj...@ek...>]

On Sat, 12 Oct 2002 10:37 am, David Goodger wrote:
> > On Sat, 12 Oct 2002 8:45 am, David Goodger wrote:
>
> Actually, Tibs wrote it.  I just checked it in to CVS.

Sorry, slackness on my part there.


> >> Each layout is represented by a class that walks the
> >> Python-specific doctree, replacing the Python-specific nodes with
> >> appropriate generic nodes. The output of the Layout Transformer
> >> is thus a generic doctree.
>
> Richard Jones wrote:
> > It's going to have to be many generic doctrees though. That is, the
> > "stylist transforms" are likely to produce multiple output doctrees (an
> > index, a package page, multiple module pages, ...)
>
> I don't know how this will work yet.  Perhaps the Writer and/or I/O object
> will handle forests (multiple trees).  Perhaps the doctree will remain
> monolithic, and a transform will insert "split here" indicators.  It's all
> up in the air.

The "split here" part worries me - that then implies the HTML Writer knows how 
to output multiple files, which I don't believe it should. It should be the 
job of the pysource "framework" (the bit that drives the parse -> transform 
-> writer mechanisms) to decide what files need to be written and how they 
should be written.


> >> One specific thing to be decided, particularly for HTML, is
> >> whether one is outputting a "cluster" of files (e.g., as javadoc
> >> does).
> >
> > Haven't looked at javadoc for a while - what's this mean?
>
> I think "cluster" here just means multiple files as opposed to one
> monolithic file.

Then this is definitely an issue that must be resolved now ;)


   Richard

Re: [Docutils-develop] Publishing to string

[David G. <go...@us...>]

Aahz wrote:
> I tried this recipe David gave to Michael Hudson::
> 
>     pub = core.Publisher()
>     options = pub.set_options()
>     pub.source = io.FileInput(options, source_path=sys.argv[1])
>     pub.destination = io.StringOutput(options)
>     pub.set_reader('standalone', None, 'restructuredtext')
>     pub.writer = OOwriter.Writer

Is "OOwriter.Writer" a class or an object?  "pub.writer" needs to be
an object.

>     content = pub.publish()
> 
> and got this traceback::

Sorry, I forgot that you need to set up the reader & writer *before*
calling pub.set_options().  The reader, parser, and writer
(collectively, the "components") specify the available options along
with their default values.

I'm considering renaming "options" (and all its uses) to "settings".
Once the command line is processed, they're not options any more.
When running Docutils programmatically, there are no true "options" to
be found.

> I'll probably find the answer before I get a response.  ;-)

Unfortunately, I have to answer mail in batch mode, evenings (EST).
But I've found though experience that (reasonable) delays in
responding to requests often result in the originators figuring out
the solutions on their own.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] docutils api docs

[David G. <go...@us...>]

Richard Jones wrote:
> This seemed like a good idea, hope I haven't overstepped the mark,
> David.

Using the competition?!?  Traitor!  How dare you?

;-)

Sure, it's fine.  If it's useful to anyone, I have no objections.
We're sure to get some good ideas from epydoc (and HappyDoc etc.).
Eventually, of course, it will be replaced with the One True API
Docset. ;-)

One nit, though.  Could you replace it with a run over the curremt CVS
code?  I recently moved the bibliographic fields that were in all the
module docstrings into simple comments.  This gets rid of ":Author:
David Goodger :Contact: goodger@users." from the API docs of almost
EVERY SINGLE MODULE ENTRY!  Sheesh!  You'd think the guy was conceited
or something.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Writing directives

[David G. <go...@us...>]

Aahz wrote:
> It appears that simply declaring something Inline doesn't do
> anything;

Correct.  It's just for classification purposes, so that later
transforms can query a node with isinstance().

> I'm assuming I need to run a transform in the writer?

No.  By the time the document gets to the Writer, it should be in
final form.  The Writer's job is simply (and only) to translate from
the Docutils doctree structure to the target format.  Some small
transforms may be required, but they should be local and
format-specific.  The "index_entry" node class is not format-specific
and should become a standard Docutils node/element (i.e. part of
nodes.py and docutils.dtd).  We can figure out its syntax (attributes
& content model) later.

> Or should the directive find its parent/predecessor-sibling during
> parsing?

I think so.  Given::

    Here's a paragraph.

    .. index:: paragraph

The directive should produce something like this XML::

    <paragraph>
    <index_entry text="paragraph"/>
    Here's a paragraph.
    </paragraph>

Thus the "nodes.Inline" base class; it's an inline element.  This kind
of content model would also allow true inline index-entries::

    Here's a `paragraph`:index:.

If the "index" role were the default for the application, it could be
dropped::

    Here's a `paragraph`.

Both of these would result in this XML::

    <paragraph>
    Here's a <index_entry>paragraph</index_entry>.
    </paragraph>

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Re: sandbox/tibs/pysource/notes thoughts.txt

[David G. <go...@us...>]

> On Sat, 12 Oct 2002 8:45 am, David Goodger wrote:

Actually, Tibs wrote it.  I just checked it in to CVS.

Read it here: 
http://docutils.sf.net/sandbox/tibs/pysource/notes/thoughts.html

>> Each layout is represented by a class that walks the
>> Python-specific doctree, replacing the Python-specific nodes with
>> appropriate generic nodes. The output of the Layout Transformer
>> is thus a generic doctree.

Richard Jones wrote:
> It's going to have to be many generic doctrees though. That is, the "stylist
> transforms" are likely to produce multiple output doctrees (an index, a
> package page, multiple module pages, ...)

I don't know how this will work yet.  Perhaps the Writer and/or I/O object
will handle forests (multiple trees).  Perhaps the doctree will remain
monolithic, and a transform will insert "split here" indicators.  It's all
up in the air.

>> One specific thing to be decided, particularly for HTML, is
>> whether one is outputting a "cluster" of files (e.g., as javadoc
>> does).
> 
> Haven't looked at javadoc for a while - what's this mean?

I think "cluster" here just means multiple files as opposed to one
monolithic file.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

[Docutils-develop] Re: sandbox/tibs/pysource/notes thoughts.txt

[Richard J. <rj...@ek...>]

On Sat, 12 Oct 2002 8:45 am, David Goodger wrote:
> I reckon that one produces a Python-specific doctree, and then
> chooses one of several transforms to produce generic doctrees.

A sound plan.


> By the way, I still vote for "static" introspection

+1


> One of the tasks undertaken by the Python Source Reader is to
> decide what to do with docstrings.  For each Python file, it
> discovers (from ``__docformat__``) if the input parser is
> "reStructuredText". If it is, then the contents of each docstring
> in that file will be parsed as such, including sorting out
> interpreted text (i.e., rendering it into links across the tree).
> If it is not, then each docstring will be treated as a literal
> block.
>
>   This admits the posibility of adding other parsers *for
>   docstrings* at a later date - which I suspect is how HappyDoc
>   does it. It is just necessary to publicise the API for the
>   Docstring class, and then someone else can provide alternate
>   plugins.
>
> The output of the Python Source Reader is thus a Python-specific
> doctree, with all docstrings fully processed (as appropriate),
> and held inside <docstring> elements.
>
> The next stage is handled by the Layout Transformer.
>
>     [What I call "stylist transforms".  --DG]
>
> This determines the layout of the document to be produced - the
> *kind* or *style* of output that the user wants (e.g., PyDoc
> style, LibRefMan style, generic docutils style, etc.).
>
> Each layout is represented by a class that walks the
> Python-specific doctree, replacing the Python-specific nodes with
> appropriate generic nodes. The output of the Layout Transformer
> is thus a generic doctree.

It's going to have to be many generic doctrees though. That is, the "stylist 
transforms" are likely to produce multiple output doctrees (an index, a 
package page, multiple module pages, ...)

The resultant docutils doctrees are passed via normal transforms to the normal 
HTML writer.

I can see the multiple page output being useful for documenting larger volumes 
of text - it'd be really nice if my Roundup documentation could all be parsed 
in one go and then I could have inter-document references verified (and point 
to specific anchors, oh my :), have the top-level Table Of Contents for all 
documents generated, ...


> One specific thing to be decided, particularly for HTML, is
> whether one is outputting a "cluster" of files (e.g., as javadoc
> does).

Haven't looked at javadoc for a while - what's this mean?


    Richard

Re: [Docutils-develop] docutils api docs

[Richard J. <rj...@ek...>]

On Fri, 11 Oct 2002 7:06 pm, Tony J Ibbs (Tibs) wrote:
> Richard Jones wrote:
> > I lieu of a "usable" version of the docutils python API doc
> > extractor, I've run epydoc over the docutils tree, producing:
> >
> >   http://docutils.sourceforge.net/apidoc/docutils.html
>
> Ooh - that's seriously pretty looking stuff!
>
> (whether it's *useful* I'll leave to others, since I've not time to look
> it over, but it is *does* look nice)

It has some really good ideas in the output (the structure of the pages, and 
the class heirarchies) that'd be nice to mimic in the docutils output - which 
at the moment is One Big Page :)

I haven't had a chance to look into what it takes to play with the current 
docutils pysource effort to produce a similar output.


   Richard