docutils-develop

[Docutils-develop] Parsing oddness

[Richard J. <rj...@ek...>]

I've got some problems in parsing a document. The following biblio element 
results in errors:

>>>>> snip
:Organization: Computer and Information Sciences::

                 New Jersey Institute of Technology
                 University Heights
                 Newark, NJ, 07102
                 (address obsolescent).
<<<<< snip

Reporter: WARNING (2) Literal block expected at line 2; none found.
Reporter: WARNING (2) Cannot extract compound bibliographic field 
"Organization".


I've hit another error in a definition list... note the line number the error 
is reported at, and the line that is actually erroneous (the last one).

>>>>> snip
S.member(ob), D.member(arg,map), G.member(src,dst)
  respectively are membership tests for the types. Each returns 1 if the
  object or pair are members of the structure or 0 otherwise.

S.add(ob), D.add(arg,map), G.add(src,dst)
  respectively add new members to the object. These are equivalent to
  G[src]=dst, D[arg]=map, S[ob]=1 but the former may be preferrable for
  graphs and sets since they are less misleading. This is an "in
  place" mutation operation -- it will raise an error if the object has
  been hashed.

X.items()
  returns a list of the members of the structure. For example::

    >>> X = kjSet([0, 1, 2, 0, 1])
    >>> X.items()
    [1, 0, 2]
    >>> X = kjGraph([(3, 0), (2, 2), (1, 2), (2, 0), (2, 0), (3, 0)])
    >>> X.items()
    [(1, 2), (3, 0), (2, 2), (2, 0)]

G.keys(), G.values()
  return the left members and right members of pairs in the graph G
  respectively. For example::

    >>> G = kjGraph([(4, 8), (0, 9), (1, 10), (4, 9), (3, 7), (3, 8), (2,
    >>> 7)])
    >>> G.keys()
    [4, 0, 1, 3, 2]
    >>> G.values()
    [8, 9, 10, 9, 7, 8, 7]

  Note that keys eliminates redundancies, whereas values does not. These
  functions are also defined for dictionaries but are not defined for
sets.

<<<<< snip

Reporter: WARNING (2) Definition list ends without a blank line; unexpected 
unindent at line 5.



     Richard

Re: [Docutils-develop] Parsing oddness

[David G. <go...@us...>]

Richard Jones wrote:
> I've got some problems in parsing a document. The following biblio
> element results in errors:
> 
> >>>>> snip
> :Organization: Computer and Information Sciences::
> 
>                  New Jersey Institute of Technology
>                  University Heights
>                  Newark, NJ, 07102
>                  (address obsolescent).
> <<<<< snip

I'll take the warnings in reverse order.

> Reporter: WARNING (2) Cannot extract compound bibliographic field
> "Organization".

This is by design of the document model.  The "organization" element
currently may only contain text, not body elements.  It was intended
to only contain the organization's name.  You present a good example
of why the status quo perhaps ought to change.  Requires thought.

However, a literal block isn't really the ideal way to represent an
address block, is it?  I've been mulling over an idea for a "verse"
directive which seems to apply here.  See
http://docutils.sf.net/spec/notes.html#body-verse [#]_. What do you
think?  How about that ';;' syntax?

.. [#] Just updated.  Especially note the examples; I love having
   *Monty Python's Flying Circus: Just The Words* and *The Fairly
   Incomplete & Rather Badly Illustrated Monty Python Songbook* on my
   reference shelf, and I refer to them frequently (can you tell?).
   Makes programming fun!  Thank Guido for Python!

> Reporter: WARNING (2) Literal block expected at line 2; none found.

This is a tricky one.  Because field list field names can be quite
long, the spec allows for arbitrary minimum indentation of the second
and subsequent lines [#]_, without inferring any significance.  In
other words, the address block is not seen as being indented at all,
but just as a second paragraph.  So as far as the parser is concerned,
there *is* no literal block to be found!

I think this is correct behavior; perhaps the docs need some
clarification.  I invite counter-arguments, to either of the preceding
statements.  ;-)

.. [#] See
   http://docutils.sf.net/spec/rst/reStructuredText.html#field-lists,
   end of third paragraph, and
   http://docutils.sf.net/spec/rst/reStructuredText.html#indentation,
   last paragraph before the literal block.

To rectify this problem, just rewrite that field list item as
follows::

    :Organization:
        Computer and Information Sciences::

            New Jersey Institute of Technology
            University Heights
            Newark, NJ, 07102
            (address obsolescent).

Now the indentation of the literal block *is* significant (it's
different from the indentation of the second line).  However, you'll
either have to wait for a change to the document model (not
guaranteed!), or rewrite the field list item, either as a single
paragraph, or something like this::

    :Organization: Computer and Information Sciences,
                   New Jersey Institute of Technology [#]_

    .. [#]
       ::

           University Heights
           Newark, NJ, 07102
           (address obsolescent).

Note that the '::' has to be on the second line of the footnote, for
the same reason as given above for the field list item.  This behavior
may change, however (there's a to-do list entry that begins with "Fix
the parser's indentation handling" in
http://docutils.sf.net/spec/notes.html#restructuredtext-parser).

> I've hit another error in a definition list... note the line number
> the error is reported at, and the line that is actually erroneous
> (the last one).

I see it.  I'll see about fixing the error reporting.  It may take a
bit of thought though, so don't hold your breath.

As always, thanks for the feedback.  It will make its way into the
code and/or the docs, anon.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Parsing oddness

[Richard J. <rj...@ek...>]

On Mon, 6 May 2002 14:26, David Goodger wrote:
> Richard Jones wrote:
> > I've got some problems in parsing a document. The following biblio
> >
> > element results in errors:
> > >>>>> snip
> > :
> > :Organization: Computer and Information Sciences::
> >
> >                  New Jersey Institute of Technology
> >                  University Heights
> >                  Newark, NJ, 07102
> >                  (address obsolescent).
> > <<<<< snip
>
> I'll take the warnings in reverse order.
>
> > Reporter: WARNING (2) Cannot extract compound bibliographic field
> > "Organization".
>
> This is by design of the document model.  The "organization" element
> currently may only contain text, not body elements.  It was intended
> to only contain the organization's name.  You present a good example
> of why the status quo perhaps ought to change.  Requires thought.

I figured that that's the problem - yes, it needs a solution :)


> However, a literal block isn't really the ideal way to represent an
> address block, is it?  I've been mulling over an idea for a "verse"
> directive which seems to apply here.  See
> http://docutils.sf.net/spec/notes.html#body-verse [#]_. What do you
> think?  How about that ';;' syntax?

Yes, just noticed those checkins. I like the idea. there's a lot of situations 
where a hard newline would be really nice. I'm pretty sure this proposal 
takes care of all the ones I have run into so far.


> .. [#] Just updated.  Especially note the examples; I love having
>    *Monty Python's Flying Circus: Just The Words* and *The Fairly
>    Incomplete & Rather Badly Illustrated Monty Python Songbook* on my
>    reference shelf, and I refer to them frequently (can you tell?).
>    Makes programming fun!  Thank Guido for Python!

Made me chuckle ;)


> > Reporter: WARNING (2) Literal block expected at line 2; none found.
>
> This is a tricky one.  Because field list field names can be quite
> long, the spec allows for arbitrary minimum indentation of the second
> and subsequent lines [#]_, without inferring any significance.  In
> other words, the address block is not seen as being indented at all,
> but just as a second paragraph.  So as far as the parser is concerned,
> there *is* no literal block to be found!

Sorry, I read the spec wrong - I corrected it soon after sending the email.


    Richard

Re: [Docutils-develop] Parsing oddness

[David G. <go...@us...>]

Richard Jones wrote:
>>> Reporter: WARNING (2) Literal block expected at line 2; none found.
>>
>> This is a tricky one.  Because field list field names can be quite
>> long, the spec allows for arbitrary minimum indentation of the second
>> and subsequent lines [#]_, without inferring any significance.  In
>> other words, the address block is not seen as being indented at all,
>> but just as a second paragraph.  So as far as the parser is concerned,
>> there *is* no literal block to be found!
> 
> Sorry, I read the spec wrong

Perhaps so, but it shows up a weakness in the spec that ought to be
corrected, one way or another.  This fails the "unsurprising" test.
I'll up the priority of that particular to-do item.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

Re: [Docutils-develop] Parsing oddness

[David G. <go...@us...>]

Richard Jones wrote:
> > I've hit another error in a definition list... note the line
> > number the error is reported at, and the line that is actually
> > erroneous (the last one).

I replied:
> I see it.  I'll see about fixing the error reporting.  It may take a
> bit of thought though, so don't hold your breath.

Fixed it: the truly erroneous line's line number is now reported.  The
fix was much easier than I thought it would be.  Turns out there were
several tests that also had wrong line numbers in their system
messages; fixed them too.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/

RE: [Docutils-develop] Parsing oddness

[Tony J I. (Tibs) <to...@ls...>]

David Goodger wrote:
> However, a literal block isn't really the ideal way to represent an
> address block, is it?  I've been mulling over an idea for a "verse"
> directive which seems to apply here.  See
> http://docutils.sf.net/spec/notes.html#body-verse. What do you
> think?  How about that ';;' syntax?

Hmm - a useful thing to be able to do.

I suspect that this *allows* me to handle (although not necessarily in
the way I'd like best) the majority of cases where I might want forced
line breaks - in the same way that "::" allows me to handle most places
where I want to escape a character.

As you say, the outstanding question is interpretation of inline markup
within a verse - i.e., in HTML terms, is it <pre> or not...

Thinking *about* verses, I'd obviously argue for allowing inline markup,
but am unfussed about lists (I don't have any examples *I can think of*
of poetry that uses them, for instance). But I trust you to work out the
details here, I think.

I'm not *too* keen on the use of ";;", but it does have a clear analogy
with "::", and it's unlikely to be used "by mistake". I assume the rules
of how it appears are identical to those for "::"? (i.e., precede with a
space to suppress a colon in the output?)

(personally, a "..verse" (or even "..sing"!) directive would be OK by me
as well...)

Tibs

..pedant:: In the second poem, should "stint" be "skint"?

--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.co.uk/
"How fleeting are all human passions compared with the massive
continuity of ducks." - Dorothy L. Sayers, "Gaudy Night"
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)

verse construct (was Re: [Docutils-develop] Parsing oddness)

[David G. <go...@us...>]

[moving to Doc-SIG for greater exposure]

David Goodger wrote:
>> However, a literal block isn't really the ideal way to represent an
>> address block, is it?  I've been mulling over an idea for a "verse"
>> directive which seems to apply here.  See
>> http://docutils.sf.net/spec/notes.html#body-verse. What do you
>> think?  How about that ';;' syntax?

Tony J Ibbs (Tibs) wrote:
> As you say, the outstanding question is interpretation of inline markup
> within a verse - i.e., in HTML terms, is it <pre> or not...

Literal blocks use <pre> exclusively and treat all whitespace and line
breaks as significant; no inline markup is recognized.  In "verse blocks"
(semi-literal blocks; anybody have a better name?) only line breaks and
indentation are treated differently from a regular paragraph; inline markup
like *emphasis* is recognized normally.

> Thinking *about* verses, I'd obviously argue for allowing inline markup,
> but am unfussed about lists

I don't think lists or anything else are necessary.  Verse blocks can be
thought of as variations of paragraphs, which don't have nested constructs
in the Docutils model.

> I'm not *too* keen on the use of ";;", but it does have a clear analogy
> with "::", and it's unlikely to be used "by mistake". I assume the rules
> of how it appears are identical to those for "::"? (i.e., precede with a
> space to suppress a colon in the output?)

Perhaps, although I don't think people will want to end a paragraph with a
semicolon.

-- 
David Goodger  <go...@us...>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/