Re: [Docutils-develop] Emit ReST back from docutils?

[Eli B. <el...@gm...>]

On Tue, Jun 11, 2013 at 3:41 PM, David Goodger <go...@py...> wrote:

> On Tue, Jun 11, 2013 at 5:11 PM, eliben <el...@gm...> wrote:
> > David Goodger <goodger <at> python.org> writes:
> >
> >>
> >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de>
> > wrote:
> >> > Hi!
> >> >
> >> > 3 days ago Guenter Milde wrote:
> >> >> On 2013-05-29, eliben wrote:
> >> >>> Hello,
> >> >>
> >> >>> I want to read ReST into a representation I can manipulate
> > programmatically
> >> >>> (say with publish_doctree) and then output the file back in ReST
> > format.
> >> >>> Does docutils have a "ReST writer" or some similar interface to
> > achieve that?
> >> >>
> >> >> Unfortunately not (yet).
> >> >> This is a long standing TODO issue. Some work has been done in the
> >> >> "lossless ..." branch of the SVN repository.
> >> >
> >> > Well, indeed this question comes up once in a while. Unfortunately the
> >> > wrong answer is given all the time. Guys, I understand you don't like
> >> > the way this problem has been solved but IMHO answering plain "no" to
> >> > people looking for a solution is really exaggerating your dislike.
> >>
> >> Whoa, you're casting aspersions. To my knowledge there's no dislike
> >> here, just ignorance. In my case, I have no "dislike" for your
> >> approach, it simply wasn't on my radar. I've undoubtedly heard of it
> >> before, but haven't used it myself (I have never needed it), so I
> >> didn't think to mention it because I didn't *remember* it. The problem
> >> here is one of awareness. I, and probably Günter, and many others,
> >> were not actively aware of the existence of xml2rst.
> >
> > Out of curiosity - why wasn't this needed before? As the "source code"
> for
> > documentation, it makes sense to me to see tools that transform ReST
> > programmatically.
>
> I can only answer for myself. Why wasn't reST -> reST processing
> needed before? Just because I never needed it. The only reST -> reST
> transformations I ever needed were provided by Emacs and its
> reStructuredText mode (to which I made only small contributions). I
> treat reST the same as Python code, as source, which I edit manually.
> There are source-code manipulation systems out there (e.g. refactoring
> tools like Bicycle Repair Man and, I believe, features of Eclipse),
> but I have never needed or used them.
>
> Any time I have faced a situation where large-scale reST source
> reworking was needed, I wrote minimal special-purpose tools to do the
> job. But these tools didn't "understand" reST much at all; they just
> did fancy search-and-replace.
>
> Do you really need reST output, or would storing the internal document
> tree (doctree) for later processing be sufficient? You can do that
> with the docutils.core.publish_doctree & .publish_from_doctree
> functions (along with pickle or equivalent)
>

I really do need reST output. My case is probably typical of recent usage
of reST (which, it seems to me, is starting to "suffer" from its
popularity). The document in question is meant to be placed into a larger
output-generation system based on Sphinx. So I really want to just have a
reST file eventually, which Sphinx will transform to HTML for me.

The "minimal special-purpose tools" you mention above are exactly what I
had to use eventually, but it made me wonder whether serializability of
doctrees into reST would make such manipulations easier; certainly it would
reduce a lot of boilerplate. Hence my question about the information lost
during parsing.


> > Is there, to the best of your knowledge, any technical reason why the
> > internal "tree" that was parsed from input ReST can't be emitted back to
> > ReST? Is there any semantically-important information that gets lost in
> the
> > parsing process?
>
> For most ordinary constructs, no and no. But for many directives,
> information does get lost. For example, the "list-table" directive
> produces a table in the doctree with no indication that it came from a
> directive. Anything producing reST would have to choose which form of
> table to emit. The "include" directive is another example. By
> carefully examining the doctree you might be able to infer than
> another file was included at a certain point, but it's not explicit.
> And I don't guarantee that you'll always be able to do this. Such
> information was only included in the doctree for debugging and
> error-reporting purposes; I never considered reST -> reST processing.
>

This is a shame. Lost information means that serialization back to reST is
difficult, as you say.

If this goes against the design goals of reST & docutils, I guess I'll have
to stick with the custom hacking for now.

Thanks for the answers,

Eli