From: eliben <el...@gm...> - 2013-05-29 21:55:10
|
Hello, I want to read ReST into a representation I can manipulate programmatically (say with publish_doctree) and then output the file back in ReST format. Does docutils have a "ReST writer" or some similar interface to achieve that? Thanks in advance. |
From: Guenter M. <mi...@us...> - 2013-05-30 07:18:03
|
On 2013-05-29, eliben wrote: > Hello, > I want to read ReST into a representation I can manipulate programmatically > (say with publish_doctree) and then output the file back in ReST format. > Does docutils have a "ReST writer" or some similar interface to achieve that? Unfortunately not (yet). This is a long standing TODO issue. Some work has been done in the "lossless ..." branch of the SVN repository. Günter |
From: Stefan M. <sm...@oe...> - 2013-06-02 10:41:17
|
Hi! 3 days ago Guenter Milde wrote: > On 2013-05-29, eliben wrote: >> Hello, > >> I want to read ReST into a representation I can manipulate programmatically >> (say with publish_doctree) and then output the file back in ReST format. >> Does docutils have a "ReST writer" or some similar interface to achieve that? > > Unfortunately not (yet). > This is a long standing TODO issue. Some work has been done in the > "lossless ..." branch of the SVN repository. Well, indeed this question comes up once in a while. Unfortunately the wrong answer is given all the time. Guys, I understand you don't like the way this problem has been solved but IMHO answering plain "no" to people looking for a solution is really exaggerating your dislike. In fact there is something similar to a ReST writer. It converts Docutils XML to reStructuredText format. See `xml2rst` in the sandbox at http://svn.code.sf.net/p/docutils/code/trunk/sandbox/xml2rst/ Grüße Stefan |
From: David G. <go...@py...> - 2013-06-02 14:55:22
|
On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <sm...@oe...> wrote: > Hi! > > 3 days ago Guenter Milde wrote: >> On 2013-05-29, eliben wrote: >>> Hello, >> >>> I want to read ReST into a representation I can manipulate programmatically >>> (say with publish_doctree) and then output the file back in ReST format. >>> Does docutils have a "ReST writer" or some similar interface to achieve that? >> >> Unfortunately not (yet). >> This is a long standing TODO issue. Some work has been done in the >> "lossless ..." branch of the SVN repository. > > Well, indeed this question comes up once in a while. Unfortunately the > wrong answer is given all the time. Guys, I understand you don't like > the way this problem has been solved but IMHO answering plain "no" to > people looking for a solution is really exaggerating your dislike. Whoa, you're casting aspersions. To my knowledge there's no dislike here, just ignorance. In my case, I have no "dislike" for your approach, it simply wasn't on my radar. I've undoubtedly heard of it before, but haven't used it myself (I have never needed it), so I didn't think to mention it because I didn't *remember* it. The problem here is one of awareness. I, and probably Günter, and many others, were not actively aware of the existence of xml2rst. Don't take offence where none was meant. > In fact there is something similar to a ReST writer. It converts > Docutils XML to reStructuredText format. See `xml2rst` in the sandbox > at > > http://svn.code.sf.net/p/docutils/code/trunk/sandbox/xml2rst/ How about adding a FAQ entry? Also, add some detailed instructions on how to use it. What do I need to install? How do I install it? I have no idea how to use XSLT, and no desire to do the research. Maybe it's an easy thing to set up, but without instructions I perceive it as a painful and time-consuming thing to do. Since I don't need it, I'm reluctant to spend time on it, and I forget all about it. If you want mindshare, you need to promote awareness. Leave it as-is, and I guarantee you that when somebody else asks this question in six months, I will have forgotten all about xml2rst. -- David Goodger <http://python.net/~goodger> |
From: Stefan M. <sm...@oe...> - 2013-06-03 16:16:05
|
Hi! Yesterday David Goodger wrote: > On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <sm...@oe...> wrote: >> In fact there is something similar to a ReST writer. It converts >> Docutils XML to reStructuredText format. See `xml2rst` in the sandbox >> at >> >> http://svn.code.sf.net/p/docutils/code/trunk/sandbox/xml2rst/ > > How about adding a FAQ entry? Good idea. Where? > Also, add some detailed instructions on how to use it. What do I need > to install? How do I install it? I have no idea how to use XSLT, and > no desire to do the research. Maybe it's an easy thing to set up, but > without instructions I perceive it as a painful and time-consuming > thing to do. This is for what I wrote the README file contained in the repository. I thought it would suffice. I'd appreciate any improvements. > If you want mindshare, you need to promote awareness. Leave it as-is, > and I guarantee you that when somebody else asks this question in six > months, I will have forgotten all about xml2rst. May be I should campaign for moving it to the main tree ;-) . Grüße Stefan |
From: David G. <go...@py...> - 2013-06-04 15:53:26
|
On Mon, Jun 3, 2013 at 11:10 AM, Stefan Merten <sm...@oe...> wrote: > Hi! > > Yesterday David Goodger wrote: >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <sm...@oe...> wrote: >>> In fact there is something similar to a ReST writer. It converts >>> Docutils XML to reStructuredText format. See `xml2rst` in the sandbox >>> at >>> >>> http://svn.code.sf.net/p/docutils/code/trunk/sandbox/xml2rst/ >> >> How about adding a FAQ entry? > > Good idea. Where? There is already a question & an answer which needs correction: http://docutils.sourceforge.net/FAQ.html#are-there-any-tools-for-html-xml-to-restructuredtext-round-tripping Go ahead and edit the answer, but be realistic about limitations, dependencies, etc. >> Also, add some detailed instructions on how to use it. What do I need >> to install? How do I install it? I have no idea how to use XSLT, and >> no desire to do the research. Maybe it's an easy thing to set up, but >> without instructions I perceive it as a painful and time-consuming >> thing to do. > > This is for what I wrote the README file contained in the repository. > I thought it would suffice. I'd appreciate any improvements. Start by showing some examples of actual usage. Once you have users, they'll ask questions, and you can incorporate the answers into the documentation. Change the name of the README file to README.txt and have it auto-converted to HTML on the site. >> If you want mindshare, you need to promote awareness. Leave it as-is, >> and I guarantee you that when somebody else asks this question in six >> months, I will have forgotten all about xml2rst. > > May be I should campaign for moving it to the main tree ;-) . If there's demand for it, sure. -- David Goodger <http://python.net/~goodger> |
From: Stefan M. <sm...@oe...> - 2013-06-03 16:19:34
|
Hi David and anyone else with a short memory, Yesterday David Goodger wrote: > On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <sm...@oe...> wrote: >> Well, indeed this question comes up once in a while. Unfortunately the >> wrong answer is given all the time. Guys, I understand you don't like >> the way this problem has been solved but IMHO answering plain "no" to >> people looking for a solution is really exaggerating your dislike. > > Whoa, you're casting aspersions. To my knowledge there's no dislike > here, just ignorance. good to learn that. Sounded differently then. > In my case, I have no "dislike" for your > approach, it simply wasn't on my radar. I've undoubtedly heard of it > before, but haven't used it myself (I have never needed it), so I > didn't think to mention it because I didn't *remember* it. Ok. In this case I apologize. > The problem > here is one of awareness. I, and probably Günter, and many others, > were not actively aware of the existence of xml2rst. > Don't take offence where none was meant. Sometimes this is hard to distinguish ;-) . Re your suggestions I write a second mail. Grüße Stefan |
From: Guenter M. <mi...@us...> - 2013-06-06 08:48:21
|
On 2013-06-03, Stefan Merten wrote: > Yesterday David Goodger wrote: >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <sm...@oe...> wrote: Dear Stefan, >>> In fact there is something similar to a ReST writer. It converts >>> Docutils XML to reStructuredText format. See `xml2rst` in the sandbox >>> at >>> http://svn.code.sf.net/p/docutils/code/trunk/sandbox/xml2rst/ I am glad you bring xml2rst back to our attention. (While I think that it cannot replace a "Docutils rst writer", it is a good and, most importantly, already working alternative.) >> How about adding a FAQ entry? > Good idea. Where? In the SVN tree, it should be docutils/FAQ.txt I wonder, whether there are other "natural" places in the documentation where a user would expect a hint on how to get reStructuredText as target format. How about adding links in http://docutils.sourceforge.net/docs/user/tools.html ? --- tools.txt (Revision 7660) +++ tools.txt (Arbeitskopie) @@ -301,9 +301,12 @@ The ``rst2xml.py`` front end produces Docutils-native XML output. This can be transformed with standard XML tools such as XSLT -processors into arbitrary final forms. +processors into arbitrary final forms. An example is the xml2rst_ processor +in the Docutils sandbox. +.. _xml2rst: ../../../sandbox/xml2rst + ODF/OpenOffice-Generating Tools =============================== @@ -324,9 +327,16 @@ etc from `Odt Writer for Docutils <odt.html>`_. +reStructuredText-Generating Tools +================================= +Currently, there is no reStructuredText writer in Docutils and therefore +an ``rst2rst.py`` tool is still missing. +To generate reStructuredText documents with Docutils, you can use +the XML (Docutils native) writer and the xml2rst_ processor. + Testing/Debugging Tools ======================= BTW: the rst2man tool is also missing in this list. >> Also, add some detailed instructions on how to use it. What do I need >> to install? How do I install it? I have no idea how to use XSLT, and >> no desire to do the research. Maybe it's an easy thing to set up, but >> without instructions I perceive it as a painful and time-consuming >> thing to do. > This is for what I wrote the README file contained in the repository. > I thought it would suffice. I'd appreciate any improvements. I think, linking to the README at the appropriate places will suffice. http://docutils.sourceforge.net/sandbox/xml2rst/ >> If you want mindshare, you need to promote awareness. Leave it as-is, >> and I guarantee you that when somebody else asks this question in six >> months, I will have forgotten all about xml2rst. > May be I should campaign for moving it to the main tree ;-) . If you find a good place and meet the `Docutils Policies`__, why not? __ http://docutils.sourceforge.net/docs/dev/policies.html However, IMV xml2rst should not become a normal reStructuredText writer but stay as an alternative to a Docutils "rst" writer and rst2rst frontend tool. To complete the "XML toolset" and establish "XML (docutils native)" as *the* format to store Docutils documents, we would also require a Docutils "XML parser". There is interest, e.g. from the side of LyX_ for LyX <-> Docutils/reStructuredText conversion (including the idea to use LyX as rst editor). One possibility for such a conversion would be the use of XSLT stylesheets to convert between XML representation of the lyx document and docutils document. Thanks, Günter |
From: eliben <el...@gm...> - 2013-06-11 22:12:12
|
David Goodger <goodger <at> python.org> writes: > > On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de> wrote: > > Hi! > > > > 3 days ago Guenter Milde wrote: > >> On 2013-05-29, eliben wrote: > >>> Hello, > >> > >>> I want to read ReST into a representation I can manipulate programmatically > >>> (say with publish_doctree) and then output the file back in ReST format. > >>> Does docutils have a "ReST writer" or some similar interface to achieve that? > >> > >> Unfortunately not (yet). > >> This is a long standing TODO issue. Some work has been done in the > >> "lossless ..." branch of the SVN repository. > > > > Well, indeed this question comes up once in a while. Unfortunately the > > wrong answer is given all the time. Guys, I understand you don't like > > the way this problem has been solved but IMHO answering plain "no" to > > people looking for a solution is really exaggerating your dislike. > > Whoa, you're casting aspersions. To my knowledge there's no dislike > here, just ignorance. In my case, I have no "dislike" for your > approach, it simply wasn't on my radar. I've undoubtedly heard of it > before, but haven't used it myself (I have never needed it), so I > didn't think to mention it because I didn't *remember* it. The problem > here is one of awareness. I, and probably Günter, and many others, > were not actively aware of the existence of xml2rst. Out of curiosity - why wasn't this needed before? As the "source code" for documentation, it makes sense to me to see tools that transform ReST programmatically. Is there, to the best of your knowledge, any technical reason why the internal "tree" that was parsed from input ReST can't be emitted back to ReST? Is there any semantically-important information that gets lost in the parsing process? Reading about the xml2rst approach, it seems somewhat needlessly complex. I have ReST as input, I want ReST as output, and I have an internal in-memory representation created by the parser. Why I need to involve XML (and even worse - XSLT) to do the transformation is beyond my understanding. Eli |
From: Roberto A. <ra...@ne...> - 2013-06-11 22:21:34
|
On 11/06/13 19:11, eliben wrote: > David Goodger <goodger <at> python.org> writes: > >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de> > wrote: >>> Hi! >>> >>> 3 days ago Guenter Milde wrote: >>>> On 2013-05-29, eliben wrote: >>>>> Hello, >>>>> I want to read ReST into a representation I can manipulate > programmatically >>>>> (say with publish_doctree) and then output the file back in ReST > format. >>>>> Does docutils have a "ReST writer" or some similar interface to > achieve that? >>>> Unfortunately not (yet). >>>> This is a long standing TODO issue. Some work has been done in the >>>> "lossless ..." branch of the SVN repository. >>> Well, indeed this question comes up once in a while. Unfortunately the >>> wrong answer is given all the time. Guys, I understand you don't like >>> the way this problem has been solved but IMHO answering plain "no" to >>> people looking for a solution is really exaggerating your dislike. >> Whoa, you're casting aspersions. To my knowledge there's no dislike >> here, just ignorance. In my case, I have no "dislike" for your >> approach, it simply wasn't on my radar. I've undoubtedly heard of it >> before, but haven't used it myself (I have never needed it), so I >> didn't think to mention it because I didn't *remember* it. The problem >> here is one of awareness. I, and probably Günter, and many others, >> were not actively aware of the existence of xml2rst. > Out of curiosity - why wasn't this needed before? As the "source code" for > documentation, it makes sense to me to see tools that transform ReST > programmatically. > > Is there, to the best of your knowledge, any technical reason why the > internal "tree" that was parsed from input ReST can't be emitted back to > ReST? Is there any semantically-important information that gets lost in the > parsing process? Not really. I wrote a rst2rst that pretty much "just worked", even if it doesn't support the more modern bits of docutils. |
From: David G. <go...@py...> - 2013-06-11 22:42:05
|
On Tue, Jun 11, 2013 at 5:11 PM, eliben <el...@gm...> wrote: > David Goodger <goodger <at> python.org> writes: > >> >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de> > wrote: >> > Hi! >> > >> > 3 days ago Guenter Milde wrote: >> >> On 2013-05-29, eliben wrote: >> >>> Hello, >> >> >> >>> I want to read ReST into a representation I can manipulate > programmatically >> >>> (say with publish_doctree) and then output the file back in ReST > format. >> >>> Does docutils have a "ReST writer" or some similar interface to > achieve that? >> >> >> >> Unfortunately not (yet). >> >> This is a long standing TODO issue. Some work has been done in the >> >> "lossless ..." branch of the SVN repository. >> > >> > Well, indeed this question comes up once in a while. Unfortunately the >> > wrong answer is given all the time. Guys, I understand you don't like >> > the way this problem has been solved but IMHO answering plain "no" to >> > people looking for a solution is really exaggerating your dislike. >> >> Whoa, you're casting aspersions. To my knowledge there's no dislike >> here, just ignorance. In my case, I have no "dislike" for your >> approach, it simply wasn't on my radar. I've undoubtedly heard of it >> before, but haven't used it myself (I have never needed it), so I >> didn't think to mention it because I didn't *remember* it. The problem >> here is one of awareness. I, and probably Günter, and many others, >> were not actively aware of the existence of xml2rst. > > Out of curiosity - why wasn't this needed before? As the "source code" for > documentation, it makes sense to me to see tools that transform ReST > programmatically. I can only answer for myself. Why wasn't reST -> reST processing needed before? Just because I never needed it. The only reST -> reST transformations I ever needed were provided by Emacs and its reStructuredText mode (to which I made only small contributions). I treat reST the same as Python code, as source, which I edit manually. There are source-code manipulation systems out there (e.g. refactoring tools like Bicycle Repair Man and, I believe, features of Eclipse), but I have never needed or used them. Any time I have faced a situation where large-scale reST source reworking was needed, I wrote minimal special-purpose tools to do the job. But these tools didn't "understand" reST much at all; they just did fancy search-and-replace. Do you really need reST output, or would storing the internal document tree (doctree) for later processing be sufficient? You can do that with the docutils.core.publish_doctree & .publish_from_doctree functions (along with pickle or equivalent). > Is there, to the best of your knowledge, any technical reason why the > internal "tree" that was parsed from input ReST can't be emitted back to > ReST? Is there any semantically-important information that gets lost in the > parsing process? For most ordinary constructs, no and no. But for many directives, information does get lost. For example, the "list-table" directive produces a table in the doctree with no indication that it came from a directive. Anything producing reST would have to choose which form of table to emit. The "include" directive is another example. By carefully examining the doctree you might be able to infer than another file was included at a certain point, but it's not explicit. And I don't guarantee that you'll always be able to do this. Such information was only included in the doctree for debugging and error-reporting purposes; I never considered reST -> reST processing. -- David Goodger <http://python.net/~goodger> |
From: David G. <go...@py...> - 2013-06-11 22:44:29
|
On Tue, Jun 11, 2013 at 5:21 PM, Roberto Alsina <ra...@ne...> wrote: > On 11/06/13 19:11, eliben wrote: >> Out of curiosity - why wasn't this needed before? As the "source code" for >> documentation, it makes sense to me to see tools that transform ReST >> programmatically. >> >> Is there, to the best of your knowledge, any technical reason why the >> internal "tree" that was parsed from input ReST can't be emitted back to >> ReST? Is there any semantically-important information that gets lost in the >> parsing process? > > Not really. I wrote a rst2rst that pretty much "just worked", even if it > doesn't support the more modern bits of docutils. You should add a link to the FAQ. Another project I've undoubtedly heard of before but didn't recall. -- David Goodger <http://python.net/~goodger> |
From: Roberto A. <ra...@ne...> - 2013-06-11 22:58:06
|
On 11/06/13 19:43, David Goodger wrote: > On Tue, Jun 11, 2013 at 5:21 PM, Roberto Alsina > <ra...@ne...> wrote: >> On 11/06/13 19:11, eliben wrote: >>> Out of curiosity - why wasn't this needed before? As the "source code" for >>> documentation, it makes sense to me to see tools that transform ReST >>> programmatically. >>> >>> Is there, to the best of your knowledge, any technical reason why the >>> internal "tree" that was parsed from input ReST can't be emitted back to >>> ReST? Is there any semantically-important information that gets lost in the >>> parsing process? >> Not really. I wrote a rst2rst that pretty much "just worked", even if it >> doesn't support the more modern bits of docutils. > You should add a link to the FAQ. Another project I've undoubtedly > heard of before but didn't recall. It never amounted to a real project, and I can't edit the FAQ myself :-) If anyone wants to add a link: https://code.google.com/p/tobogan/source/browse/trunk/rst2rst.py |
From: Eli B. <el...@gm...> - 2013-06-12 01:13:16
|
On Tue, Jun 11, 2013 at 3:41 PM, David Goodger <go...@py...> wrote: > On Tue, Jun 11, 2013 at 5:11 PM, eliben <el...@gm...> wrote: > > David Goodger <goodger <at> python.org> writes: > > > >> > >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de> > > wrote: > >> > Hi! > >> > > >> > 3 days ago Guenter Milde wrote: > >> >> On 2013-05-29, eliben wrote: > >> >>> Hello, > >> >> > >> >>> I want to read ReST into a representation I can manipulate > > programmatically > >> >>> (say with publish_doctree) and then output the file back in ReST > > format. > >> >>> Does docutils have a "ReST writer" or some similar interface to > > achieve that? > >> >> > >> >> Unfortunately not (yet). > >> >> This is a long standing TODO issue. Some work has been done in the > >> >> "lossless ..." branch of the SVN repository. > >> > > >> > Well, indeed this question comes up once in a while. Unfortunately the > >> > wrong answer is given all the time. Guys, I understand you don't like > >> > the way this problem has been solved but IMHO answering plain "no" to > >> > people looking for a solution is really exaggerating your dislike. > >> > >> Whoa, you're casting aspersions. To my knowledge there's no dislike > >> here, just ignorance. In my case, I have no "dislike" for your > >> approach, it simply wasn't on my radar. I've undoubtedly heard of it > >> before, but haven't used it myself (I have never needed it), so I > >> didn't think to mention it because I didn't *remember* it. The problem > >> here is one of awareness. I, and probably Günter, and many others, > >> were not actively aware of the existence of xml2rst. > > > > Out of curiosity - why wasn't this needed before? As the "source code" > for > > documentation, it makes sense to me to see tools that transform ReST > > programmatically. > > I can only answer for myself. Why wasn't reST -> reST processing > needed before? Just because I never needed it. The only reST -> reST > transformations I ever needed were provided by Emacs and its > reStructuredText mode (to which I made only small contributions). I > treat reST the same as Python code, as source, which I edit manually. > There are source-code manipulation systems out there (e.g. refactoring > tools like Bicycle Repair Man and, I believe, features of Eclipse), > but I have never needed or used them. > > Any time I have faced a situation where large-scale reST source > reworking was needed, I wrote minimal special-purpose tools to do the > job. But these tools didn't "understand" reST much at all; they just > did fancy search-and-replace. > > Do you really need reST output, or would storing the internal document > tree (doctree) for later processing be sufficient? You can do that > with the docutils.core.publish_doctree & .publish_from_doctree > functions (along with pickle or equivalent) > I really do need reST output. My case is probably typical of recent usage of reST (which, it seems to me, is starting to "suffer" from its popularity). The document in question is meant to be placed into a larger output-generation system based on Sphinx. So I really want to just have a reST file eventually, which Sphinx will transform to HTML for me. The "minimal special-purpose tools" you mention above are exactly what I had to use eventually, but it made me wonder whether serializability of doctrees into reST would make such manipulations easier; certainly it would reduce a lot of boilerplate. Hence my question about the information lost during parsing. > > Is there, to the best of your knowledge, any technical reason why the > > internal "tree" that was parsed from input ReST can't be emitted back to > > ReST? Is there any semantically-important information that gets lost in > the > > parsing process? > > For most ordinary constructs, no and no. But for many directives, > information does get lost. For example, the "list-table" directive > produces a table in the doctree with no indication that it came from a > directive. Anything producing reST would have to choose which form of > table to emit. The "include" directive is another example. By > carefully examining the doctree you might be able to infer than > another file was included at a certain point, but it's not explicit. > And I don't guarantee that you'll always be able to do this. Such > information was only included in the doctree for debugging and > error-reporting purposes; I never considered reST -> reST processing. > This is a shame. Lost information means that serialization back to reST is difficult, as you say. If this goes against the design goals of reST & docutils, I guess I'll have to stick with the custom hacking for now. Thanks for the answers, Eli |
From: David G. <go...@py...> - 2013-06-12 01:38:45
|
On Tue, Jun 11, 2013 at 8:12 PM, Eli Bendersky <el...@gm...> wrote: > > > > On Tue, Jun 11, 2013 at 3:41 PM, David Goodger <go...@py...> wrote: > >> On Tue, Jun 11, 2013 at 5:11 PM, eliben <el...@gm...> wrote: >> > David Goodger <goodger <at> python.org> writes: >> > >> >> >> >> On Sun, Jun 2, 2013 at 5:40 AM, Stefan Merten <smerten <at> oekonux.de >> > >> > wrote: >> >> > Hi! >> >> > >> >> > 3 days ago Guenter Milde wrote: >> >> >> On 2013-05-29, eliben wrote: >> >> >>> Hello, >> >> >> >> >> >>> I want to read ReST into a representation I can manipulate >> > programmatically >> >> >>> (say with publish_doctree) and then output the file back in ReST >> > format. >> >> >>> Does docutils have a "ReST writer" or some similar interface to >> > achieve that? >> >> >> >> >> >> Unfortunately not (yet). >> >> >> This is a long standing TODO issue. Some work has been done in the >> >> >> "lossless ..." branch of the SVN repository. >> >> > >> >> > Well, indeed this question comes up once in a while. Unfortunately >> the >> >> > wrong answer is given all the time. Guys, I understand you don't like >> >> > the way this problem has been solved but IMHO answering plain "no" to >> >> > people looking for a solution is really exaggerating your dislike. >> >> >> >> Whoa, you're casting aspersions. To my knowledge there's no dislike >> >> here, just ignorance. In my case, I have no "dislike" for your >> >> approach, it simply wasn't on my radar. I've undoubtedly heard of it >> >> before, but haven't used it myself (I have never needed it), so I >> >> didn't think to mention it because I didn't *remember* it. The problem >> >> here is one of awareness. I, and probably Günter, and many others, >> >> were not actively aware of the existence of xml2rst. >> > >> > Out of curiosity - why wasn't this needed before? As the "source code" >> for >> > documentation, it makes sense to me to see tools that transform ReST >> > programmatically. >> >> I can only answer for myself. Why wasn't reST -> reST processing >> needed before? Just because I never needed it. The only reST -> reST >> transformations I ever needed were provided by Emacs and its >> reStructuredText mode (to which I made only small contributions). I >> treat reST the same as Python code, as source, which I edit manually. >> There are source-code manipulation systems out there (e.g. refactoring >> tools like Bicycle Repair Man and, I believe, features of Eclipse), >> but I have never needed or used them. >> >> Any time I have faced a situation where large-scale reST source >> reworking was needed, I wrote minimal special-purpose tools to do the >> job. But these tools didn't "understand" reST much at all; they just >> did fancy search-and-replace. >> >> Do you really need reST output, or would storing the internal document >> tree (doctree) for later processing be sufficient? You can do that >> with the docutils.core.publish_doctree & .publish_from_doctree >> functions (along with pickle or equivalent) >> > > I really do need reST output. My case is probably typical of recent usage > of reST (which, it seems to me, is starting to "suffer" from its > popularity). The document in question is meant to be placed into a larger > output-generation system based on Sphinx. So I really want to just have a > reST file eventually, which Sphinx will transform to HTML for me. > > The "minimal special-purpose tools" you mention above are exactly what I > had to use eventually, but it made me wonder whether serializability of > doctrees into reST would make such manipulations easier; certainly it would > reduce a lot of boilerplate. Hence my question about the information lost > during parsing. > > >> > Is there, to the best of your knowledge, any technical reason why the >> > internal "tree" that was parsed from input ReST can't be emitted back to >> > ReST? Is there any semantically-important information that gets lost in >> the >> > parsing process? >> >> For most ordinary constructs, no and no. But for many directives, >> information does get lost. For example, the "list-table" directive >> produces a table in the doctree with no indication that it came from a >> directive. Anything producing reST would have to choose which form of >> table to emit. The "include" directive is another example. By >> carefully examining the doctree you might be able to infer than >> another file was included at a certain point, but it's not explicit. >> And I don't guarantee that you'll always be able to do this. Such >> information was only included in the doctree for debugging and >> error-reporting purposes; I never considered reST -> reST processing. >> > > This is a shame. Lost information means that serialization back to reST is > difficult, as you say. > Directives are kind of like macros in C, they're high-level constructs that resolve to low-level code. Nobody can expect to be able to de-compile the original macros from the object code. It's similar in reST. If you want to be able to go back to the source, you have to add a lot of details into the doctree first. > If this goes against the design goals of reST & docutils, I guess I'll > have to stick with the custom hacking for now. > It doesn't really "go against" any design goals, this was just never a goal. It's over a decade later, goals can change. However, I have no need for this functionality, so it will have to come from someone else. There may be ways to get what you want without a full-blown reST-to-reST processing system. For example, parse the input to a doctree, identify the position in the doctree where you want to make a change, then use the position info (source file & line number) to edit the source. It's not trivial, and some changes to Docutils may be necessary (e.g. more precise line numbers, indentation levels), but I think it can be done -- if someone is willing to do the work. -- David Goodger <http://python.net/~goodger> |
From: Guenter M. <mi...@us...> - 2013-06-13 09:33:20
|
On 2013-06-11, eliben wrote: > Out of curiosity - why wasn't this needed before? This has been requested several times. To me, it also appears as a "gap" in the toolchain provided by Docutils. However, the need has never been so urgent that someone finished a standard "rst-writer". There is, the ambitious unfinished project of a "lossless" rst2rst conversion in a branch of the Docutils SVN repository. > Is there, to the best of your knowledge, any technical reason why the > internal "tree" that was parsed from input ReST can't be emitted back to > ReST? Is there any semantically-important information that gets lost in the > parsing process? "Lossless" re-writing of the literal input is hard to achieve and would require storing lots of additional information. Writing back valid rst source code that produces an identical doctree is much simpler (use just one table model, one set of section heading underlines, one set of list markers, inline input via "input" directive, ...). Starting from this, one could add options (like keep input directives ...) step by step. But also for the alternatives of "parse to doctree" -> "publish from doctree" and "write to XML" -> "publish from XML", it might be sensible to have the possibility to customize * the resolution of the "input" directive * the application of the several "transforms" before or after storage. > Reading about the xml2rst approach, it seems somewhat needlessly complex. I > have ReST as input, I want ReST as output, and I have an internal in-memory > representation created by the parser. Why I need to involve XML (and even > worse - XSLT) to do the transformation is beyond my understanding. I understand the reasons quite well: Going from the internal representation to an XML representation comes "for free", as Docutils already includes an XML writer. For anyone more proficient in XML/XSLT than Python, it may seem "natural" to go this route. Also, for me XML seem best suited for storage of parsed rst (i.e. a doctree representation). This is why I'd welcome support for "publish form native XML" in Docutils. Günter |
From: Stefan M. <sm...@oe...> - 2013-06-13 17:16:16
|
Hi eliben! Yesterday eliben wrote: > Reading about the xml2rst approach, it seems somewhat needlessly complex. I > have ReST as input, I want ReST as output, and I have an internal in-memory > representation created by the parser. Why I need to involve XML (and even > worse - XSLT) to do the transformation is beyond my understanding. `xml2rst` has been created to transform Docutils XML into reST syntax. This allows for arbitrary XML to get transformed to Docutils XML and then be output as reST. Indeed this is the approach to convert another ASCII based syntax to reST. I use it as a migration tool. That the Docutils XML document comes from reST in the first place is a special case which is supported but this was not the design goal. Grüße Stefan |
From: Stefan M. <sm...@oe...> - 2013-11-11 12:33:55
|
Hi Günter et al! 5 months (158 days) ago Guenter Milde wrote: > On 2013-06-03, Stefan Merten wrote: >> May be I should campaign for moving it to the main tree ;-) . > > If you find a good place and meet the `Docutils Policies`__, why not? > > __ http://docutils.sourceforge.net/docs/dev/policies.html > > However, IMV xml2rst should not become a normal reStructuredText writer > but stay as an alternative to a Docutils "rst" writer and rst2rst > frontend tool. See other thread. I'll think about rewrapping code in `xml2rst` to make it a reStructuredText writer. > To complete the "XML toolset" and establish "XML (docutils native)" as *the* > format to store Docutils documents, we would also require a Docutils "XML > parser". True. Indeed I thought this would exist already. But after looking at the `doctree` reader again I just learned that there is no such thing like a Docutils XML reader. Oops! But frankly a Docutils XML reader should not be that difficult. > There is interest, e.g. from the side of LyX_ for LyX <-> > Docutils/reStructuredText conversion (including the idea to use LyX as > rst editor). One possibility for such a conversion would be the use of XSLT > stylesheets to convert between XML representation of the lyx document and > docutils document. Sounds really interesting. May be we need to think about extending the XML representation then to convey more information. But that certainly would be a next step. Grüße Stefan |
From: Stefan M. <sm...@oe...> - 2013-11-11 13:08:50
|
Hi all! 4 hours ago Stefan Merten wrote: > 5 months (158 days) ago Guenter Milde wrote: >> To complete the "XML toolset" and establish "XML (docutils native)" as *the* >> format to store Docutils documents, we would also require a Docutils "XML >> parser". > > True. Indeed I thought this would exist already. But after looking at > the `doctree` reader again I just learned that there is no such thing > like a Docutils XML reader. Oops! > > But frankly a Docutils XML reader should not be that difficult. Well, I just refactored `odf2docutlis` and created some base classes for general use. `odfdocutilslib/parsers/xml.py` contains `XmlParser` and `XmlVisitor` for generic support for processing XML input. `XmlParser` is a generic XML parser base class. A subclass must define only some class attributes to make it work. See `odfdocutilslib/python.py` for an example. `XmlVisitor` is a base class for an XML visitor. A subclass only needs to implement the methods for visiting the XML elements needed. This is in the spirit of `nodes.NodeVisitor` and should look familiar. Again see `odfdocutilslib/python.py` for an example of using it. `odfdocutilslib/parsers/xslt.py` contains `XsltParser` which is a generic base class for XSLT based parsers. A subclass only needs to define the XSLT to use for processing. See `odfdocutilslib/xslt.py` for an example. `odfdocutilslib/writers/xslt.py` contains `XsltWriter` which is a support class to work with `XsltParser`. In `odfdocutils.py` there is also `ZipFileInput` for generic support of reading input from a ZIP file. Though this is not directly related to XML many XML bundles - such as an ODF file - come as a ZIP file. I think this is a nice foundation for getting forward. For instance writing a Docutils XML parser now means only to implement an `XmlVisitor` containing methods rebuilding the doctree from the XML elements. This should be fairly easy. For the XSLT stuff I'm going to use this to implement a reStructuredText writer based on the XSLT based parser in `xml2rst`. I'd appreciate if you look at the code and make suggestions for improving it. I also suggest to move the base classes to the main sources after review. Grüße Stefan |
From: Stefan M. <sm...@oe...> - 2013-12-08 11:59:12
|
Hi all! 3 weeks (27 days) ago Stefan Merten wrote: > Well, I just refactored `odf2docutlis` and created some base classes > for general use. Meanwhile I moved all the relevant stuff to an own package in the sandbox: http://sourceforge.net/p/docutils/code/HEAD/tree/trunk/sandbox/docutils_xml/ > I'd appreciate if you look at the code and make suggestions for > improving it. I also suggest to move the base classes to the main > sources after review. I called the package `docutils_xml` and "my" applications `odf2docutils` and `xml2rst` now use this package. I added lots of tests and both applications work as expected so I think the code is fine - and now reflects the Docutils style :-) . To me this is now ready to move to the main tree. I reused the structure from the main tree so integration should be fairly easy. Any problem with such a move anyone? Grüße Stefan |