From: Magnus <ma...@th...> - 2003-04-30 00:29:59
|
I'm writing a text, and I have python listings here and there. Today I'm doing this: .. include:: c1_4.py :literal: I feel that I'd like to have the python code pretty-printed though. My idea was that it would be convenient to type something like .. include:: c1_4.py :python: and to make something happen so that backend specific processing is done to make the code look good. (Syntax coloring and maybe a slightly different background color in in HTML, some mucking with bold, italic and possibly fonts in PDF to make it look good in b/w.) Has anyone done something like this? Is the approach right from a reST syntax point of view? How would one do something like this in the most convenient way? Pointers? Another way would obviously be to preprocess the python files, and to include c1_4_py.html etc instead, but it seems tricker to do that with readable reST files when there are several backends involved. I would prefer to have docutils fire off a filter when needed. -- Magnus Lycka (It's really Lyckå), ma...@th... Thinkware AB, Sweden, www.thinkware.se I code Python ~ The shortest path from thought to working program |
From: David G. <go...@py...> - 2003-04-30 04:25:53
|
Magnus Lyckå wrote: > My idea was that it would be convenient to type something like > > .. include:: c1_4.py > :python: > > and to make something happen so that backend specific processing > is done to make the code look good. (Syntax coloring and maybe a > slightly different background color in in HTML, some mucking with > bold, italic and possibly fonts in PDF to make it look good in b/w.) The following is in the to-do list (<http://docutils.sf.net/spec/notes.html#colorize-python>): _`colorize.python`: Colorize Python code. Fine for HTML output, but what about other formats? Revert to a literal block? Do we need some kind of "alternate" mechanism? Perhaps use a "pending" transform, which could switch its output based on the "format" in use. Use a factory function "transformFF()" which returns either "HTMLTransform()" instance or "GenericTransform" instance? There are any number of Python-to-HTML pretty printers. The problem for me was what to do for non-HTML output. Your message unstuck my brain a bit, and triggered an idea. If we take a Python-to-HTML pretty-printer and make it output a Docutils internal doctree (as per nodes.py) instead of HTML, then each output format's stylesheet (or equivalent) mechanism could take care of the rest. The pretty-printer code could turn this doctree fragment: <literal_block xml:space="preserve"> print 'This is Python code.' for i in range(10): print i </literal_block> into something like this ("</>" is end-tag shorthand): <literal_block xml:space="preserve" class="python"> <keyword>print</> <string>'This is Python code.'</> <keyword>for</> <identifier>i</> <keyword>in</> <expression>range(10)</>: <keyword>print</> <expression>i</> </literal_block> (Here, the <keyword> etc. tags are just examples off the top of my head. They're not necessarily a recommended implementation. "class" attributes on a <span> equivalent [perhaps "phrase"] may be the way to go.) > Has anyone done something like this? Not that I know of. Please give it a try! > Is the approach right from a reST syntax point of view? My first instinct is to go with something like this: .. python:: print 'This is Python code.' for i in range(10): print i For external file inclusions, follow the pattern of the "raw" directive:: .. python:: :file: cl_4.py > How would one do something like this in the most convenient way? > > Pointers? Read <http://docutils.sf.net/spec/howto/rst-directives.html>. Read the source for existing directives. There's a Python colorizer in the Python Cookbook (recipe 15.1), by Jurgen Hermann, from MoinMoin. There are others as well. -- David Goodger http://starship.python.net/~goodger Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv |
From: Tony J I. (Tibs) <to...@ls...> - 2003-04-30 08:09:28
|
David Goodger wrote: > If we take a Python-to-HTML pretty-printer and make it > output a Docutils internal doctree (as per nodes.py) > instead of HTML, then each output format's stylesheet > (or equivalent) mechanism could take care of the rest. And, indeed, by extension this would be a general approach that could be used for pretty-printing any language. > My first instinct is to go with something like this: > > .. python:: > > print 'This is Python code.' > for i in range(10): > print i > > For external file inclusions, follow the pattern of the "raw" > directive:: > > .. python:: :file: cl_4.py If it is possible to support (in theory) any language, not just Python, then I think that Magnus Lyckå's original idea of > .. include:: c1_4.py > :python: may be more useful? Similarly, for the "block" statement above, one would want to indicate that this is code-to-be-colourised, of the Python flavour (and the application would then try to locate an appropriate colouriser by some as yet undetermined means). I prefer the general approach to the specific because I can quite easily see writing documents that discuss multiple languages - not least Python, Pyrex and C, for instance. Tibs -- Tony Ibbs, Senior Software Engineer, Laser-Scan Ltd. [+44]1223 420414 - http://www.laser-scan.com |
From: Stefan M. <sm...@oe...> - 2003-04-30 11:04:39
|
-----BEGIN PGP SIGNED MESSAGE----- Hi all! 2 hours ago Tony J Ibbs (Tibs) wrote: > David Goodger wrote: >> My first instinct is to go with something like this: >>=20 >> .. python:: >>=20 >> print 'This is Python code.' >> for i in range(10): >> print i >>=20 >> For external file inclusions, follow the pattern of the "raw" >> directive:: >>=20 >> .. python:: :file: cl_4.py >=20 > If it is possible to support (in theory) any language, not just Python, > then I think that Magnus Lyck=E5's original idea of >=20 >> .. include:: c1_4.py >> :python: >=20 > may be more useful? Similarly, for the "block" statement above, one > would want to indicate that this is code-to-be-colourised, of the Pytho= n > flavour (and the application would then try to locate an appropriate > colouriser by some as yet undetermined means). >=20 > I prefer the general approach to the specific because I can quite easil= y > see writing documents that discuss multiple languages - not least > Python, Pyrex and C, for instance. I don't know whether this already exists in reST - my practice is still little ;-) - but in SDF I found the filter feature an helpful idea allowing a block to be processed - and that means: interpreted - by a filter. For instance as simple blocks something like this is possible in SDF:: !block table Header1 Header2 1 A 2 B !endblock This way the three lines between the markup were interpreted by filter `table'. Or:: !block example # Some code example here !endblock Because it is possible to create new filters this is a quite flexible approach. Magnus' problem would be solved by a special ``python`` filter then. Hope this helps Stefan -----BEGIN PGP SIGNATURE----- Version: 2.6.3in Charset: noconv Comment: Processed by Mailcrypt 3.5.7, an Emacs/PGP interface iQCVAwUBPq+tagnTZgC3zSk5AQGi0wP8DN1dhYy5wMaEAX2U2PkG7Zp7lmse2vSX PB25xiRe2NIoMjRa4TJcGg61fNGOcBr5kyyKv12kFIOu8r5hMqlX8aHFSJAGF1C0 tXrlGyiBaCrxoawAAhzYiFF4qGH1mrG166sOFQSUtmD08iVOoqpFIMFTyRTNhWaB xNp2DON4MsM=3D =3DyhPx -----END PGP SIGNATURE----- |
From: Aahz <aa...@py...> - 2003-04-30 12:55:00
|
On Wed, Apr 30, 2003, Stefan Merten wrote: > > I don't know whether this already exists in reST - my practice is > still little ;-) - but in SDF I found the filter feature an helpful > idea allowing a block to be processed - and that means: interpreted - > by a filter. > > For instance as simple blocks something like this is possible in SDF:: > > !block table > Header1 Header2 > 1 A > 2 B > !endblock > > This way the three lines between the markup were interpreted by filter > `table'. Or:: > > !block example > # Some code example here > !endblock In reST, these are handled by custom directives:: .. table:: or:: .. example:: For more discussion of exactly how to implement custom directives, we should probably go to the docutils-develop list. -- Aahz (aa...@py...) <*> http://www.pythoncraft.com/ "In many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it." --Tim Peters on Python, 16 Sep 93 |
From: Magnus <ma...@th...> - 2003-04-30 12:27:20
|
At 00:24 2003-04-30 -0400, David Goodger wrote: >My first instinct is to go with something like this: > > .. python:: > > print 'This is Python code.' > for i in range(10): > print i You are right that there should be a way to apply such things to embedded content as well as to included files. But rather than to include just pretty python as a standard, I'd much rather see a generic plug-in mechanism for formatting filters. maybe something like .. format:: python print 'This is Python code.' And now for something completely different .. format:: css :file: the_larch.css This is very similar to ".. raw::". Then we need some way to register formats. Maybe it could be as easy as: .../formats/ python/ html.py pdf.py css/ html.py xhtml.py ps.py In this case, the "formats" directory has a sub directory for each type of input, i.e. python and css. There are python filters for the html and pdf backends, an css filters for the html, xhtml and ps backends. I guess __init__.py could be used to construct more elaborate schemes, but either way, there could be an API looking something like this: Assuming we have a string containing python code and are creating an html file, we will do: from filters.python import html data = html.format(input) (Obviosuly we won't type 'python' explicitly in the code since that's a name that we read from a file as a string, but you know what I mean. By ending the formatting filters with... if __name__ == '__main__': import sys data = format(sys.stdin.read()) sys.stdout.write(data) ... they will be as usable from the command line as from python. If an external program is available, the filter might be as simple as: def format(input): return os.popen('echo %s | python2html' % input).read() -- Magnus Lycka (It's really Lyckå), ma...@th... Thinkware AB, Sweden, www.thinkware.se I code Python ~ The shortest path from thought to working program |
From: David G. <go...@py...> - 2003-04-30 14:23:12
|
Magnus Lyckå wrote: > But rather than to include just pretty python as a standard, > I'd much rather see a generic plug-in mechanism for formatting > filters. We've already got one: it's called the directive mechanism. I'd rather not add another mechanism on top, unless and until it proves truly necessary. > maybe something like > > .. format:: python > > print 'This is Python code.' > > And now for something completely different > > .. format:: css > :file: the_larch.css Whether you call it ".. format:: python" or ".. python::" doesn't matter much, they're equivalent. "python::" is less typing, but "format:: python" does add some meaning to the document. Either way, I don't mind. However, I don't want to begin with "format:: python" if there's only one format. Begin with ".. python::" and add a generic form when there are multiple choices. Directives can have synonyms for convenience. > Then we need some way to register formats. Maybe it could be > as easy as: > > .../formats/ > python/ > html.py > pdf.py > css/ > html.py > xhtml.py > ps.py > > In this case, the "formats" directory has a sub directory for each > type of input, i.e. python and css. There are python filters for > the html and pdf backends, an css filters for the html, xhtml and > ps backends. A slew of output-format-specific modules is not a good way to proceed. We want only one piece of code per input data format (Python, css, etc.), which converts to the internal doctree structure. Docutils itself already provides writers for each output format (HTML, XML, TeX, etc.), which convert *from* the internal doctree structure. Please re-read my last post in this thread, beginning with "unstuck my brain". You may need to familiarize yourself with the Docutils doctree structure, described in <http://docutils.sf.net/spec/pep-0258.html#document-tree> and <http://docutils.sf.net/spec/doctree.html>, and coded in docutils/nodes.py. -- David Goodger http://starship.python.net/~goodger Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv |
From: Magnus <ma...@th...> - 2003-04-30 22:41:55
|
I hope I'm not just reiterating some old discussion you've been through before here... At 10:22 2003-04-30 -0400, David Goodger wrote: >Magnus Lyck=E5 wrote: >>But rather than to include just pretty python as a standard, >>I'd much rather see a generic plug-in mechanism for formatting >>filters. > >We've already got one: it's called the directive mechanism. I'd rather=20 >not add another mechanism on top, unless and until it proves truly= necessary. Great, I still know all to little about this. (Waiting to be educated in Charleroi in June. :) Are there in-line versions of the directives as well? Maybe I want a python style expression in the middle of a paragraph? Is it possible to have site specific directives without having to patch docutils? I.e, do they work as plug-ins? My quick look at the code seems to suggest that you would hand-edit the dict _directive_registry in directives/__init__.py, but maybe I missed something. (Either way, I assume it's better to fix that then to add another layer of code.) >A slew of output-format-specific modules is not a good way to proceed. We= =20 >want only one piece of code per input data format (Python, css, etc.),=20 >which converts to the internal doctree structure. Docutils itself already= =20 >provides writers for each output format (HTML, XML, TeX, etc.), which=20 >convert *from* the internal doctree structure. Please re-read my last=20 >post in this thread, beginning with "unstuck my brain". You may need to=20 >familiarize yourself with the Docutils doctree structure, described in=20 ><http://docutils.sf.net/spec/pep-0258.html#document-tree> and=20 ><http://docutils.sf.net/spec/doctree.html>, and coded in docutils/nodes.py. But doesn't this imply that Docutils internally must know details about python source code presentation, and about all other formats we might want to display. I think it's good to keep such things outside the core of Docutils if this is to be able to scale. I can certainly imagine that people would like to pretty-print: Python, C, C++, Java, HTML, CSS, XML, DTD, reST (!), csv files (as tables), YAML, SVG etc. Ok, maybe not all of these will materialize, but in general, I think it's a good idea to delegate things like this to the directive code. As far as I understand, that means delegating writer work, or possibly some very generic data-driven internal system. It seems to me that the docutils writers, such as html4css1.py knows about the directives, such as danger, through functions like visit_danger. I can't see that that will scale, or that it will be possible to make site-customized versions of docutils with a reasonable effort. Except the filter code itself, I would expect to edit style shhets to add support for new formats, but not the docutils code itself. I'd expect to edits stylesheets for my personal preference anyway, so that's not a big deal. It should also be possible to make the style sheets modular in a flexible way. With my idea of a python directory, that might contain html4css1.py and html4css1.css, as well as pdf.py and a style sheet for pdf containing the relevant elements, all called python-[something]. Of course, this all depends on the scope you have in mind for docutils. For me, it looks so good that I'd like to use if far beyond PEPs and python code documentation. I could imagine writing books in reST. -- Magnus Lycka (It's really Lyckå), ma...@th... Thinkware AB, Sweden, www.thinkware.se I code Python ~ The shortest path from thought to working program=20 |
From: Aahz <aa...@py...> - 2003-04-30 23:30:15
|
On Thu, May 01, 2003, Magnus Lyck=E5 wrote: > > I hope I'm not just reiterating some old discussion you've > been through before here... Not quite, but close to it. ;-) > Are there in-line versions of the directives as well? Maybe > I want a python style expression in the middle of a paragraph? That's interpreted text and roles:: :chapter:`Foo` for example. > Is it possible to have site specific directives without having > to patch docutils? I.e, do they work as plug-ins? My quick look > at the code seems to suggest that you would hand-edit the dict > _directive_registry in directives/__init__.py, but maybe I > missed something. (Either way, I assume it's better to fix that > then to add another layer of code.) From OOdirectives.py (part of my hacked OpenOffice writer):: from docutils.parsers.rst import directives from docutils.parsers.rst.languages import en registry =3D directives._directive_registry registry['include-code'] =3D ('OOdirectives', 'include_code') registry['include-output'] =3D ('OOdirectives', 'include_output') en.directives['include-code'] =3D 'include-code' en.directives['include-output'] =3D 'include-output' There's been some recent discussion about adding convenience functions to handle this, but it works well enough currently. >David Goodger: >> >>A slew of output-format-specific modules is not a good way to >>proceed. We want only one piece of code per input data format >>(Python, css, etc.), which converts to the internal doctree >>structure. Docutils itself already provides writers for each >>output format (HTML, XML, TeX, etc.), which convert *from* the >>internal doctree structure. Please re-read my last post in this >>thread, beginning with "unstuck my brain". You may need to >>familiarize yourself with the Docutils doctree structure, described >>in <http://docutils.sf.net/spec/pep-0258.html#document-tree> >>and <http://docutils.sf.net/spec/doctree.html>, and coded in >>docutils/nodes.py. > > But doesn't this imply that Docutils internally must know details > about python source code presentation, and about all other formats we > might want to display. I think it's good to keep such things outside > the core of Docutils if this is to be able to scale. > > I can certainly imagine that people would like to pretty-print: > Python, C, C++, Java, HTML, CSS, XML, DTD, reST (!), csv files (as > tables), YAML, SVG etc. Not really. The idea is that the types of tokens one would want to print is far fewer than the variety of input formats. So as long as there are a sufficient variety of internal token types, these should be easy to add one at a time at the parser level. > It seems to me that the docutils writers, such as html4css1.py > knows about the directives, such as danger, through functions like > visit_danger. I can't see that that will scale, or that it will > be possible to make site-customized versions of docutils with a > reasonable effort. Except that ``danger`` is both a directive name and an internal node name. You could use mechanisms other than the ``danger`` directive to generate ``danger`` nodes; if you do that, the output will still be able to handle those nodes. > Of course, this all depends on the scope you have in mind for > docutils. For me, it looks so good that I'd like to use if far > beyond PEPs and python code documentation. I could imagine > writing books in reST. Some of us are already writing books in reST. ;-) --=20 Aahz (aa...@py...) <*> http://www.pythoncraft.= com/ "In many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it." --Tim Peters on Python, 16 Sep 93 |
From: David G. <go...@py...> - 2003-05-01 02:39:25
|
Magnus Lyckå wrote: > I hope I'm not just reiterating some old discussion you've > been through before here... ... > Great, I still know all to little about this. (Waiting to be > educated in Charleroi in June. :) Please feel free to educate yourself via the contents of the spec, spec/rst, and docs directories, and the source code itself. I doubt anyone would be reckless enough to talk about the Docutils *internals* at Charleroi. I imagine (please correct me if I'm wrong) that Michael Hudson's lightning talk will be about *using* reStructuredText from an author's perspective, possibly delving into the developer-using-Docutils-as-library perspective. > Are there in-line versions of the directives as well? Maybe > I want a python style expression in the middle of a paragraph? There's interpreted text with roles, which are essentially an extension mechanism for inline markup, and substitutions, which allow inline directives indirectly. > Is it possible to have site specific directives without having > to patch docutils? I.e, do they work as plug-ins? The docutils.parsers.rst.directives.register_directive function was recently added (to __init__.py). It bypasses the language lookup features. > (Either way, I assume it's better to fix that > then to add another layer of code.) Yes. If the register_directive function doesn't do the job, then fix *that*. >> Please re-read my last post in this thread, beginning with "unstuck my >> brain". ... > But doesn't this imply that Docutils internally must know details > about python source code presentation, and about all other formats > we might want to display. The directives that process these format certainly will need to know details, but the Docutils doctree itself won't. > I think it's good to keep such things > outside the core of Docutils if this is to be able to scale. Agreed. I'm leaning toward adding a single new general-purpose element, "phrase", equivalent to HTML's <span>. Which is why I originally wrote (Here, the <keyword> etc. tags are just examples off the top of my head. They're not necessarily a recommended implementation. "class" attributes on a <span> equivalent [perhaps "phrase"] may be the way to go.) In other words, <keyword> etc. were just placeholders. Here's the example rewritten using the generic "phrase": <literal_block xml:space="preserve" class="python"> <phrase class="keyword">print</> <phrase class="string">'This is Python code.'</> <phrase class="keyword">for</> <phrase class="identifier">i</> <phrase class="keyword">in</> <phrase class="expression">range(10)</>: <phrase class="keyword">print</> <phrase class="expression">i</> </literal_block> It's more verbose but more easily extensible and more appropriate for the case at hand. > Ok, maybe not all of these will materialize, but in general, I > think it's a good idea to delegate things like this to the directive > code. As far as I understand, that means delegating writer work, > or possibly some very generic data-driven internal system. I don't follow. > Except the filter code itself, I would expect to edit style > shhets to add support for new formats, but not the docutils > code itself. Exactly. > It should also be possible to make the style sheets modular in a > flexible way. Stylesheets are small; I don't see a reason for leaving out parts just for the sake of modularity. Modularity adds complexity; best to avoid it here if possible. In any case, CSS already allows modular stylesheets. > Of course, this all depends on the scope you have in mind for > docutils. For me, it looks so good that I'd like to use if far > beyond PEPs and python code documentation. I could imagine > writing books in reST. Aahz, David Abrahams, Patrick O'Brien, and others are doing more than imagining. There's a lot of work to be done before Docutils is world-class though, especially to support book-length documents; see <http://docutils.sf.net/spec/notes.html#to-do> for a taste. If you're willing and able to help, I'd be glad to have you join the team! Discussions this technical should really go to docutils-develop though. :-) -- David Goodger http://starship.python.net/~goodger Programmer/sysadmin for hire: http://starship.python.net/~goodger/cv |
From: Magnus <ma...@th...> - 2003-05-01 11:13:14
|
At 22:38 2003-04-30 -0400, David Goodger wrote: >Magnus Lyck=E5 wrote: >>I think it's good to keep such things >>outside the core of Docutils if this is to be able to scale. > >Agreed. I'm leaning toward adding a single new general-purpose element,=20 >"phrase", equivalent to HTML's <span>. ... > <literal_block xml:space=3D"preserve" class=3D"python"> > <phrase class=3D"keyword">print</> <phrase > class=3D"string">'This is Python code.'</> > <phrase class=3D"keyword">for</> <phrase > class=3D"identifier">i</> <phrase class=3D"keyword">in</> <phrase > class=3D"expression">range(10)</>: > <phrase class=3D"keyword">print</> <phrase > class=3D"expression">i</> > </literal_block> > >It's more verbose but more easily extensible and more appropriate for the= =20 >case at hand. > >>Ok, maybe not all of these will materialize, but in general, I >>think it's a good idea to delegate things like this to the directive >>code. As far as I understand, that means delegating writer work, >>or possibly some very generic data-driven internal system. > >I don't follow. If I understand you correctly, your thought is that all the drivers should be extended to know how to render "keyword", "string", "expression" etc. But that means that I need to patch the drivers if I make a new format which will introduce some new concept. My idea is that the "directive" (if that's what it should be) should be able to carry explicit instructions on how to render output on specific backends if the writer doesn't know this on its own. I guess one could imagine that the writer tries to render everything for its backend, and if it runs into something it doesn't know about, it will see if the directive code knows how to render it. As a hypothetical example, imagine that I want to include equations written in troff style. Like this: .. format:: eqn .EQ delim %% .EN %sum from i=3Do to inf c sup i~=3D~lim from {m -> inf} sum from i=3D0 to m sup i% .EQ delim off .EN If I have a troff backend which uses eqn, I'll just pass this on. If I render HTML, I'll probably use some external filter which converts this to a GIF, and if it's a LaTeX backend, I might have some elatorate external applications that convert eqn code to LaTeX. This means that the directive has to behave differently depending on what backend we have. We don't want vanilla docutils to become such a beast that it knows how to describe every kind of data mankind might want to describe... ;) Or am I just being stupid now? I'm just a newbie here, and I don't want to "tell you" how to do things, ok? I just try to describe how I'm thinking. -- Magnus Lycka (It's really Lyckå), ma...@th... Thinkware AB, Sweden, www.thinkware.se I code Python ~ The shortest path from thought to working program=20 |
From: David G. <go...@py...> - 2003-05-01 21:50:03
|
Discussion moved to docutils-develop; subscribe at <http://lists.sf.net/lists/listinfo/docutils-develop>. |
From: Michael H. <mw...@py...> - 2003-05-01 14:37:04
|
David Goodger <go...@py...> writes: > Magnus Lyckå wrote: >> I hope I'm not just reiterating some old discussion you've >> been through before here... > ... >> Great, I still know all to little about this. (Waiting to be >> educated in Charleroi in June. :) > > Please feel free to educate yourself via the contents of the spec, > spec/rst, and docs directories, and the source code itself. I doubt > anyone would be reckless enough to talk about the Docutils *internals* > at Charleroi. I imagine (please correct me if I'm wrong) that Michael > Hudson's lightning talk will be about *using* reStructuredText from an > author's perspective, possibly delving into the > developer-using-Docutils-as-library perspective. You're not at all wrong! Cheers, M. -- Now this is what I don't get. Nobody said absolutely anything bad about anything. Yet it is always possible to just pull random flames out of ones ass. -- http://www.advogato.org/person/vicious/diary.html?start=60 |