From: David G. <go...@us...> - 2002-10-29 03:40:06
|
These questions are more suitable for the docutils-develop list; cross-posting. Ian Bicking wrote: > To serve as an instructional sample I'm making a very simply Wiki -- > I'd like to use reST as the markup, since it's not an example in > parsing (and I like the reST markup more than most Wiki markups to > boot). Great! Please share the results if you can. I know that some intregration with MoinMoin has been done, although incomplete I believe. See http://twistedmatrix.com/users/jh.twistd/moin/moin.cgi/RestSample > So, to do this I need to convert reST to HTML. I looked around in > buildhtml.py to get an idea, but I found it surprisingly difficult > to figure out just what I would need to do. buildhtml.py is a complicated beast, trying to be a kind of "make", recursively building HTML for regular and PEP files with integrated local config file support. It's not a good example to begin with. > Could someone perhaps tell me what the easiest way to do this is? First, make sure you're using the latest code from CVS or from the snapshot: <http://docutils.sf.net/docutils-snapshot.tgz>. Look in the docutils/core.py file at the "publish_string" convenience function. It is probably what you want. Beyond the docstrings, there is no related developer documentation yet. Contributions are, of course, most welcome! Perhaps the tools/pep2html.py module is a better example, function "fix_rst_pep". Docutils is called in one (logical) line of code:: output = core.publish_string( source=''.join(input_lines), source_path=inpath, destination_path=outfile.name, reader_name='pep', parser_name='restructuredtext', writer_name='pep_html', settings=docutils_settings) > I'm not sure of the relevance of the options, the importance of > FileIO, or some of the other abstractions involved in this > operation. I've since renamed "options" to "settings": they're runtime configuration settings (see http://docutils.sf.net/docs/tools.html#configuration-file-entries). For now, just use a convenience function with no explicit settings; the defaults should be fine. FileIO is an implementation detail; use strings with "publish_string" or file-like objects with "publish_file". Be sure to read the docstrings. > Also -- is there a good hook in reST that I can use for Wiki links? > The linking mechanism reST has is fine for internal and external > links, but doesn't fit right for Wiki links (which fall somewhere in > between). If I could capture failed internal links and turn them > into inter-wiki-page links, that would be perfect. Failing that, if > there was some way I could introduce another markup easily that > would be great. There's mention of Wikis as an example potential Reader component in PEP 258 (http://docutils.sf.net/spec/pep-0258.html#readers). A Wiki Reader could subclass the standard parser, or we could build in hooks if they prove general enough. The PEP Reader (docutils/readers/pep.py) first subclassed the parser (still does, minimally: the "Inliner" class), but most of the code was moved into the parser proper for other client code to use. I would suggest that Wiki links retain the trailing "_" of regular reStructuredText links, and a mechanism be added to do some kind of lookup from a Wiki target database. The extra syntax (trailing "_") removes ambiguity inherent in CamelCase WikiLinks (such as anything written by Angus MacDuff). > If there isn't an easy way, I'll probably just look for WikiNames in > the generated HTML, and do the substitution then. I'll have to do > some dynamic fixup anyway, to distinguish between existent Wiki > pages and to-be-created links. A proper integrated approach would be best IMO, and welcome. Please subscribe to docutils-develop (http://lists.sf.net/lists/listinfo/docutils-develop) and discuss it there. I'll be happy to help. -- David Goodger <go...@us...> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ |
From: Ian B. <ia...@co...> - 2002-10-29 07:52:50
|
On Mon, 2002-10-28 at 21:40, David Goodger wrote: > > To serve as an instructional sample I'm making a very simply Wiki -- > > I'd like to use reST as the markup, since it's not an example in > > parsing (and I like the reST markup more than most Wiki markups to > > boot). > > Great! Please share the results if you can. I know that some > intregration with MoinMoin has been done, although incomplete I > believe. See > http://twistedmatrix.com/users/jh.twistd/moin/moin.cgi/RestSample I will, though I'm not really making a full Wiki -- I just thought it was a good example web application that's neat, yet potentially very simple (especially when you don't make your own parser). > Perhaps the tools/pep2html.py module is a better example, function > "fix_rst_pep". Docutils is called in one (logical) line of code:: > > output = core.publish_string( > source=''.join(input_lines), > source_path=inpath, > destination_path=outfile.name, > reader_name='pep', > parser_name='restructuredtext', > writer_name='pep_html', > settings=docutils_settings) Thanks, that's just what I wanted. > > Also -- is there a good hook in reST that I can use for Wiki links? > > The linking mechanism reST has is fine for internal and external > > links, but doesn't fit right for Wiki links (which fall somewhere in > > between). If I could capture failed internal links and turn them > > into inter-wiki-page links, that would be perfect. Failing that, if > > there was some way I could introduce another markup easily that > > would be great. > > There's mention of Wikis as an example potential Reader component in > PEP 258 (http://docutils.sf.net/spec/pep-0258.html#readers). A Wiki > Reader could subclass the standard parser, or we could build in hooks > if they prove general enough. The PEP Reader > (docutils/readers/pep.py) first subclassed the parser (still does, > minimally: the "Inliner" class), but most of the code was moved into > the parser proper for other client code to use. > > I would suggest that Wiki links retain the trailing "_" of regular > reStructuredText links, and a mechanism be added to do some kind of > lookup from a Wiki target database. The extra syntax (trailing "_") > removes ambiguity inherent in CamelCase WikiLinks (such as anything > written by Angus MacDuff). That's definitely what I'd like to do -- I have never like WikiNames anyway -- false positives and mangled case limit the medium, IMHO. There isn't any database of Wiki targets -- it's important that you be able to make a link to a page that does not yet exist. I think it would be fine if Wiki links were made if no internal link could be found (though with all the automatically-generated targets, there could be some clashes). Otherwise I'd need some other punctuation -- like `wiki link`^ or somesuch. There's a fair amount of seldom-used punctuation that could be used. In the case of Wiki links being a failback when no internal link was found, it seems like the easiest way to handle it would be some sort of error handler... maybe you already have something like this. As to adding new punctuation-based markup, I have no idea what interface would make sense, except to somehow subclass the parser. If the parser made it easy to add certain styles of markup (either trailing punctuation, or enclosing punctuation), then subclassing doesn't seem like a big burden. As I think about it, I'm leaning toward new punctuation. Where should I look for this? Ian |
From: Garth K. <ga...@de...> - 2002-10-29 17:06:42
|
> As I think about it, I'm leaning toward new punctuation. > Where should I look for this? Please, don't do the punctuation. Stick with the trailing ``_``. Trust me on this one. :) Regards, Garth. |
From: Ian B. <ia...@co...> - 2002-10-29 19:47:22
|
On Tue, 2002-10-29 at 11:06, Garth Kidd wrote: > > As I think about it, I'm leaning toward new punctuation. > > Where should I look for this? > > Please, don't do the punctuation. Stick with the trailing ``_``. Trust > me on this one. :) Why, what's your experience? Too hard to do, or too messy when you're through? My only concern is that I won't be able to distinguish between internal and external links if _ gets more overloaded. Ian |
From: Garth K. <ga...@de...> - 2002-10-30 00:24:46
|
>>> As I think about it, I'm leaning toward new punctuation. >>> Where should I look for this? >> >> Please, don't do the punctuation. Stick with the trailing >> ``_``. Trust me on this one. :) > > Why, what's your experience? Too hard to do, or too messy > when you're through? Too hard to deal with the people who find the existing reST syntax hard enough to deal with before you throw even more punctuation into the mix? :) > My only concern is that I won't be able to distinguish between > internal and external links if _ gets more overloaded. Not at all. It's up to the consumer of the document to resolve links. In reST terminology, that's either a parser or a formatter or something else -- I can never guess -- but it's not the *parser* that does it. If you can make a link resolve within a document, fantastic! If not, resolve it to other documents in the Wiki. That precedence also lets people break titled sections in a document (automatic target) into a separate page on the Wiki, in which case all of the links in the document will suddenly point to the new page when the title is removed from the source document. The same is true of links in the text. Let's say a site you used to link to has disappeared. Pull a copy out of the Internet Archive, save as a Wiki page, and remove the ``.. _target:`` entry. Done. If you used different punctuation, someone would have to wade through the document changing all of the punctuation to do these things, which is overly painful. The one thing I *can't* do is help you figure out how to do this in code, because I haven't spent much time with reST in ages (okay, ever), but I reckon changing target resolution is going to be a whole lot easier than adding YetMorePunctuation_. Regards, Garth. |
From: Ian B. <ia...@co...> - 2002-10-30 00:29:06
|
On Tue, 2002-10-29 at 18:24, Garth Kidd wrote: > If you can make a link resolve within a document, fantastic! If not, > resolve it to other documents in the Wiki. That precedence also lets > people break titled sections in a document (automatic target) into a > separate page on the Wiki, in which case all of the links in the > document will suddenly point to the new page when the title is removed > from the source document. > > The same is true of links in the text. Let's say a site you used to link > to has disappeared. Pull a copy out of the Internet Archive, save as a > Wiki page, and remove the ``.. _target:`` entry. Done. Those are good arguments, you've convinced me. Now I just need to know how to do it :) Ian |
From: David G. <go...@us...> - 2002-10-30 01:59:26
|
I agree with Garth that new markup is not the way to go here. But I also agree with Ian that "trust me" just ain't good enough ;-), so I'm glad some good convincing arguments came up. [Ian] >> My only concern is that I won't be able to distinguish between >> internal and external links if _ gets more overloaded. [Garth] > Not at all. It's up to the consumer of the document to resolve links. In > reST terminology, that's either a parser or a formatter or something > else -- I can never guess -- but it's not the *parser* that does it. Transforms, actually. Hyperlinks are resolved in the transforms in docutils.transforms.references. The docutils.transforms.universal.FinalChecks transform checks for unresolved references and signals errors. A transform replacing or preceding this one would be the place to do the Wiki-linking. > The one thing I *can't* do is help you figure out how to do this in > code, because I haven't spent much time with reST in ages (okay, ever), Garth *did* do a very cool overhaul of the unit test framework, for which he has our eternal gratitude. -- David Goodger <go...@us...> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ |
From: Ian B. <ia...@co...> - 2002-10-30 07:46:36
|
On Tue, 2002-10-29 at 19:59, David Goodger wrote: > Transforms, actually. Hyperlinks are resolved in the transforms in > docutils.transforms.references. The > docutils.transforms.universal.FinalChecks transform checks for unresolved > references and signals errors. A transform replacing or preceding this one > would be the place to do the Wiki-linking. Wow, it actually works. That was easier than I thought it would be. Here's what I did: from docutils import nodes from docutils.readers import standalone from docutils.transforms import Transform class WikiLinkResolver(nodes.SparseNodeVisitor): def visit_reference(self, node): if node.resolved or not node.hasattr('refname'): return refname = node['refname'] node.resolved = 1 node['refuri'] = refname del node['refname'] class WikiLink(Transform): default_priority = 400 def apply(self): visitor = WikiLinkResolver(self.document) self.document.walk(visitor) class Reader(standalone.Reader): supported = standalone.Reader.supported + ('wiki',) default_transforms = standalone.Reader.default_transforms \ + (WikiLink,) ###################################### # Then to actually use it I had to do: from docutils import readers # WikiReST is the module I gave above, and it's located here: sys.path.append('/usr/home/ianb/prog/WebwareWiki/Wiki') readers._reader_aliases['wiki'] = 'WikiReST' # Then I use 'wiki' for the reader_name in docutils.core.publish_string ###################################### Of course, it would be good if there was a better hook for adding a reader like that. For instance, a better use of __import__ in docutils.readers.__init__ would have kept me from having to add something to the path (I could have used "readers._reader_aliases['wiki'] = 'Wiki.WikiReST'"), and of course it would be better still if I could have simply passed in the class to an argument to publish_string. Of course, now that I look at it, there's a reader argument to publish_string -- but I have to give it an instance, and that seem awkward to make -- how can I instantiate the reader if I have to pass the parser to the constructor, and the parser is created in publish_string? I'm sure there's a better way, or you can add a better way ;) But it works without heroic efforts, and that's very cool. Oh, and what exactly is the default_priority bit? If I set it high, will Wiki links be created before internal links (hence subsuming internal links)? Ian |
From: David G. <go...@us...> - 2002-10-31 02:44:03
|
Ian Bicking wrote: > Wow, it actually works. That was easier than I thought it would be. Always good to hear ;-) > Here's what I did: ... > class WikiLink(Transform): > > default_priority = 400 ... > Oh, and what exactly is the default_priority bit? If I set it high, > will Wiki links be created before internal links (hence subsuming > internal links)? default_priority is used for ordering the transforms. From PEP 258 (http://docutils.sf.net/spec/pep-0258.html#transformer): Transform classes each have a ``default_priority`` attribute which is used by the Transformer to apply transforms in order (low to high). The default priority can be overridden when adding transforms to the Transformer object. Perhaps "priority" isn't the right term? (Better term, anyone?) The transform with the lowest priority value is applied first. There's the connotation that "highest priority" goes first, opposite from what's used. See http://docutils.sf.net/spec/transforms.html for a complete list of transforms and the order they are applied. I would say that the WikiLink transform should be applied after references.InternalTargets, and therefore should have a priority value of 680. The value of 400 used above will apply WikiLink before any of the standard hyperlink resolution transforms, causing *all* references to be Wiki references (i.e., none would be "resolved" prior to WikiLink seeing them). > # Then to actually use it I had to do: > > from docutils import readers > # WikiReST is the module I gave above, and it's located here: > sys.path.append('/usr/home/ianb/prog/WebwareWiki/Wiki') > readers._reader_aliases['wiki'] = 'WikiReST' > > # Then I use 'wiki' for the reader_name in > docutils.core.publish_string That would be fine if your Reader was a module installed in docutils/readers. Since it isn't... > it would be better still if I could have simply passed in the class > to an argument to publish_string. Of course, now that I look at it, > there's a reader argument to publish_string Bingo! > -- but I have to give it an instance, and that seem awkward to make > -- how can I instantiate the reader if I have to pass the parser to > the constructor, and the parser is created in publish_string? The Reader will instantiate the parser automatically. Like this:: import WikiReST from docutils import core reader = WikiReST.Reader(parser=None, parser_name='restructuredtext') output = docutils.core.publish_string(reader=reader, parser=reader.parser, ...) I've just added default parameter values to the ``docutils.readers.Reader.__init__`` method to make Reader instantiation easier:: reader = WikiReST.Reader() > Of course, it would be good if there was a better hook for adding a > reader like that. As I mentioned, that interface is only meant for installed modules. > For instance, a better use of __import__ in > docutils.readers.__init__ would have kept me from having to add > something to the path In fact, there's a to-do item that proposes removing the temptation of using sys.path magic: * In ``docutils.readers.get_reader_class`` (& ``parsers`` & ``writers`` too), should we be importing "standalone" or "docutils.readers.standalone"? (This would avoid importing top-level modules if the module name is not in docutils/readers. Potential nastiness.) So it becomes a simple choice: get a standard (installed) component by name, or pass a custom component object explicitly. -- David Goodger <go...@us...> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ |
From: Aahz <aa...@py...> - 2002-10-31 04:36:03
|
On Wed, Oct 30, 2002, David Goodger wrote: > > Perhaps "priority" isn't the right term? (Better term, anyone?) The > transform with the lowest priority value is applied first. There's > the connotation that "highest priority" goes first, opposite from > what's used. Perhaps an entirely different system might be better. Instead of making ordering implict based on the priority ("order" might be a better word, particularly since increasing numbers work well with order)), instead have some kind of ``TransformList`` class with methods ``insert_before()`` and ``insert_after()`` that take a transform that's the insert point (0 and -1 indicate first and last as usual), plus either a new transform or an ordered sequence of transforms to be inserted. People are much more likely to remember the names of transforms than either the number or the exact order. -- Aahz (aa...@py...) <*> http://www.pythoncraft.com/ Project Vote Smart: http://www.vote-smart.org/ |
From: David G. <go...@us...> - 2002-10-31 05:18:50
|
> On Wed, Oct 30, 2002, David Goodger wrote: >> Perhaps "priority" isn't the right term? (Better term, anyone?) The >> transform with the lowest priority value is applied first. There's >> the connotation that "highest priority" goes first, opposite from >> what's used. Aahz wrote: > "order" might be a better word, particularly since increasing > numbers work well with order Perhaps, although "order" is such an overloaded term that it almost always requires clarification. Some other candidate terms: sequence, index, rank/ranking, grade, position. > Perhaps an entirely different system might be better. Instead of making > ordering implict based on the priority ..., instead > have some kind of ``TransformList`` class with methods > ``insert_before()`` and ``insert_after()`` that take a transform that's > the insert point (0 and -1 indicate first and last as usual), plus either > a new transform or an ordered sequence of transforms to be inserted. I suspect that such a system would have its own problems. To position transform A before transform B, B would have to be registered already. Or B would have to know about the relationship, which may not always be the case. The previous system, similar to the above, failed and was replaced by the current one because it was too coarse. An absolute numbering is very clear and simple IMO. > People are much more likely to remember the names of transforms than > either the number or the exact order. Although few of us will ever have to look at the transforms, and only infrequently. The spec/transforms.txt file lists them all, in order, when we do need to refer to them. -- David Goodger <go...@us...> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ |
From: <gr...@us...> - 2002-11-08 16:00:51
|
is it correct that the numbering scheme is not taken from source. this:: ordered list with numbering. 1. one 2. two another list with Uppercase letters. A. an A B. a B how does it look like. produces this:: ordered list with numbering. 1. one 2. two another list with Uppercase letters. 1. an A 2. a B how does it look like. -- BINGO: completely leverage other's inexpensive benefits --- Engelbert Gruber -------+ SSG Fintl,Gruber,Lassnig / A6410 Telfs Untermarkt 9 / Tel. ++43-5262-64727 ----+ |
From: David G. <go...@py...> - 2002-11-08 21:57:17
|
gr...@us... wrote: > is it correct that the numbering scheme is not taken from source. No. It works in HTML. What output format/Writer are you talking about? -- David Goodger <go...@py...> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/ |
From: engelbert g. <be...@ch...> - 2002-11-09 07:21:01
|
On Fri, 8 Nov 2002, David Goodger wrote: > gr...@us... wrote: > > is it correct that the numbering scheme is not taken from source. > > No. It works in HTML. What output format/Writer are you talking > about? tools/html.py, whatever it uses. i see to it closer. -- engelbert gruber email: eng...@ss... |
From: Garth K. <ga...@de...> - 2002-10-31 17:29:12
|
> > The one thing I *can't* do is help you figure out how to do this in > > code, because I haven't spent much time with reST in ages (okay, > > ever), > > Garth *did* do a very cool overhaul of the unit test > framework, for which he has our eternal gratitude. :) |