From: Dave K. <dku...@re...> - 2007-03-15 23:08:06
|
Here is a proposal for automatically loadable user directives. I'd like to suggest that this be considered along-side Felix's extensions proposal. This one is much less ambitious and not nearly so powerful as Felix's. However, it is, I think, quiet a bit easier for users to do. And, these two proposals are not mutually exclusive; there is no reason that we not do both. You can find the proposal here: http://svn.berlios.de/viewcvs/*checkout*/docutils/trunk/sandbox/dkuhlman/ OpenDocument/docs/plugins_proposal.html Dave |
From: Dave K. <dku...@re...> - 2007-03-15 23:32:26
|
Dave Kuhlman <dkuhlman <at> rexx.com> writes: > > Here is a proposal for automatically loadable user directives. I'd > like to suggest that this be considered along-side Felix's > extensions proposal. This one is much less ambitious and not > nearly so powerful as Felix's. However, it is, I think, quiet a > bit easier for users to do. And, these two proposals are not > mutually exclusive; there is no reason that we not do both. > > You can find the proposal here: > > http://svn.berlios.de/viewcvs/*checkout*/docutils/trunk/sandbox/dkuhlman/ > OpenDocument/docs/plugins_proposal.html Phooey. I don't know how to make a long link through the Gmane interface, and messages to the list that I send from my email account are rejected. So here is a link that works, I hope: http://www.rexx.com/~dkuhlman/plugins_proposal.html Dave |
From: David G. <go...@py...> - 2007-03-21 14:02:46
|
On 3/15/07, Dave Kuhlman <dku...@re...> wrote: > Here is a proposal for automatically loadable user directives. I'd > like to suggest that this be considered along-side Felix's > extensions proposal. I haven't had time to go throught the proposal thoroughly yet. I have serious reservations about the setuptools approach, but I haven't gone through that either. Perhaps this weekend. But this reply has been sitting in my scratch file for too long. Past time to send it out, for what it is.. > So here is a link that works, I hope: > > http://www.rexx.com/~dkuhlman/plugins_proposal.html Note that it's also available at http://docutils.sf.net/sandbox/dkuhlman/OpenDocument/docs/plugins_proposal.html Still pretty long. I recommend you move the project up a level, outside of your personal sandbox into a project sandbox: svn mv sandbox/dkuhlman/OpenDocument sandbox/OpenDocument As http://docutils.sf.net/docs/dev/policies.html#the-sandbox says, "project directories are recommended over personal directories, which discourage collaboration". A couple of administrative issues: * Please don't abuse the PEP system. PEP stands for Python (not Docutils) Enhancement Proposal. Please change your proposal to an ordinary standalone document. I don't think the Python developers would want such a proposal as a PEP. Actually, I am one of the PEP editors, and I know I wouldn't. It would be easy to implement DEPs, but do we really need to? * There's no need to check in .html files that are processed from .txt. Please remove the .html files from the SVN repository, and see http://docutils.sf.net/docs/dev/website.html for the procedure to enable auto-updating of the web site. -- David Goodger <http://python.net/~goodger> |
From: G. M. <mi...@us...> - 2007-03-16 12:31:41
|
On 15.03.07, Dave Kuhlman wrote: > Here is a proposal for automatically loadable user directives. > http://svn.berlios.de/viewcvs/*checkout*/docutils/trunk/sandbox/dkuhlman/OpenDocument/docs/plugins_proposal.html > I'd like to suggest that this be considered along-side Felix's > extensions proposal. http://svn.berlios.de/viewcvs/docutils/branches/plugins/docs/howto/extensions.txt?view=markup One point that (as far as I see) touches both proposals: Problem ------- * reStructured Text documents that rely on extensions to docutils will lead to errors in non-extended installations. (clear) * their non-compatibility with the standard docutils should be visible **in the document** on a prominent place. * There are precedence cases for such a situation: - LaTeX packages (``\usepackage{PACKAGE}``) - Python modules (``import``) * The typical solution is a kind of import statement at the top of the source that - triggers the evaluation of the extension. - is a clear visible indicator of the compilation requirements - throws a more precise error than a missing directive (or function) --> the user will see which plug-in is missing. Proposal -------- My proposal is a new "use-module" directive for the import of plug-ins. It would foot the extension of docutils with plug-ins on established principles. * The directive:: .. use-module:: PLUG_IN will cause docutils to import the Python module PLUG_IN. * A missing module throws a legible Python exception. (It is also feasible that the directives implementation catches this exception and transforms it to an even more telling docutils error message.) * There is no overhead if a document does not use a plug-in, the modules are searched and loaded only on demand. * No registration needed, installation of a plug-in is reduced to the installation of a Python module (A registration could, however, provide a "canonical" alias as part of the docutils package.) * The plug-in could be - part of a different package - reside in a "docutils plug-ins directory" - installed by a user in a private extension to the PYTHONPATH - reside in the current working directory. Security ~~~~~~~~ Allowing any kind of Python module to be imported is a big security challenge. * It must be possible to disable this directive (similar to "raw" and "include"). * A more fine-grained approach could restrict it to "registered" modules or modules in an "approved" location. Name considerations ~~~~~~~~~~~~~~~~~~~ "use-module" is an adaption of the "\usepackage" command from LaTeX. It makes clear that the documents uses (i.e. depends on) an external resource which happens to be a Python module. "import" maps nicely to the Python expression it calls. However, is it clear that we are not 'importing' a rst source? In css, "@import" is used to "include" another style sheet. Also, "import" might be confused with the "include" directive more easily. Thanks Günter |
From: David G. <go...@py...> - 2007-03-21 14:06:21
|
T24gMy8xNi8wNywgRy4gTWlsZGUgPG1pbGRlQHVzZXJzLmJlcmxpb3MuZGU+IHdyb3RlOgo+IE9u ZSBwb2ludCB0aGF0IChhcyBmYXIgYXMgSSBzZWUpIHRvdWNoZXMgYm90aCBwcm9wb3NhbHM6Cj4K PiBQcm9ibGVtCj4gLS0tLS0tLQo+Cj4gKiByZVN0cnVjdHVyZWQgVGV4dCBkb2N1bWVudHMgdGhh dCByZWx5IG9uIGV4dGVuc2lvbnMgdG8gZG9jdXRpbHMKPiAgIHdpbGwgbGVhZCB0byBlcnJvcnMg aW4gbm9uLWV4dGVuZGVkIGluc3RhbGxhdGlvbnMuIChjbGVhcikKPgo+ICogdGhlaXIgbm9uLWNv bXBhdGliaWxpdHkgd2l0aCB0aGUgc3RhbmRhcmQgZG9jdXRpbHMgc2hvdWxkIGJlCj4gICB2aXNp YmxlICoqaW4gdGhlIGRvY3VtZW50Kiogb24gYSBwcm9taW5lbnQgcGxhY2UuCgpGZWxpeCBhbmQg SSBkZWJhdGVkIHRoaXMgcG9pbnQgYXQgUHlDb24uICBGZWxpeCB0aG91Z2h0IGV4cGxpY2l0Cmlu ZGljYXRpb25zIG9mIGRlcGVuZGVuY2llcyBpbiBkb2N1bWVudHMgd291bGQgYmUgb3Zlcmx5IGlu dHJ1c2l2ZSBhbmQKb2Jub3hpb3VzLiAgT24gbm9uLWV4dGVuZGVkIHN5c3RlbXMsIGFuIGVycm9y IG1lc3NhZ2UgY291bGQgaW5jbHVkZQp0ZXh0IGxpa2UgInNlZSA8VVJMPiBmb3IgRG9jdXRpbHMg cGx1Zy1pbiBtb2R1bGVzIi4KCkJ1dCB0aGUgbW9yZSBJIHRoaW5rIGFib3V0IGl0LCB0aGUgbW9y ZSBJIGFncmVlIHdpdGggR8O8bnRlci4gIFdpdGhvdXQKZXhwbGljaXQgaW5kaWNhdGlvbnMgKGku ZS4gYSBkaXJlY3RpdmUgZXF1aXZhbGVudCB0byBQeXRob24ncwoiaW1wb3J0IiksIERvY3V0aWxz IHdvdWxkIGhhdmUgdG8gbG9hZCBhbGwgcGx1Zy1pbnMgdW5jb25kaXRpb25hbGx5LAphbmQgdGhl cmUgd291bGQgYmUgbm8gd2F5IHRvIGtub3cgdGhlIGRpZmZlcmVuY2UgYmV0d2VlbiBhIG1pc3Nw ZWxsZWQKZGlyZWN0aXZlIGFuZCBhIHBsdWctaW4gdGhhdCBpc24ndCBpbnN0YWxsZWQuICBQbHVn LWlucyBjb3VsZCBiZQpkZXZpc2VkIHRoYXQgYWZmZWN0IGRvY3VtZW50cyAodmlhIHRoZSBwYXJz ZXIsIHNheSkgd2l0aG91dCBhbnkKZGlyZWN0aXZlcyBiZWluZyB1c2VkLiAgV2l0aG91dCBleHBs aWNpdCBpbmRpY2F0aW9ucywgYWxsIGRvY3VtZW50cwp3b3VsZCBiZWNvbWUgYW1iaWd1b3VzLgoK PiBQcm9wb3NhbAo+IC0tLS0tLS0tCj4KPiBNeSBwcm9wb3NhbCBpcyBhIG5ldyAidXNlLW1vZHVs ZSIgZGlyZWN0aXZlIGZvciB0aGUgaW1wb3J0IG9mCj4gcGx1Zy1pbnMuICBJdCB3b3VsZCBmb290 IHRoZSBleHRlbnNpb24gb2YgZG9jdXRpbHMgd2l0aCBwbHVnLWlucyBvbgo+IGVzdGFibGlzaGVk IHByaW5jaXBsZXMuCgoiZm9vdCI/Cgo+ICogVGhlIHBsdWctaW4gY291bGQgYmUKPgo+ICAgLSBw YXJ0IG9mIGEgZGlmZmVyZW50IHBhY2thZ2UKCkkgZG9uJ3QgdGhpbmsgdGhpcyBpcyBhIGdvb2Qg aWRlYS4gIEhvdyBkbyB5b3UgcHJvcG9zZSBsb2FkaW5nIGZyb20gYQpwYWNrYWdlPyAgQW5kIHNl ZSBQWVRIT05QQVRIIGJlbG93LgoKPiAgIC0gcmVzaWRlIGluIGEgImRvY3V0aWxzIHBsdWctaW5z IGRpcmVjdG9yeSIKClNpbWlsYXJseSB0byB0aGUgRE9DVVRJTFNDT05GSUcgZW52aXJvbm1lbnQg dmFyaWFibGUsIHRoZXJlIGNvdWxkIGJlIGEKRE9DVVRJTFNQTFVHSU5TIHZhcmlhYmxlIHRvby4g IEFsc28gc2ltaWxhcmx5IHRvIGNvbmZpZ3VyYXRpb24gZmlsZXMsCnRoZXJlIGNvdWxkIGJlIGEg ZGVmYXVsdCBzZXQgb2YgbG9jYXRpb25zIHRvIHNlYXJjaCBmb3IgcGx1Zy1pbgptb2R1bGVzLgoK PiAgIC0gaW5zdGFsbGVkIGJ5IGEgdXNlciBpbiBhIHByaXZhdGUgZXh0ZW5zaW9uIHRvIHRoZSBQ WVRIT05QQVRICgpJIHRoaW5rIGxvYWRpbmcgcGx1Zy1pbiBtb2R1bGVzIGZyb20gUFlUSE9OUEFU SCBpcyBOT1QgYSBnb29kIGlkZWEuICBJCnNlZSBhIGNsZWFyIG5lZWQgZm9yIHBsdWctaW4gbW9k dWxlcyB0byBiZSBzZXBhcmF0ZSBmcm9tIHJlZ3VsYXIKUHl0aG9uIG1vZHVsZXMuICBUaGV5J3Jl IG5vdCBnZW5lcmFsLXB1cnBvc2UgUHl0aG9uIG1vZHVsZXMsIGFuZApub3RoaW5nIG91dHNpZGUg b2YgRG9jdXRpbHMgc2hvdWxkIG5lZWQgdG8gaW1wb3J0IHRoZW0sIHNvIHRoZXJlJ3Mgbm8KbmVl ZCBmb3IgdGhlbSB0byBiZSBvbiBQWVRIT05QQVRILgoKPiAgIC0gcmVzaWRlIGluIHRoZSBjdXJy ZW50IHdvcmtpbmcgZGlyZWN0b3J5LgoKVGhhdCB3b3VsZCBiZSBPSywgZ29vZCBmb3IgZGV2ZWxv cG1lbnQvdGVzdGluZy4KCj4gU2VjdXJpdHkKPiB+fn5+fn5+fgo+Cj4gQWxsb3dpbmcgYW55IGtp bmQgb2YgUHl0aG9uIG1vZHVsZSB0byBiZSBpbXBvcnRlZCBpcyBhIGJpZyBzZWN1cml0eQo+IGNo YWxsZW5nZS4KPgo+ICogSXQgbXVzdCBiZSBwb3NzaWJsZSB0byBkaXNhYmxlIHRoaXMgZGlyZWN0 aXZlIChzaW1pbGFyIHRvICJyYXciCj4gICBhbmQgImluY2x1ZGUiKS4KPgo+ICogQSBtb3JlIGZp bmUtZ3JhaW5lZCBhcHByb2FjaCBjb3VsZCByZXN0cmljdCBpdCB0byAicmVnaXN0ZXJlZCIKPiAg IG1vZHVsZXMgb3IgbW9kdWxlcyBpbiBhbiAiYXBwcm92ZWQiIGxvY2F0aW9uLgoKSSdtIG5vdCBz dXJlIGFib3V0IHRoYXQuICBUaGUgc2VjdXJpdHkgaXNzdWVzIHBvc2VkIGJ5ICJyYXciIGFuZAoi aW5jbHVkZSIgYXJpc2UgZnJvbSB0aHJvdWdoLXRoZS13ZWIgdXNlIG9mIERvY3V0aWxzIChpbiB3 aWtpcyBldGMuKS4KVGhlIGFkbWluaXN0cmF0b3Igb2YgYSB3ZWIgc2l0ZS9hcHAgY29udHJvbHMg dGhlIHBsdWctaW4gbW9kdWxlcwpwcm92aWRlZC4gIElmIGEgdXNlciBoYXMgZW5vdWdoIGFjY2Vz cyB0byB0aGUgZmlsZSBzeXN0ZW0gaW4gb3JkZXIgdG8KaW5zdGFsbCBhIHBsdWctaW4sIHRoZSBz ZWN1cml0eSBvZiBEb2N1dGlscyBpcyBub3QgYW4gaXNzdWUuCgpJIGRvbid0IHRoaW5rIHdlIG5l ZWQgdG8gcHJvdmlkZSBhbnkgZGlzYWJsaW5nIGZ1bmN0aW9uYWxpdHkgZm9yCnBsdWctaW5zLiAg SWYgeW91IGRvbid0IHdhbnQgYSBwbHVnLWluIGZvciB5b3VyIHNpdGUvYXBwLCBqdXN0IGRvbid0 Cmluc3RhbGwgaXQuCgo+IE5hbWUgY29uc2lkZXJhdGlvbnMKPiB+fn5+fn5+fn5+fn5+fn5+fn5+ CgpJIHByZWZlciAicmVxdWlyZSIuICBJZiB1bmF2YWlsYWJsZSwgdGhlIERvY3V0aWxzIGVycm9y IG1lc3NhZ2Ugd291bGQKYmUgInRoZSByZXF1aXJlZCBwbHVnLWluIGlzIG5vdCBhdmFpbGFibGUi LgoKInVzZS1tb2R1bGUiIGlzIGN1bWJlcnNvbWUsIGFuZCBleHBvc2VzIHRoZSBpbXBsZW1lbnRh dGlvbi4KCkkgYWdyZWUgdGhhdCAiaW1wb3J0IiBpcyB0b28gY2xvc2UgdG8gImluY2x1ZGUiLgoK LS0gCkRhdmlkIEdvb2RnZXIgPGh0dHA6Ly9weXRob24ubmV0L35nb29kZ2VyPgo= |
From: G. M. <mi...@us...> - 2007-03-21 15:46:13
|
On 21.03.07, David Goodger wrote: > On 3/16/07, G. Milde <mi...@us...> wrote: > > > > Problem > > ------- > > > > * reStructured Text documents that rely on extensions to docutils > > will lead to errors in non-extended installations. (clear) > > > > * their non-compatibility with the standard docutils should be > > visible **in the document** on a prominent place. > Felix and I debated this point at PyCon. Felix thought explicit > indications of dependencies in documents would be overly intrusive and > obnoxious. Let me quote from the reST design goals: Unobtrusive. The markup that is used should be as simple and unobtrusive as possible. The simplicity of markup constructs should be roughly proportional to their frequency of use. Less common constructs, for which there is no natural or obvious markup, should be distinctive. For me, this translates to * Using a non-standard plug in is IMO less common, so the markup should be *distinctive* (and explicit). * If features provided by a plug-in are used frequently, their markup should become simpler. In this case, the plug-in should be made a part of docutils thus obsoleting the need for an import directive. > But the more I think about it, the more I agree with Günter. ... > Without explicit indications, all documents would become ambiguous. This is my point. > > Proposal > > -------- > > > > My proposal is a new "use-module" directive for the import of > > plug-ins. It would foot the extension of docutils with plug-ins on > > established principles. > "foot"? base: vb. n. {Footing}.] 2. To set on foot; to establish; to land. [Obs.] I have to admit that ``wordnet foot -famlv``:: foot used as a verb is uncommon (polysemy count = 3) > > * The plug-in could be > > > > - part of a different package > I don't think this is a good idea. How do you propose loading from a > package? And see PYTHONPATH below. My basic idea was to use the "path" or "module" argument to the import directive to a Python import statement. + No additional configuration needed. + Installation of a docutils plug-in == installation of a Python module: no additional tools, no additional documentation to write and read, re-use existing expertise of package writers and administrators. - Mingles docutils add-ons and Python modules Let us assume the Pygments package provides a syntax highlight plug-in in the writers sub-package. Then :: .. require:: pygments.writers.docutils would translate to import pygments.writers.docutils > > - reside in a "docutils plug-ins directory" > Similarly to the DOCUTILSCONFIG environment variable, there could be a > DOCUTILSPLUGINS variable too. Also similarly to configuration files, > there could be a default set of locations to search for plug-in > modules. Although I wanted to avoid a parallel structure, I can live with that. > > - reside in the current working directory. > That would be OK, good for development/testing. So, the cwd would be one of the default locations... > > Security > > ~~~~~~~~ > > > > Allowing any kind of Python module to be imported is a big security > > challenge. > I'm not sure about that. ... I am glad to hear this. Makes things easier. > > Name considerations > > ~~~~~~~~~~~~~~~~~~~ > I prefer "require". If unavailable, the Docutils error message would > be "the required plug-in is not available". OK. Thanks Guenter |
From: Beni C. <cb...@us...> - 2007-04-07 22:46:17
|
On 3/21/07, G. Milde <mi...@us...> wrote: > On 21.03.07, David Goodger wrote: > > I don't think this is a good idea. How do you propose loading from a > > package? And see PYTHONPATH below. > > > > - reside in a "docutils plug-ins directory" > > > Similarly to the DOCUTILSCONFIG environment variable, there could be a > > DOCUTILSPLUGINS variable too. Also similarly to configuration files, > > there could be a default set of locations to search for plug-in > > modules. > > Although I wanted to avoid a parallel structure, I can live with that. > These 2 approches can be merged: the import would always be done from a package (`docutils.plugins` or whatever) but `docutils.plugins.__path__` would be initialized from ``DOCUTILSPLUGINS`` (and/or a config file and/or command-line option). This has several properties that sound nice to me: * the default installation location is within the docutils installation * other plugin locations can be easily added * plugins can require and use other plugins with a simple import, irrespective of installation location. > > > - reside in the current working directory. > > > That would be OK, good for development/testing. > > So, the cwd would be one of the default locations... > For development/testing - fine - but not by default, please! That would be a security risk! I can easily imagine a situation where the cwd is the directory where the document resides, and this directory is writable by another user (perhaps through some web interface). In such a case the user that was supposed to only provide textual input to docutils will gain the ability to execute arbitrary python code. More generally, it's very hard to *always* know what is the cwd and what modules it contains (just like having ``.`` in your PATH is bad practice). -- Beni Cherniavsky <cb...@us...> (I read email only on weekends) |
From: Dave K. <dku...@re...> - 2007-03-22 00:20:32
|
On Wed, Mar 21, 2007 at 10:02:41AM -0400, David Goodger wrote: > > Still pretty long. I recommend you move the project up a level, > outside of your personal sandbox into a project sandbox: > > svn mv sandbox/dkuhlman/OpenDocument sandbox/OpenDocument Done. > > As http://docutils.sf.net/docs/dev/policies.html#the-sandbox says, > "project directories are recommended over personal directories, which > discourage collaboration". > > A couple of administrative issues: > > * Please don't abuse the PEP system. PEP stands for Python (not > Docutils) Enhancement Proposal. Please change your proposal to an > ordinary standalone document. Done. I didn't mean for it to be confused with a real PEP. But, it seemed like a reasonable format. Now it is a more plain Docutils doc. > > * There's no need to check in .html files that are processed from > .txt. Please remove the .html files from the SVN repository, and > see http://docutils.sf.net/docs/dev/website.html for the procedure > to enable auto-updating of the web site. Aw. My generated docs were so pretty. OK. I've removed the .html and .odt files. Dave |