docstring-checkins Mailing List for Docstring Processing System (Page 3)
Status: Pre-Alpha
Brought to you by:
goodger
You can subscribe to this list here.
2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(53) |
Sep
(54) |
Oct
(26) |
Nov
(27) |
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2002 |
Jan
(60) |
Feb
(85) |
Mar
(94) |
Apr
(40) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: David G. <go...@us...> - 2002-03-28 04:38:52
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv15477/dps/spec Modified Files: gpdi.dtd Log Message: - Added "refids" attribute to footnote, citation, and system_message, for backlinks. Index: gpdi.dtd =================================================================== RCS file: /cvsroot/docstring/dps/spec/gpdi.dtd,v retrieving revision 1.39 retrieving revision 1.40 diff -C2 -d -r1.39 -r1.40 *** gpdi.dtd 16 Mar 2002 05:50:44 -0000 1.39 --- gpdi.dtd 28 Mar 2002 04:38:48 -0000 1.40 *************** *** 65,68 **** --- 65,72 ---- " refid IDREF #IMPLIED "> + <!-- Space-separated list of id references, for backlinks. --> + <!ENTITY % refids.att + " refids IDREFS #IMPLIED "> + <!-- Internal reference to the `name` attribute of an element. On a *************** *** 375,382 **** <!ATTLIST footnote %basic.atts; %auto.att;> <!ELEMENT citation (label, (%body.elements;)+)> ! <!ATTLIST citation %basic.atts;> <!ELEMENT label (#PCDATA)> --- 379,389 ---- <!ATTLIST footnote %basic.atts; + %refids.att; %auto.att;> <!ELEMENT citation (label, (%body.elements;)+)> ! <!ATTLIST citation ! %basic.atts; ! %refids.att;> <!ELEMENT label (#PCDATA)> *************** *** 429,433 **** <!ATTLIST system_message %basic.atts; ! %refid.att; level NMTOKEN #IMPLIED type CDATA #IMPLIED> --- 436,440 ---- <!ATTLIST system_message %basic.atts; ! %refids.att; level NMTOKEN #IMPLIED type CDATA #IMPLIED> |
From: David G. <go...@us...> - 2002-03-28 04:36:56
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv14008/dps/spec Modified Files: pep-0258.txt Log Message: Overhauled; changed title. Index: pep-0258.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/pep-0258.txt,v retrieving revision 1.6 retrieving revision 1.7 diff -C2 -d -r1.6 -r1.7 *** pep-0258.txt 15 Feb 2002 22:53:17 -0000 1.6 --- pep-0258.txt 28 Mar 2002 04:36:53 -0000 1.7 *************** *** 1,7 **** PEP: 258 ! Title: DPS Generic Implementation Details Version: $Revision$ Last-Modified: $Date$ ! Author: dgo...@bi... (David Goodger) Discussions-To: do...@py... Status: Draft --- 1,7 ---- PEP: 258 ! Title: Docutils Design Specification Version: $Revision$ Last-Modified: $Date$ ! Author: go...@us... (David Goodger) Discussions-To: do...@py... Status: Draft *************** *** 14,23 **** Abstract ! This PEP documents generic implementation details for a Python ! Docstring Processing System (DPS). The rationale and high-level ! concepts of the DPS are documented in PEP 256, "Docstring ! Processing System Framework" [1]. No changes to the core Python language are required by this PEP. --- 14,27 ---- Abstract ! This PEP documents design issues and implementation details for ! Docutils, a Python Docstring Processing System (DPS). The ! rationale and high-level concepts of a DPS are documented in PEP ! 256, "Docstring Processing System Framework" [1]. No changes to the core Python language are required by this PEP. + Its deliverables consist of a package for the standard library and + its documentation. + + @@@ s/dps/Docutils/ from here down *************** *** 29,39 **** 1. What to examine: ! a) If the '__all__' variable is present in the module being ! documented, only identifiers listed in '__all__' are examined for docstrings. ! b) In the absense of '__all__', all identifiers are examined, ! except those whose names are private (names begin with '_' ! but don't begin and end with '__'). c) 1a and 1b can be overridden by a parameter or command-line --- 33,43 ---- 1. What to examine: ! a) If the "__all__" variable is present in the module being ! documented, only identifiers listed in "__all__" are examined for docstrings. ! b) In the absense of "__all__", all identifiers are examined, ! except those whose names are private (names begin with "_" ! but don't begin and end with "__"). c) 1a and 1b can be overridden by a parameter or command-line *************** *** 58,75 **** concatenated. See "Additional Docstrings" below. 3. How: ! Whenever possible, Python modules should be parsed by the ! docstring processing system, not imported. There are security ! reasons for not importing untrusted code. Also, docstrings are ! to be recognized in places where the bytecode compiler ignores ! string literal expressions (2b and 2c above), meaning importing ! the module will lose these docstrings. Of course, standard ! Python parsing tools such as the 'parser' library module may ! be used. When the Python source code for a module is not available (i.e. only the .pyc file exists) or for C extension modules, to ! access docstrings the module must be imported. Since attribute docstrings and additional docstrings are ignored --- 62,83 ---- concatenated. See "Additional Docstrings" below. + d) @@@ 2.2-style "properties" with attribute docstrings? + 3. How: ! Whenever possible, Python modules should be parsed by Docutils, ! not imported. There are security reasons for not importing ! untrusted code. Information from the source is lost when using ! introspection to examine an imported module, such as comments ! and the order of definitions. Also, docstrings are to be ! recognized in places where the bytecode compiler ignores string ! literal expressions (2b and 2c above), meaning importing the ! module will lose these docstrings. Of course, standard Python ! parsing tools such as the "parser" library module may be used. When the Python source code for a module is not available (i.e. only the .pyc file exists) or for C extension modules, to ! access docstrings the module can only be imported, and any ! limitations must be lived with. Since attribute docstrings and additional docstrings are ignored *************** *** 79,82 **** --- 87,91 ---- module may take a slight performance hit. + Attribute Docstrings -------------------- *************** *** 98,102 **** b) At the top level of a class definition: a class attribute. ! c) At the top level of the '__init__' method definition of a class: an instance attribute. --- 107,111 ---- b) At the top level of a class definition: a class attribute. ! c) At the top level of the "__init__" method definition of a class: an instance attribute. *************** *** 116,121 **** b) For context 1c above, the target must be of the form ! 'self.attrib', where 'self' matches the '__init__' method's ! first parameter (the instance parameter) and 'attrib' is a simple indentifier as in 3a. --- 125,130 ---- b) For context 1c above, the target must be of the form ! "self.attrib", where "self" matches the "__init__" method's ! first parameter (the instance parameter) and "attrib" is a simple indentifier as in 3a. *************** *** 125,129 **** Examples:: ! g = 'module attribute (global variable)' """This is g's docstring.""" --- 134,138 ---- Examples:: ! g = 'module attribute (module-global variable)' """This is g's docstring.""" *************** *** 137,147 **** """This is self.i's docstring.""" Additional Docstrings --------------------- Many programmers would like to make extensive use of docstrings for API documentation. However, docstrings do take up space in the running program, so some of these programmers are reluctant to ! 'bloat up' their code. Also, not all API documentation is applicable to interactive environments, where __doc__ would be displayed. --- 146,160 ---- """This is self.i's docstring.""" + Additional Docstrings --------------------- + (This idea was adapted from PEP 216, Docstring Format [3], by + Moshe Zadka.) + Many programmers would like to make extensive use of docstrings for API documentation. However, docstrings do take up space in the running program, so some of these programmers are reluctant to ! "bloat up" their code. Also, not all API documentation is applicable to interactive environments, where __doc__ would be displayed. *************** *** 166,170 **** pass ! Issue: This breaks 'from __future__ import' statements in Python 2.1 for multiple module docstrings. The Python Reference Manual specifies: --- 179,183 ---- pass ! Issue: This breaks "from __future__ import" statements in Python 2.1 for multiple module docstrings. The Python Reference Manual specifies: *************** *** 173,180 **** only lines that can appear before a future statement are: ! * the module docstring (if any), ! * comments, ! * blank lines, and ! * other future statements. Resolution? --- 186,193 ---- only lines that can appear before a future statement are: ! * the module docstring (if any), ! * comments, ! * blank lines, and ! * other future statements. Resolution? *************** *** 187,193 **** 3. Or should we not even worry about this? There shouldn't be ! __future__ statements in production code, after all. Modules ! with __future__ statements will have to put up with the ! single-docstring limitation. Choice of Docstring Format --- 200,207 ---- 3. Or should we not even worry about this? There shouldn't be ! __future__ statements in production code, after all. Will ! modules with __future__ statements simply have to put up with ! the single-docstring limitation? ! Choice of Docstring Format *************** *** 203,208 **** format being used, a case-insensitive string matching the input parser's module or package name (i.e., the same name as required ! to 'import' the module or package), or a registered alias. If no ! __docformat__ is specified, the default format is 'plaintext' for now; this may be changed to the standard format once determined. --- 217,222 ---- format being used, a case-insensitive string matching the input parser's module or package name (i.e., the same name as required ! to "import" the module or package), or a registered alias. If no ! __docformat__ is specified, the default format is "plaintext" for now; this may be changed to the standard format once determined. *************** *** 214,347 **** exists; RFC 1766 is currently being revised to allow 3-letter codes). If no language identifier is specified, the default is ! 'en' for English. The language identifier is passed to the parser and can be used for language-dependent markup features. - DPS Structure - ============= ! - package 'dps' ! - function 'dps.main()' (in 'dps/__init__.py') ! - package 'dps.parsers' ! - module 'dps.parsers.model'; see 'Input Parser API' below. - - package 'dps.formatters' ! - module 'dps.formatters.model'; see 'Output Formatter API' ! below. ! - package 'dps.languages' - - module 'dps.languages.en' (English) ! - others to be added ! - utility modules: 'dps.nodes', 'dps.statemachine', 'dps.utils' ! Command-Line Interface ! ====================== ! XXX To be determined. ! System Python API ! ================= ! XXX To be determined. ! Input Parser API ! ================ ! Each input parser is a module or package exporting a 'Parser' ! class, with the following interface: ! class Parser: ! def __init__(self, inputstring, warninglevel=1, ! errorlevel=3, language='en'): ! """Initialize the Parser instance.""" ! def parse(self): ! """Parse the input string and return a tree.""" ! XXX This needs a lot of work. What is required for this API? ! A model 'Parser' class implementing the full interface along with ! utility functions can be found in the 'dps.parsers.model' module. ! Output Formatter API ! ==================== ! Each output formatter is a module or package exporting a ! 'Formatter' class, with the following interface: ! class Formatter: ! def __init__(self, domtree, language='en', ! showwarnings=0): ! """Initialize the Formatter instance.""" - def format(self): - """Return a formatted string from the DOM tree.""" ! XXX This also needs a lot of work. What is required for this API? ! XXX How to handle unimplemented elements? ! A model 'Formatter' class implementing the full interface along ! with utility functions can be found in the 'dps.formatters.model' ! module. ! Language Module API ! =================== ! Language modules will contain language-dependent strings and ! mappings. They will be named for their language identifier (as ! defined in 'Choice of Docstring Format' above), converting dashes ! to underscores. - XXX Specifics to be determined. ! Intermediate Data Structure ! =========================== ! A single intermediate data structure is used by the docstring ! processing system, in the interfaces between parsers, the DPS ! itself, and formatters. It is not required that this data ! structure be used internally by any of the componentes. This data ! structure is similar to a DOM tree whose schema is documented in ! an XML DTD (eXtensible Markup Language Document Type Definition), ! which comes in three parts: ! - the Python Plaintext Document Interface DTD, ppdi.dtd [6], ! - the Generic Plaintext Document Interface DTD, gpdi.dtd [7], ! - and the OASIS Exchange Table Model, soextbl.dtd [8]. ! The DTD defines a rich set of elements, suitable for any input ! syntax or output format. The input parser and the output ! formatter share the same intermediate data structure. The ! processing system may do transformations on the data from the ! input parser before passing it on to the output formatter. The ! DTD retains all information necessary to reconstruct the original ! input text, or a reasonable facsimile thereof. ! XXX Specifics (about the DOM tree) to be determined. ! Output Management ! ================= ! XXX To be determined. - Type of output: filesystem only, or in-memory data structure too? - File/directory naming & structure conventions. In-memory data - structure should follow filesystem naming; file/directory == - leaf/node. Use a directory hierarchy rather than long file names. - (The files generated by pythondoc used compound file names, like - 'packagename.modulename.classname.html', which were often too long - for the 38-character MacOS file name length limit. This is one of - the reasons pythondoc couldn't run on MacOS). Error Handling --- 228,578 ---- exists; RFC 1766 is currently being revised to allow 3-letter codes). If no language identifier is specified, the default is ! "en" for English. The language identifier is passed to the parser and can be used for language-dependent markup features. ! Docutils Project Model ! ====================== ! :: ! +--------------------------+ ! | Docutils: | ! | docutils.core.Publisher, | ! | docutils.core.publish() | ! +--------------------------+ ! / \ ! / \ ! 1,3,5 / \ 6,8 ! +--------+ +--------+ ! | READER | =======================> | WRITER | ! +--------+ +--------+ ! // \ / \ ! // \ / \ ! 2 // 4 \ 7 / 9 \ ! +--------+ +------------+ +------------+ +--------------+ ! | PARSER |...| reader | | writer |...| DISTRIBUTOR? | ! +--------+ | transforms | | transforms | | | ! | | | | | - one file | ! | - docinfo | | - styling | | - many files | ! | - titles | | - writer- | | - objects in | ! | - linking | | specific | | memory | ! | - lookups | | - etc. | +--------------+ ! | - reader- | +------------+ ! | specific | ! | - parser- | ! | specific | ! | - layout | ! | - etc. | ! +------------+ ! The numbers indicate the path a document would take through the ! code. Double-width lines between reader & parser and between ! reader & writer, indicating that data sent along these paths ! should be standard (pure & unextended) DPS doc trees. ! Single-width lines signify that internal tree extensions or ! completely unrelated representations are possible, but they must ! be supported internally at both ends. ! Publisher ! --------- ! The "dps.core" module contains a "Publisher" facade class and ! "publish" convenience function. Publisher encapsulates the ! high-level logic of a Docutils system. The Publisher.publish() ! method passes its input to its Reader, then passes the resulting ! document tree through its Writer to its destination. ! Readers ! ------- ! Readers understand the input context (where the data is coming ! from), send the whole input or discrete "chunks" to the parser, ! and provide the context to bind the chunks together back into a ! cohesive whole. Using transforms_, Readers also resolve ! references, footnote numbers, interpreted text processing, and ! anything else that requires context-sensitive computation. ! Each reader is a module or package exporting a "Reader" class with ! a "read" method. The base "Reader" class can be found in the ! dps/readers/__init__.py module. ! Most Readers will have to be told what parser to use. So far (see ! the list of examples below), only the Python Source Reader ! (PySource) will be able to determine the syntax on its own. ! Responsibilities: ! - Do raw input on the source ("Reader.scan()"). ! - Pass the raw text to the parser, along with a fresh doctree ! root ("Reader.parse()"). ! - Run transforms over the doctree(s) ("Reader.transform()"). ! Examples: ! - Standalone/Raw/Plain: Just read a text file and process it. The ! reader needs to be told which parser to use. Parser-specific ! readers? ! - Python Source: See `Python Source Reader`_ above. ! - Email: RFC-822 headers, quoted excerpts, signatures, MIME parts. ! - PEP: RFC-822 headers, "PEP xxxx" and "RFC xxxx" conversion to ! URIs. Either interpret PEPs' indented sections or convert existing ! PEPs to reStructuredText (or both?). ! - Wiki: Global reference lookups of "wiki links" incorporated into ! transforms. (CamelCase only or unrestricted?) Lazy indentation? ! - Web Page: As standalone, but recognize meta fields as meta tags. ! Support for templates of some sort? (After <body>, before </body>?) ! - FAQ: Structured "question & answer(s)" constructs. ! - Compound document: Merge chapters into a book. Master TOC file? ! Parsers ! ------- ! Parsers analyze their input and produce a Docutils `document ! tree`_. They don't know or care anything about the source or ! destination of the data. ! Each input parser is a module or package exporting a "Parser" ! class with a "parse" method. The base "Parser" class can be found ! in the dps/parsers/__init__.py module. ! Responsibilities: Given raw input text and a doctree root node, ! populate the doctree by parsing the input text. ! Example: The only parser implemented so far is for the ! reStructuredText markup. ! Transforms ! ---------- ! Transforms change the document tree from one form to another, add ! to the tree, or prune it. Transforms are run by Reader and Writer ! objects. Some transforms are Reader-specific, some are ! Parser-specific, and others are Writer-specific. The choice and ! order of transforms is specified in the Reader and Writer objects. ! Each transform is a class in a module in the dps/transforms ! package, a subclass of dps.tranforms.Transform. ! Responsibilities: ! - Modify a doctree in-place, either purely transforming one ! structure into another, or adding new structures based on the ! doctree and/or external data. ! Examples (in "dps.transforms"): ! - frontmatter.DocInfo: conversion of document metadata ! (bibliographic information). ! - references.Hyperlinks: resolution of hyperlinks. ! - document.Merger: combining multiple populated doctrees into one. ! ! ! Writers ! ------- ! ! Writers produce the final output (HTML, XML, TeX, etc.). Writers ! translate the internal document tree structure into the final data ! format, possibly running output-specific transforms_ first. ! ! Each writer is a module or package exporting a "Writer" class with ! a "write" method. The base "Writer" class can be found in the ! dps/writers/__init__.py module. ! ! Responsibilities: ! ! - Run transforms over the doctree(s). ! ! - Translate doctree(s) into specific output formats. ! ! - Transform references into format-native forms. ! ! - Write output to the destination (possibly via a "Distributor"). ! ! Examples: ! ! - XML: Various forms, such as DocBook. Also, raw doctree XML. ! ! - HTML ! ! - TeX ! ! - Plain text ! ! - reStructuredText? ! ! ! Distributors ! ------------ ! ! Distributors exist for each method of storing the results of ! processing: ! ! - In a single file on disk. ! ! - In a tree of directories and files on disk. ! ! - In a single tree-shaped data structure in memory. ! ! - Some other set of data structures in memory. ! ! @@@ Distributors are currently just an idea; they may or may not ! be practical. Issues: ! ! Is it better for the writer to control the distributor, or ! vice versa? Or should they be equals? ! ! Looking at the list of writers, it seems that only HTML would ! require anything other than monolithic output. Perhaps merge ! the HTML "distributor" into "writer" variants? ! ! Perhaps translator/writer instead of writer/distributor? ! ! Responsibilities: ! ! - Do raw output to the destination. ! ! - Transform references per incarnation (method of distribution). ! ! Examples: ! ! - Single file. ! ! - Multiple files & directories. ! ! - Objects in memory. ! ! ! DPS Package Structure ! ========================== ! ! - Package "dps". ! ! - Module "dps.core" contains facade class "Publisher" and ! convenience function "publish()". See `Publisher API`_ below. ! ! - Module "dps.nodes" contains the Docutils document tree element ! class library plus Visitor pattern base classes. See ! `Document Tree`_ below. ! ! - Module "dps.roman" contains Roman numeral conversion ! routines. ! ! - Module "dps.statemachine" contains a finite state machine ! specialized for regular-expression-based text filters. The ! reStructuredText parser implementation is based on this ! module. ! ! - Module "dps.urischemes" contains a mapping of known URI ! schemes ("http", "ftp", "mail", etc.). ! ! - Module "dps.utils" contains utility functions and classes, ! including a logger class ("Reporter"; see `Error Handling`_ ! below). ! ! - Package "dps.parsers": markup parsers_. ! ! - Function "get_parser_class(parsername)" returns a parser ! module by name. Class "Parser" is the base class of ! specific parsers. (dps/parsers/__init__.py) ! ! - Package "dps.parsers.restructuredtext": the reStructuredText ! parser. ! ! - Alternate markup parsers may be added. ! ! - Package "dps.readers": context-aware input readers. ! ! - Function "get_reader_class(readername)" returns a reader ! module by name or alias. Class "Reader" is the base class ! of specific readers. (dps/readers/__init__.py) ! ! - Module "dps.readers.standalone": reads independent document ! files. ! ! - Readers to be added for: Python source code (structure & ! docstrings), PEPs, email, FAQ, and perhaps Wiki and others. ! ! - Package "dps.writers": output format writers. ! ! - Function "get_writer_class(writername)" returns a writer ! module by name. Class "Writer" is the base class of ! specific writers. (dps/writers/__init__.py) ! ! - Module "dps.writers.pprint" is a simple internal document ! tree writer; it writes indented pseudo-XML. ! ! - Module "dps.writers.html" is a simple HyperText Markup ! Language document tree writer for HTML 4.01 and CSS1. ! ! @@@ Change name to html4_css1.py? Support aliases? ! ! - Writers to be added: HTML 3.2 or 4.01-loose, XML (various ! forms, such as DocBook and the raw internal doctree), TeX, ! plaintext, reStructuredText, and perhaps others. ! ! - Package "dps.transforms": tree transform classes. ! ! - Class "Transform" is the base class of specific transforms; ! see `Transform API`_ below. (dps/transforms/__init__.py) ! ! - Each module contains related transform classes. ! ! - Package "dps.languages": Language modules contain ! language-dependent strings and mappings. They are named for ! their language identifier (as defined in `Choice of Docstring ! Format`_ above), converting dashes to underscores. ! ! - Function "getlanguage(languagecode)", returns matching ! language module. (dps/languages/__init__.py) ! ! - Module "dps.languages.en" (English). ! ! - Other languages to be added. ! ! ! Front-End Tools ! =============== ! ! @@@ To be determined. ! ! @@@ Document tools & summarize their command-line interfaces. ! ! ! Document Tree ! ============= ! ! A single intermediate data structure is used internally by the ! DPS, in the interfaces between components; it is defined in the ! dps.nodes module. It is not required that this data structure be ! used *internally* by any of the components, just *between* ! components. This data structure is similar to a DOM tree whose ! schema is documented in an XML DTD (eXtensible Markup Language ! Document Type Definition), which comes in two parts: ! ! - the Generic Plaintext Document Interface DTD, gpdi.dtd [6], and ! ! - the OASIS Exchange Table Model, soextbl.dtd [7]. ! ! The DTD defines a rich set of elements, suitable for many input ! and output formats. The DTD retains all information necessary to ! reconstruct the original input text, or a reasonable facsimile ! thereof. Error Handling *************** *** 349,353 **** When the parser encounters an error in markup, it inserts a system ! message (DTD element 'system_message). There are five levels of system messages: --- 580,584 ---- When the parser encounters an error in markup, it inserts a system ! message (DTD element "system_message"). There are five levels of system messages: *************** *** 356,370 **** handled separately from the others. ! - Level-1, "INFO": a minor issue that can be ignored. There is no ! effect on the processing. Typically level-1 system messages are ! not reported. - Level-2, "WARNING": an issue that should be addressed. If ignored, there may be unpredictable problems with the output. ! - Level-3, "ERROR": an error that should be addressed. If ! ignored, the output will contain errors. ! - Level-4, "SEVERE": a severe error that must be addressed. Typically level-4 system messages are turned into exceptions which halt processing. If ignored, the output will contain --- 587,604 ---- handled separately from the others. ! - Level-1, "INFO": a minor issue that can be ignored. There is ! little or no effect on the processing. Typically level-1 system ! messages are not reported. - Level-2, "WARNING": an issue that should be addressed. If ignored, there may be unpredictable problems with the output. + Typically level-2 system messages are reported but do not halt + processing ! - Level-3, "ERROR": a major issue that should be addressed. If ! ignored, the output will contain errors. Typically level-3 ! system messages are reported but do not halt processing ! - Level-4, "SEVERE": a critical error that must be addressed. Typically level-4 system messages are turned into exceptions which halt processing. If ignored, the output will contain *************** *** 373,379 **** Although the initial message levels were devised independently, they have a strong correspondence to VMS error condition severity ! levels [9]; the names in quotes for levels 1 through 4 were borrowed from VMS. Error handling has since been influenced by ! the log4j project [10]. --- 607,613 ---- Although the initial message levels were devised independently, they have a strong correspondence to VMS error condition severity ! levels [8]; the names in quotes for levels 1 through 4 were borrowed from VMS. Error handling has since been influenced by ! the log4j project [9]. *************** *** 381,391 **** [1] PEP 256, Docstring Processing System Framework, Goodger ! http://www.python.org/peps/pep-0256.html [2] PEP 224, Attribute Docstrings, Lemburg ! http://www.python.org/peps/pep-0224.html ! [3] PEP 257, Docstring Conventions, Goodger, Van Rossum ! http://www.python.org/peps/pep-0257.html [4] http://www.rfc-editor.org/rfc/rfc1766.txt --- 615,625 ---- [1] PEP 256, Docstring Processing System Framework, Goodger ! http://www.python.org/peps/pep-0256.html [2] PEP 224, Attribute Docstrings, Lemburg ! http://www.python.org/peps/pep-0224.html ! [3] PEP 216, Docstring Format, Zadka ! http://www.python.org/peps/pep-0216.html [4] http://www.rfc-editor.org/rfc/rfc1766.txt *************** *** 393,408 **** [5] http://lcweb.loc.gov/standards/iso639-2/englangn.html ! [6] http://docstring.sourceforge.net/spec/ppdi.dtd ! ! [7] http://docstring.sourceforge.net/spec/gpdi.dtd ! [8] http://docstring.sourceforge.net/spec/soextblx.dtd ! [9] http://www.openvms.compaq.com:8000/73final/5841/ 5841pro_027.html#error_cond_severity ! [10] http://jakarta.apache.org/log4j/ ! [11] http://www.python.org/sigs/doc-sig/ --- 627,640 ---- [5] http://lcweb.loc.gov/standards/iso639-2/englangn.html ! [6] http://docstring.sourceforge.net/spec/gpdi.dtd ! [7] http://docstring.sourceforge.net/spec/soextblx.dtd ! [8] http://www.openvms.compaq.com:8000/73final/5841/ 5841pro_027.html#error_cond_severity ! [9] http://jakarta.apache.org/log4j/ ! [10] http://www.python.org/sigs/doc-sig/ *************** *** 421,425 **** This document borrows ideas from the archives of the Python ! Doc-SIG [11]. Thanks to all members past & present. --- 653,657 ---- This document borrows ideas from the archives of the Python ! Doc-SIG [10]. Thanks to all members past & present. |
From: David G. <go...@us...> - 2002-03-28 04:36:13
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv13589/dps/spec Modified Files: pep-0256.txt Log Message: Overhauled. Index: pep-0256.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/pep-0256.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** pep-0256.txt 4 Aug 2001 15:44:49 -0000 1.2 --- pep-0256.txt 28 Mar 2002 04:36:11 -0000 1.3 *************** *** 3,7 **** Version: $Revision$ Last-Modified: $Date$ ! Author: dgo...@bi... (David Goodger) Discussions-To: do...@py... Status: Draft --- 3,7 ---- Version: $Revision$ Last-Modified: $Date$ ! Author: go...@us... (David Goodger) Discussions-To: do...@py... Status: Draft *************** *** 13,38 **** Abstract ! Python modules, functions, classes and methods have a string ! attribute called __doc__. If the first expression inside their ! definition is a literal string, that string is assigned to the ! __doc__ attribute, called a documentation string or docstring. It ! is often used to summarize the interface of the module, function, ! class or method. - There is no standard format (markup) for docstrings, nor are there - standard tools for extracting docstrings and transforming them - into useful structured formats (e.g., HTML, DocBook, TeX). Those - tools that do exist are for the most part unmaintained and unused. The issues surrounding docstring processing have been contentious ! and difficult to resolve. ! ! This PEP proposes a Docstring Processing System (DPS) framework. ! It separates out the components (program and conceptual), enabling ! the resolution of individual issues either through consensus (one ! solution) or through divergence (many). It promotes standard ! interfaces which will allow a variety of plug-in components (input ! parsers and output formatters) to be used. ! This PEP presents the concepts of a DPS framework independently of implementation details. --- 13,33 ---- Abstract ! Python lends itself to inline documentation. With its built-in ! docstring syntax, a limited form of Literate Programming [1]_ is ! easy to do in Python. However, there are no satisfactory standard ! tools for extracting and processing Python docstrings. The lack ! of a standard toolset is a significant gap in Python's ! infrastructure; this PEP aims to fill the gap. The issues surrounding docstring processing have been contentious ! and difficult to resolve. This PEP proposes a generic Docstring ! Processing System (DPS) framework, which separates out the ! components (program and conceptual), enabling the resolution of ! individual issues either through consensus (one solution) or ! through divergence (many). It promotes standard interfaces which ! will allow a variety of plug-in components (input context readers, ! markup parsers, and output format writers) to be used. ! The concepts of a DPS framework are presented independently of implementation details. *************** *** 40,167 **** Rationale - Python lends itself to inline documentation. With its built-in - docstring syntax, a limited form of Literate Programming [1] is - easy to do in Python. However, there are no satisfactory standard - tools for extracting and processing Python docstrings. The lack - of a standard toolset is a significant gap in Python's - infrastructure; this PEP aims to fill the gap. - There are standard inline documentation systems for some other ! languages. For example, Perl has POD (plain old documentation) ! and Java has Javadoc, but neither of these mesh with the Pythonic ! way. POD is very explicit, but takes after Perl in terms of ! readability. Javadoc is HTML-centric; except for '@field' tags, ! raw HTML is used for markup. There are also general tools such as ! Autoduck and Web (Tangle & Weave), useful for multiple languages. ! There have been many attempts to write autodocumentation systems for Python (not an exhaustive list): ! - Marc-Andre Lemburg's doc.py [2] ! - Daniel Larsson's pythondoc & gendoc [3] ! - Doug Hellmann's HappyDoc [4] ! - Laurence Tratt's Crystal [5] ! - Ka-Ping Yee's htmldoc & pydoc [6] (pydoc.py is now part of the Python standard library; see below) ! - Tony Ibbs' docutils [7] ! - Edward Loper's STminus formalization and related efforts [8] These systems, each with different goals, have had varying degrees of success. A problem with many of the above systems was ! over-ambition. They provided a self-contained set of components: ! a docstring extraction system, an input parser, an internal ! processing system and one or more output formatters. Inevitably, ! one or more components had serious shortcomings, preventing the ! system from being adopted as a standard tool. ! ! Throughout the existence of the Python Documentation Special ! Interest Group (Doc-SIG) [9], consensus on a single standard ! docstring format has never been reached. A lightweight, implicit ! markup has been sought, for the following reasons (among others): ! ! 1. Docstrings written within Python code are available from within ! the interactive interpreter, and can be 'print'ed. Thus the ! use of plaintext for easy readability. ! ! 2. Programmers want to add structure to their docstrings, without ! sacrificing raw docstring readability. Unadorned plaintext ! cannot be transformed ('up-translated') into useful structured ! formats. ! ! 3. Explicit markup (like XML or TeX) has been widely considered ! unreadable by the uninitiated. ! ! 4. Implicit markup is aesthetically compatible with the clean and ! minimalist Python syntax. ! ! Early on, variants of Setext (Structure Enhanced Text) [10], ! including Digital Creation's StructuredText [11], were proposed ! for Python docstring formatting. Hereafter we will collectively ! call these variants 'STexts'. Although used by some (including in ! most of the above-listed autodocumentation tools), these markup ! schemes have failed to become standard because: ! ! - STexts have been incomplete: lacking 'essential' constructs that ! people want to use in their docstrings, STexts are rendered less ! than ideal. Note that these 'essential' constructs are not ! universal; everyone has their own requirements. ! ! - STexts have been sometimes surprising: bits of text are marked ! up unexpectedly, leading to user frustration. ! ! - SText implementations have been buggy. ! ! - Some STexts have have had no formal specification except for the ! implementation itself. A buggy implementation meant a buggy ! spec, and vice-versa. ! ! - There has been no mechanism to get around the SText markup rules ! when a markup character is used in a non-markup context. ! ! Recognizing the deficiencies of STexts, some people have proposed ! using explicit markup of some kind. There have been proposals for ! using XML, HTML, TeX, POD, and Javadoc at one time or another. ! Proponents of STexts have vigorously opposed these proposals, and ! the debates have continued off and on for at least five years. It has become clear (to this author, at least) that the "all or ! nothing" approach cannot succeed, since no all-encompassing ! proposal could possibly be agreed upon by all interested parties. ! A modular component approach, where components may be multiply ! implemented, is the only chance at success. By separating out the ! issues, we can form consensus more easily (smaller fights ;-), and ! accept divergence more readily. Each of the components of a docstring processing system should be developed independently. A 'best of breed' system should be ! chosen and/or developed and eventually included in Python's ! standard library. ! Pydoc & Other Existing Systems ! Pydoc is part of the Python 2.1 standard library. It extracts and ! displays docstrings from within the Python interactive ! interpreter, from the shell command line, and from a GUI window ! into a web browser (HTML). In the case of GUI/HTML, except for ! some heuristic hyperlinking of identifier names, no formatting of ! the docstrings is done. They are presented within <p><small><tt> ! tags to avoid unwanted line wrapping. Unfortunately, the result ! is not pretty. The functionality proposed in this PEP could be added to or used ! by pydoc when serving HTML pages. However, the proposed docstring ! processing system's functionality is much more than pydoc needs ! (in its current form). Either an independent tool will be ! developed (which pydoc may or may not use), or pydoc could be ! expanded to encompass this functionality and *become* the ! docstring processing system (or one such system). That decision ! is beyond the scope of this PEP. Similarly for other existing docstring processing systems, their --- 35,119 ---- Rationale There are standard inline documentation systems for some other ! languages. For example, Perl has POD [2]_ and Java has Javadoc ! [3]_, but neither of these mesh with the Pythonic way. POD syntax ! is very explicit, but takes after Perl in terms of readability. ! Javadoc is HTML-centric; except for '@field' tags, raw HTML is ! used for markup. There are also general tools such as Autoduck ! [4]_ and Web (Tangle & Weave) [5]_, useful for multiple languages. ! There have been many attempts to write auto-documentation systems for Python (not an exhaustive list): ! - Marc-Andre Lemburg's doc.py [6]_ ! - Daniel Larsson's pythondoc & gendoc [7]_ ! - Doug Hellmann's HappyDoc [8]_ ! - Laurence Tratt's Crystal [9]_ ! - Ka-Ping Yee's htmldoc & pydoc [10]_ (pydoc.py is now part of the Python standard library; see below) ! - Tony Ibbs' docutils [11]_ ! - Edward Loper's STminus formalization and related efforts [12]_ These systems, each with different goals, have had varying degrees of success. A problem with many of the above systems was ! over-ambition combined with inflexibility. They provided a ! self-contained set of components: a docstring extraction system, ! a markup parser, an internal processing system and one or more ! output format writers. Inevitably, one or more aspects of each ! system had serious shortcomings, and they were not easily extended ! or modified, preventing them from being adopted as standard tools. It has become clear (to this author, at least) that the "all or ! nothing" approach cannot succeed, since no monolithic ! self-contained system could possibly be agreed upon by all ! interested parties. A modular component approach designed for ! extension, where components may be multiply implemented, may be ! the only chance for success. By separating out the issues, we can ! form consensus more easily (smaller fights ;-), and accept ! divergence more readily. Each of the components of a docstring processing system should be developed independently. A 'best of breed' system should be ! chosen, either merged from existing systems, and/or developed ! anew. This system should be included in Python's standard ! library. ! PyDoc & Other Existing Systems ! PyDoc became part of the Python standard library as of release ! 2.1. It extracts and displays docstrings from within the Python ! interactive interpreter, from the shell command line, and from a ! GUI window into a web browser (HTML). Although a very useful ! tool, PyDoc has several deficiencies, including: ! ! - In the case of the GUI/HTML, except for some heuristic ! hyperlinking of identifier names, no formatting of the ! docstrings is done. They are presented within <p><small><tt> ! tags to avoid unwanted line wrapping. Unfortunately, the result ! is not attractive. ! ! - PyDoc extracts docstrings and structural information (class ! identifiers, method signatures, etc.) from imported module ! objects. There are security issues involved with importing ! untrusted code. Also, information from the source is lost when ! importing, such as comments, "additional docstrings" (string ! literals in non-docstring contexts; see PEP 258 [13]_), and the ! order of definitions. The functionality proposed in this PEP could be added to or used ! by PyDoc when serving HTML pages. The proposed docstring ! processing system's functionality is much more than PyDoc needs in ! its current form. Either an independent tool will be developed ! (which PyDoc may or may not use), or PyDoc could be expanded to ! encompass this functionality and *become* the docstring processing ! system (or one such system). That decision is beyond the scope of ! this PEP. Similarly for other existing docstring processing systems, their *************** *** 183,190 **** - First line is a one-line synopsis. ! PEP 257, Docstring Conventions [12], documents these issues. ! 2. Docstring processing system generic implementation details. ! Documents issues such as: - High-level spec: what a DPS does. --- 135,143 ---- - First line is a one-line synopsis. ! PEP 257, Docstring Conventions [14]_, documents some of these ! issues. ! 2. Docstring processing system design specification. Documents ! issues such as: - High-level spec: what a DPS does. *************** *** 192,257 **** - Command-line interface for executable script. ! - System Python API - Docstring extraction rules. ! - Input parser API. ! - Intermediate internal data structure: output from input ! parser, input to output formatter. ! - Output formatter API. ! - Output management. These issues are applicable to any docstring processing system ! implementation. PEP 258, DPS Generic Implementation Details ! [13], documents these issues. 3. Docstring processing system implementation. ! 4. Input markup specifications: docstring syntax. 5. Input parser implementations. ! 6. Output formats (HTML, XML, TeX, DocBook, info, etc.). ! ! 7. Output formatter implementations. ! ! Components 1, 2, and 3 will be the subject of individual companion ! PEPs, although they may be merged into this PEP once consensus is ! reached. If there is only one implementation, PEPs for components ! 2 & 3 can be combined. Multiple PEPs will be necessary for each ! of components 4, 5, 6, and 7. An alternative to the PEP mechanism ! may be used instead, since these are not directly related to the ! Python language. ! ! The following diagram shows an overview of the framework. ! Interfaces are indicated by double-borders. The ASCII diagram is ! very wide; please turn off line wrapping to view it: ! +========================+ ! | Command-Line Interface | ! +========================+ ! | Executable Script | ! +------------------------+ ! | ! | calls ! v ! +===========================================+ returns +---------+ ! | System Python API |==========>| output | ! +--------+ +===========================================+ | objects | ! _ writes | Python | reads | Docstring Processing System | +---------+ ! / \ ==============>| module |<===========| | ! \_/ +--------+ | input | transformation, | output | +--------+ ! | +-------------+ follows | docstring | integration, | object | writes | output | ! --+-- consults | docstring |<-----------| extraction | linking | management |===========>| files | ! | --------->| conventions | +============+=====+=====+=====+============+ +--------+ ! / \ +-------------+ | parser API | | formatter API | ! / \ +-------------+ +===========+======+ +======+===========+ +--------+ ! author consults | markup | implements | input | intermediate | output | implements | output | ! --------->| syntax spec |<-----------| parser | data structure | formatter |----------->| format | ! +-------------+ +-----------+-------------------+-----------+ +--------+ --- 145,190 ---- - Command-line interface for executable script. ! - System Python API. - Docstring extraction rules. ! - Readers, which encapsulate the input context . ! - Parsers. ! - Document tree: the intermediate internal data structure. The ! output of the Parser and Reader, and the input to the Writer ! all share the same data structure. ! - Transforms, which modify the document tree. ! ! - Writers for output formats. ! ! - Distributors, which handle output management (one file, many ! files, or objects in memory). These issues are applicable to any docstring processing system ! implementation. PEP 258, Docutils Design Specification [13 ]_, ! documents these issues. 3. Docstring processing system implementation. ! 4. Input markup specifications: docstring syntax. PEP 2xx, ! reStructuredText Standard Docstring Format [15]_, proposes a ! standard syntax. 5. Input parser implementations. ! 6. Input context readers ("modes": Python source code, PEP, ! standalone text file, email, etc.) and implementations. + 7. Output formats (HTML, XML, TeX, DocBook, info, etc.) and writer + implementations. ! Components 1, 2/3, and 4/5 are the subject of individual companion ! PEPs. If there is another implementation of the framework or ! syntax/parser, additional PEPs may be required. Multiple ! implementations of each of components 6 and 7 will be required; ! the PEP mechanism may be overkill for these components. *************** *** 259,263 **** A SourceForge project has been set up for this work at ! http://docstring.sf.net. --- 192,196 ---- A SourceForge project has been set up for this work at ! http://docstring.sourceforge.net/. *************** *** 266,298 **** [1] http://www.literateprogramming.com/ ! [2] http://www.lemburg.com/files/python/SoftwareDescriptions.html#doc.py ! [3] http://starship.python.net/crew/danilo/pythondoc/ ! [4] http://happydoc.sf.net/ ! [5] http://www.btinternet.com/~tratt/comp/python/crystal/index.html ! [6] http://www.lfw.org/python/ ! [7] http://homepage.ntlworld.com/tibsnjoan/docutils/ ! [8] http://www.cis.upenn.edu/~edloper/pydoc/ ! [9] http://www.python.org/sigs/doc-sig/ ! [10] http://www.bsdi.com/setext/ ! [11] http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage/ ! [12] PEP 257, Docstring Conventions, Goodger, Van Rossum ! http://www.python.org/peps/pep-0257.html ! [13] PEP 258, DPS Generic Implementation Details, Goodger http://www.python.org/peps/pep-0258.html ! [14] PEP 216, Docstring Format, Zadka ! http://www.python.org/peps/pep-0216.html --- 199,235 ---- [1] http://www.literateprogramming.com/ ! [2] Perl "Plain Old Documentation" ! http://www.perldoc.com/perl5.6/pod/perlpod.html ! [3] http://java.sun.com/j2se/javadoc/ ! [4] http://www.helpmaster.com/hlp-developmentaids-autoduck.htm ! [5] http://www-cs-faculty.stanford.edu/~knuth/cweb.html ! [6] http://www.lemburg.com/files/python/SoftwareDescriptions.html#doc.py ! [7] http://starship.python.net/crew/danilo/pythondoc/ ! [8] http://happydoc.sourceforge.net/ ! [9] http://www.btinternet.com/~tratt/comp/python/crystal/ ! [10] http://www.python.org/doc/current/lib/module-pydoc.html ! [11] http://homepage.ntlworld.com/tibsnjoan/docutils/ ! [12] http://www.cis.upenn.edu/~edloper/pydoc/ ! [13] PEP 258, Docutils Design Specification, Goodger http://www.python.org/peps/pep-0258.html ! [14] PEP 257, Docstring Conventions, Goodger, Van Rossum ! http://www.python.org/peps/pep-0257.html + [15] PEP 2xx, reStructuredText Standard Docstring Format, Goodger + http://www.python.org/peps/pep-02xx.html + + [16] http://www.python.org/sigs/doc-sig/ *************** *** 304,313 **** Acknowledgements ! This document borrows text from PEP 216, Docstring Format, by ! Moshe Zadka [14]. It is intended as a reorganization of PEP 216 ! and its approach. ! ! This document also borrows ideas from the archives of the Python ! Doc-SIG. Thanks to all members past & present. --- 241,246 ---- Acknowledgements ! This document borrows ideas from the archives of the Python ! Doc-SIG [16]_. Thanks to all members past & present. |
From: David G. <go...@us...> - 2002-03-28 04:35:20
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv13077/dps/spec Modified Files: pep-0257.txt Log Message: updated Index: pep-0257.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/pep-0257.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -C2 -d -r1.2 -r1.3 *** pep-0257.txt 4 Aug 2001 15:45:43 -0000 1.2 --- pep-0257.txt 28 Mar 2002 04:35:18 -0000 1.3 *************** *** 3,10 **** Version: $Revision$ Last-Modified: $Date$ ! Author: dgo...@bi... (David Goodger), ! gu...@di... (Guido van Rossum) Discussions-To: do...@py... ! Status: Draft Type: Informational Created: 29-May-2001 --- 3,10 ---- Version: $Revision$ Last-Modified: $Date$ ! Author: go...@us... (David Goodger), ! gu...@py... (Guido van Rossum) Discussions-To: do...@py... ! Status: Active Type: Informational Created: 29-May-2001 *************** *** 33,39 **** If you violate the conventions, the worst you'll get is some dirty ! looks. But some software (such as the Docstring Processing System ! [1]) will be aware of the conventions, so following them will get ! you the best results. --- 33,39 ---- If you violate the conventions, the worst you'll get is some dirty ! looks. But some software (such as the Docutils docstring ! processing system [1] [2]) will be aware of the conventions, so ! following them will get you the best results. *************** *** 54,61 **** String literals occurring elsewhere in Python code may also act as ! documentation. They are not recognized by the Python bytecode compiler and are not accessible as runtime object attributes (i.e. not assigned to __doc__), but two types of extra docstrings ! are extracted by software tools: 1. String literals occurring immediately after a simple assignment --- 54,61 ---- String literals occurring elsewhere in Python code may also act as ! documentation. They are not recognized by the Python bytecode compiler and are not accessible as runtime object attributes (i.e. not assigned to __doc__), but two types of extra docstrings ! may be extracted by software tools: 1. String literals occurring immediately after a simple assignment *************** *** 66,72 **** are called "additional docstrings". ! Please see PEP 258 "DPS Generic Implementation Details" [2] for a detailed description of attribute and additional docstrings. For consistency, always use """triple double quotes""" around docstrings. Use r"""raw triple double quotes""" if you use any --- 66,74 ---- are called "additional docstrings". ! Please see PEP 258 "Docutils Design Specification" [2] for a detailed description of attribute and additional docstrings. + XXX Mention docstrings of 2.2 properties. + For consistency, always use """triple double quotes""" around docstrings. Use r"""raw triple double quotes""" if you use any *************** *** 89,93 **** ... ! Notes: - Triple quotes are used even though the string fits on one line. --- 91,95 ---- ... ! Notes: - Triple quotes are used even though the string fits on one line. *************** *** 107,114 **** the function/method parameters (which can be obtained by introspection). Don't do:: ! def function(a, b): """function(a, b) -> list""" ! This type of docstring is only appropriate for C functions (such as built-ins), where introspection is not possible. --- 109,116 ---- the function/method parameters (which can be obtained by introspection). Don't do:: ! def function(a, b): """function(a, b) -> list""" ! This type of docstring is only appropriate for C functions (such as built-ins), where introspection is not possible. *************** *** 121,125 **** elaborate description. The summary line may be used by automatic indexing tools; it is important that it fits on one line and is ! separated from the rest of the docstring by a blank line. The entire docstring is indented the same as the quotes at its --- 123,129 ---- elaborate description. The summary line may be used by automatic indexing tools; it is important that it fits on one line and is ! separated from the rest of the docstring by a blank line. The ! summary line may be on the same line as the opening quotes or on ! the next line. The entire docstring is indented the same as the quotes at its *************** *** 191,199 **** def complex(real=0.0, imag=0.0): """Form a complex number. ! Keyword arguments: real -- the real part (default 0.0) imag -- the imaginary part (default 0.0) ! """ if imag == 0.0 and real == 0.0: return complex_zero --- 195,203 ---- def complex(real=0.0, imag=0.0): """Form a complex number. ! Keyword arguments: real -- the real part (default 0.0) imag -- the imaginary part (default 0.0) ! """ if imag == 0.0 and real == 0.0: return complex_zero *************** *** 211,215 **** http://www.python.org/peps/pep-0256.html ! [2] PEP 258, DPS Generic Implementation Details, Goodger http://www.python.org/peps/pep-0258.html --- 215,219 ---- http://www.python.org/peps/pep-0256.html ! [2] PEP 258, Docutils Design Specification, Goodger http://www.python.org/peps/pep-0258.html *************** *** 217,222 **** Life. ! [4] PEP 8, Style Guide for Python Code, van Rossum, Warsaw ! http://www.python.org/peps/pep-0008.html [5] http://www.python.org/sigs/doc-sig/ --- 221,225 ---- Life. ! [4] http://www.python.org/doc/essays/styleguide.html [5] http://www.python.org/sigs/doc-sig/ *************** *** 231,235 **** The "Specification" text comes mostly verbatim from the Python ! Style Guide by Guido van Rossum [4]. This document borrows ideas from the archives of the Python --- 234,238 ---- The "Specification" text comes mostly verbatim from the Python ! Style Guide essay by Guido van Rossum [4]. This document borrows ideas from the archives of the Python |
From: David G. <go...@us...> - 2002-03-28 04:35:01
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv12802/dps/spec Modified Files: dps-notes.txt Log Message: updated Index: dps-notes.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/dps-notes.txt,v retrieving revision 1.32 retrieving revision 1.33 diff -C2 -d -r1.32 -r1.33 *** dps-notes.txt 16 Mar 2002 06:11:18 -0000 1.32 --- dps-notes.txt 28 Mar 2002 04:34:58 -0000 1.33 *************** *** 20,36 **** (indeed, is it still meant to be correct? [Yes, it is.]). ! - Rework PEP 257, separating style from spec from tools, wrt DPS. See Doc-SIG from 2001-06-19/20. - - PEP 256: - - - Incorporate "modes" (one or two sets). - - Draw the framework diagram properly as a graphic (once PEPs - support graphics !-). - - - PEP 258: - - - Mention 2.2-style "properties" with attribute docstrings. - - Document! --- 20,26 ---- (indeed, is it still meant to be correct? [Yes, it is.]). ! - Rework PEP 257, separating style from spec from tools, wrt DPS? See Doc-SIG from 2001-06-19/20. - Document! *************** *** 49,55 **** - Add validation? See http://pytrex.sourceforge.net, RELAX NG. - - Incorporate readers/"input modes", using Tony Ibbs' 2001-08-03 - Doc-SIG post 'Suggestions for reST "modes"' as a base. - - Write modules for common transforms. See `Unimplemented Transforms`_ below. --- 39,42 ---- *************** *** 88,101 **** SGML-ID-friendly id's and a one-to-one mapping for later lookup. - - Implement a "name mangling" scheme: name "section title" becomes - id "section_title", name "1" becomes id "footnote1" (or just - "1"?). - - - Whenever possible, use the "name mangling" scheme. When that's - not possible (duplicate names, no name), and use an arbitrary - sequential id. - - - ID as required. - - Remove dependency on names as sole distinguishing characteristic. Use IDs instead. --- 75,78 ---- *************** *** 111,115 **** non-existent mappings. In the Writer or in a transform? ! - Add "refids" attribute to all target types, for backlinks. - Considerations for an HTML Writer [#]_: --- 88,93 ---- non-existent mappings. In the Writer or in a transform? ! - Add support for "refids" attribute on footnotes & citations, for ! backlinks. - Considerations for an HTML Writer [#]_: *************** *** 298,478 **** the output formatter. The same intermediate data format would be used between each of these, being transformed as it progresses. - - - Docutils Project Model - ====================== - - Here's the latest project model:: - - +--------------------------+ - | Docutils: | - | docutils.core.Publisher, | - | docutils.core.publish() | - +--------------------------+ - / \ - / \ - 1,3,5 / \ 6,8 - +--------+ +--------+ - | READER | =======================> | WRITER | - +--------+ (purely presentational) +--------+ - // \ / \ - // \ / \ - 2 // 4 \ 7 / 9 \ - +--------+ +------------+ +------------+ +--------------+ - | PARSER |...| reader | | writer |...| DISTRIBUTOR | - +--------+ | transforms | | transforms | | | - | | | | | - one file | - | - docinfo | | - styling | | - many files | - | - titles | | - writer- | | - objects in | - | - linking | | specific | | memory | - | - lookups | | - etc. | +--------------+ - | - reader- | +------------+ - | specific | - | - parser- | - | specific | - | - layout | - | - etc. | - +------------+ - - The numbers indicate the path a document would take through the code. - Double-width lines between reader & parser and between reader & - writer, indicating that data sent along these paths should be standard - (pure & unextended) DPS doc trees. Single-width lines signify that - internal tree extensions are OK (but must be supported internally at - both ends), and may in fact be totally unrelated to the DPS doc tree - structure. I've added "reader-specific" and "layout" transforms to the - list of transforms. BTW, these transforms are not necessarily all in - one directory; it's a nebulous grouping (it's hard to draw ASCII - clouds). - - - Issues - ------ - - - Transforms. How to specify which transforms (and in what order) - apply to each combination of reader, parser/syntax, writer, and - distributor? Even if we restrict ourselves to one parser, there - will eventually be a multitude of readers, writers, and distributor - options. - - Or are readers & writers independent? Then we have reader/parser - and writer/distributor combinations to consider. - - - Components - ---------- - - Production -> Publication -> Distribution - - Producer -> Publisher -> Distributor - - - Parsers - ``````` - - Responsibilities: Given raw input text and an empty doctree, populate - the doctree by parsing the input text. - - - Readers - ``````` - - Most Readers will have to be told what parser to use. So far (see the - list of examples below), only the Python Source Reader (PySource) will - be able to determine the syntax on its own. - - Responsibilities: - - - Do raw input on the source. - - Pass the raw text to the parser, along with a fresh doctree. - - Combine and collate doctrees if necessary. - - Run transforms over the doctree(s). - - Examples: - - - Standalone/Raw/Plain: Just read a text file and process it. The - reader needs to be told which parser to use. Parser-specific - readers? - - Python Source: See `Python Source Reader`_ above. - - Email: RFC-822 headers, quoted excerpts, signatures, MIME parts. - - PEP: RFC-822 headers, "PEP xxxx" and "RFC xxxx" conversion to - URIs. Either interpret PEPs' indented sections or convert existing - PEPs to reStructuredText (or both?). - - Wiki: Global reference lookups of "wiki links" incorporated into - transforms. (CamelCase only or unrestricted?) Lazy indentation? - - Web Page: As standalone, but recognize meta fields as meta tags. - Support for templates of some sort? (After <body>, before </body>?) - - FAQ: Structured "question & answer(s)" constructs. - - Compound document: Merge chapters into a book. Master TOC file? - - - Transforms - `````````` - - Responsibilities: - - - Modify a doctree in-place. - - Examples: - - - Already implemented: DocInfo, DocTitle (in frontmatter.py); - Hyperlinks, Footnotes, Substitutions (in references.py). - - Unimplemented: See `Unimplemented Transforms`_ above. - - - Writers - ``````` - - Responsibilities: - - - Transform doctree into specific output formats. - - Transform references into format-native forms. - - Examples: - - - XML: Various forms, such as DocBook. Also, raw doctree XML. - - HTML - - TeX - - Plain text - - reStructuredText? - - - Distributors - ```````````` - - (Writer/distributor components may change places. After rearranging, - the model would look like this:: - - - 1,3,5 6,8 - +--------+ +-------------+ - | READER | =======================> | DISTRIBUTOR | - +--------+ (purely presentational) +-------------+ - // \ / \ - // \ / \ - 2 // 4 \ 7 / 9 \ - +--------+ +------------+ +------------+ +--------+ - | PARSER |...| reader | | writer |...| WRITER | - +--------+ | transforms | | transforms | +--------+ - | ... | | ... | - - We'll wait and see which arrangement works out best. Is it better for - the writer to control the distributor, or vice versa? Or should they - be equals? - - Looking at the list of writers, it seems that only HTML would require - anything other than monolithic output. Perhaps merge the HTML - "distributor" into "writer" variants?) - - Responsibilities: - - - Do raw output to the destination. - - Transform references per incarnation (type of distribution). - - Examples: - - - Single file - - Multiple files & directories - - Objects in memory --- 276,279 ---- |
From: David G. <go...@us...> - 2002-03-28 04:33:08
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv11189/dps/spec Modified Files: pysource-reader.txt Log Message: syntax fix Index: pysource-reader.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/pysource-reader.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -C2 -d -r1.1 -r1.2 *** pysource-reader.txt 18 Oct 2001 03:35:21 -0000 1.1 --- pysource-reader.txt 28 Mar 2002 04:33:05 -0000 1.2 *************** *** 70,75 **** identifier in either prefix or suffix form:: ! Use method:`Keeper.storedata` to store the object's data in the ! `Keeper.data`:instance_attribute. The role may be one of 'package', 'module', 'class', 'method', --- 70,75 ---- identifier in either prefix or suffix form:: ! Use :method:`Keeper.storedata` to store the object's data in ! `Keeper.data`:instance_attribute:. The role may be one of 'package', 'module', 'class', 'method', |
From: David G. <go...@us...> - 2002-03-28 04:32:25
|
Update of /cvsroot/docstring/dps/test/test_transforms In directory usw-pr-cvs1:/tmp/cvs-serv10686/dps/test/test_transforms Modified Files: test_hyperlinks.py Log Message: updated Index: test_hyperlinks.py =================================================================== RCS file: /cvsroot/docstring/dps/test/test_transforms/test_hyperlinks.py,v retrieving revision 1.7 retrieving revision 1.8 diff -C2 -d -r1.7 -r1.8 *** test_hyperlinks.py 16 Mar 2002 05:48:11 -0000 1.7 --- test_hyperlinks.py 28 Mar 2002 04:32:22 -0000 1.8 *************** *** 134,138 **** <system_message level="1" refid="id1" type="INFO"> <paragraph> ! Duplicate implicit target name: "implicit" <paragraph> <reference refname="implicit"> --- 134,138 ---- <system_message level="1" refid="id1" type="INFO"> <paragraph> ! Duplicate implicit target name: "implicit". <paragraph> <reference refname="implicit"> *************** *** 217,221 **** <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "ztarget" <target dupname="ztarget" id="id1"> <paragraph> --- 217,221 ---- <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "ztarget". <target dupname="ztarget" id="id1"> <paragraph> *************** *** 412,416 **** <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "target" <target dupname="target" id="id1" refuri="second"> """], --- 412,416 ---- <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "target". <target dupname="target" id="id1" refuri="second"> """], |
From: David G. <go...@us...> - 2002-03-28 04:32:09
|
Update of /cvsroot/docstring/dps/test/test_transforms In directory usw-pr-cvs1:/tmp/cvs-serv10491/dps/test/test_transforms Modified Files: test_footnotes.py Log Message: updated Index: test_footnotes.py =================================================================== RCS file: /cvsroot/docstring/dps/test/test_transforms/test_footnotes.py,v retrieving revision 1.11 retrieving revision 1.12 diff -C2 -d -r1.11 -r1.12 *** test_footnotes.py 16 Mar 2002 05:47:58 -0000 1.11 --- test_footnotes.py 28 Mar 2002 04:32:06 -0000 1.12 *************** *** 225,229 **** <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "five" <paragraph> Auto-numbered footnote 5 again (duplicate). --- 225,229 ---- <system_message level="2" refid="id1" type="WARNING"> <paragraph> ! Duplicate explicit target name: "five". <paragraph> Auto-numbered footnote 5 again (duplicate). *************** *** 374,378 **** <system_message level="2" refid="id3" type="WARNING"> <paragraph> ! Duplicate explicit target name: "five" <paragraph> Auto-numbered footnote 5 again (duplicate). --- 374,378 ---- <system_message level="2" refid="id3" type="WARNING"> <paragraph> ! Duplicate explicit target name: "five". <paragraph> Auto-numbered footnote 5 again (duplicate). |
From: David G. <go...@us...> - 2002-03-16 06:12:00
|
Update of /cvsroot/docstring/dps/test In directory usw-pr-cvs1:/tmp/cvs-serv29165/dps/test Modified Files: DPSTestSupport.py Log Message: transform API change Index: DPSTestSupport.py =================================================================== RCS file: /cvsroot/docstring/dps/test/DPSTestSupport.py,v retrieving revision 1.8 retrieving revision 1.9 diff -C2 -d -r1.8 -r1.9 *** DPSTestSupport.py 8 Mar 2002 04:30:36 -0000 1.8 --- DPSTestSupport.py 16 Mar 2002 06:11:58 -0000 1.9 *************** *** 225,229 **** self.parser.parse(self.input, doctree) for transformClass in (self.transforms + universal.test_transforms): ! transformClass().transform(doctree) output = doctree.pformat() self.compareOutput(self.input, output, self.expected) --- 225,229 ---- self.parser.parse(self.input, doctree) for transformClass in (self.transforms + universal.test_transforms): ! transformClass(doctree).transform() output = doctree.pformat() self.compareOutput(self.input, output, self.expected) *************** *** 241,245 **** print doctree.pformat() for transformClass in self.transforms: ! transformClass().transform(doctree) output = doctree.pformat() print '-' * 70 --- 241,245 ---- print doctree.pformat() for transformClass in self.transforms: ! transformClass(doctree).transform() output = doctree.pformat() print '-' * 70 |
From: David G. <go...@us...> - 2002-03-16 06:11:21
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv28911/dps/spec Modified Files: dps-notes.txt Log Message: updated Index: dps-notes.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/dps-notes.txt,v retrieving revision 1.31 retrieving revision 1.32 diff -C2 -d -r1.31 -r1.32 *** dps-notes.txt 13 Mar 2002 02:48:34 -0000 1.31 --- dps-notes.txt 16 Mar 2002 06:11:18 -0000 1.32 *************** *** 5,8 **** --- 5,9 ---- :Revision: $Revision$ + .. contents:: To Do *************** *** 100,104 **** Use IDs instead. ! - Perhaps get rid of "name" attributes altoghether? - Perhaps keep a name->id mapping file? This could be stored --- 101,106 ---- Use IDs instead. ! - Perhaps get rid of "name" attributes altoghether? Certainly get ! rid of the "_:1:_" abominations. - Perhaps keep a name->id mapping file? This could be stored *************** *** 109,112 **** --- 111,116 ---- non-existent mappings. In the Writer or in a transform? + - Add "refids" attribute to all target types, for backlinks. + - Considerations for an HTML Writer [#]_: *************** *** 120,125 **** attribute (<HTML>?). ! - "Class" attributes should have no underscores (CSS requirement). .. [#] Source: `HTML 4.0 in Netscape and Explorer`__. __ http://www.webreference.com/dev/html4nsie/index.html --- 124,133 ---- attribute (<HTML>?). ! - Docutils identifiers (the "class" and "id" attributes) will ! conform to the regular expression ``[a-z][-a-z0-9]*``. See ! ``docutils.utils.id()``. + .. _HTML 4.01 spec: http://www.w3.org/TR/html401 + .. _CSS1 spec: http://www.w3.org/TR/REC-CSS1 .. [#] Source: `HTML 4.0 in Netscape and Explorer`__. __ http://www.webreference.com/dev/html4nsie/index.html *************** *** 198,213 **** If a document is split up, each segment will need navigation links: parent, children (small TOC), previous (preorder), next (preorder). - - - Table of Contents - ----------------- - - This runs over the entire tree, and locates <section> elements. It - produces a <contents> subtree, which can be inserted at the - appropriate place, with links to the <section> elements. It needs to - make sure that the links it uses are *real*, so ideally it will use - the "implicit" link for a section when it exists, and it will have to - invent one when the implicit link isn't there (presumably because the - section is the twelfth "Introduction" in the document...). --- 206,209 ---- |
From: David G. <go...@us...> - 2002-03-16 06:11:13
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv28848/dps/spec Modified Files: doctree.txt Log Message: updated Index: doctree.txt =================================================================== RCS file: /cvsroot/docstring/dps/spec/doctree.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -C2 -d -r1.7 -r1.8 *** doctree.txt 16 Jan 2002 02:43:13 -0000 1.7 --- doctree.txt 16 Mar 2002 06:11:11 -0000 1.8 *************** *** 56,62 **** - _`Bibliographic elements`: docinfo_, author_, authors_, organization_, contact_, version_, revision_, status_, date_, ! copyright_, abstract_ ! - _`Structural elements`: section_, transition_ - _`Body elements`: --- 56,62 ---- - _`Bibliographic elements`: docinfo_, author_, authors_, organization_, contact_, version_, revision_, status_, date_, ! copyright_ ! - _`Structural elements`: document_, section_, topic_, transition_ - _`Body elements`: *************** *** 76,80 **** - _`Inline elements`: emphasis_, strong_, interpreted_, literal_, reference_, target_, footnote_reference_, substitution_reference_, ! image_ --- 76,80 ---- - _`Inline elements`: emphasis_, strong_, interpreted_, literal_, reference_, target_, footnote_reference_, substitution_reference_, ! image_, problematic_ *************** *** 215,219 **** ``%reference.atts;`` ! =============== The ``%reference.atts;`` parameter entity --- 215,219 ---- ``%reference.atts;`` ! ==================== The ``%reference.atts;`` parameter entity |
From: David G. <go...@us...> - 2002-03-16 06:11:03
|
Update of /cvsroot/docstring/dps In directory usw-pr-cvs1:/tmp/cvs-serv28785/dps Modified Files: HISTORY.txt Log Message: updated Index: HISTORY.txt =================================================================== RCS file: /cvsroot/docstring/dps/HISTORY.txt,v retrieving revision 1.47 retrieving revision 1.48 diff -C2 -d -r1.47 -r1.48 *** HISTORY.txt 13 Mar 2002 02:48:27 -0000 1.47 --- HISTORY.txt 16 Mar 2002 06:11:01 -0000 1.48 *************** *** 131,135 **** - Moved 'normname()' from restructuredtext.states. - Added ``newdocument()``: document Node creation. ! * dps/test_*.py: Moved to new test/ directory. --- 131,135 ---- - Moved 'normname()' from restructuredtext.states. - Added ``newdocument()``: document Node creation. ! - Added name -> id conversion, ``id()``. * dps/test_*.py: Moved to new test/ directory. *************** *** 160,163 **** --- 160,164 ---- - Changed 'bibliographic_labels' to 'labels'. - Added labels for admonitions. + - Added label for table of contents title. * dps/transforms: Subpackage added. Thanks to Ueli Schlaepfer for the *************** *** 232,235 **** --- 233,237 ---- - Added "citation" and "citation_reference". - Added "raw". + - Added "refid" attribute to "title" for two-way Tables of Contents. * spec/pdpi.dtd: |
From: David G. <go...@us...> - 2002-03-16 06:09:47
|
Update of /cvsroot/docstring/dps/dps In directory usw-pr-cvs1:/tmp/cvs-serv28258/dps/dps Modified Files: __init__.py Log Message: *** empty log message *** Index: __init__.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/__init__.py,v retrieving revision 1.4 retrieving revision 1.5 diff -C2 -d -r1.4 -r1.5 *** __init__.py 7 Feb 2002 02:03:01 -0000 1.4 --- __init__.py 16 Mar 2002 06:09:45 -0000 1.5 *************** *** 8,12 **** :Copyright: This module has been placed in the public domain. ! This is the Docstring Processing System (DPS) Python package. Package Structure --- 8,12 ---- :Copyright: This module has been placed in the public domain. ! This is the Docstring Processing System (DPS) Python package. Package Structure |
From: David G. <go...@us...> - 2002-03-16 06:09:29
|
Update of /cvsroot/docstring/dps/dps/languages In directory usw-pr-cvs1:/tmp/cvs-serv28138/dps/dps/languages Modified Files: en.py Log Message: - Added label for "contents". Index: en.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/languages/en.py,v retrieving revision 1.9 retrieving revision 1.10 diff -C2 -d -r1.9 -r1.10 *** en.py 4 Mar 2002 04:47:52 -0000 1.9 --- en.py 16 Mar 2002 06:09:23 -0000 1.10 *************** *** 40,44 **** 'tip': 'Tip', 'warning': 'Warning', ! } """Mapping of node class name to label text.""" --- 40,44 ---- 'tip': 'Tip', 'warning': 'Warning', ! 'contents': 'Contents'} """Mapping of node class name to label text.""" |
From: David G. <go...@us...> - 2002-03-16 06:07:46
|
Update of /cvsroot/docstring/dps/dps In directory usw-pr-cvs1:/tmp/cvs-serv27402/dps/dps Modified Files: nodes.py Log Message: - Added support for "pending" elements. - Changed ID attribute generation. Index: nodes.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/nodes.py,v retrieving revision 1.36 retrieving revision 1.37 diff -C2 -d -r1.36 -r1.37 *** nodes.py 11 Mar 2002 03:43:26 -0000 1.36 --- nodes.py 16 Mar 2002 06:07:41 -0000 1.37 *************** *** 18,25 **** """ ! import sys import xml.dom.minidom from types import IntType, SliceType, StringType, TupleType from UserString import MutableString # ============================== --- 18,28 ---- """ ! import sys, os import xml.dom.minidom from types import IntType, SliceType, StringType, TupleType from UserString import MutableString + from dps import utils + import dps + # ============================== *************** *** 39,44 **** --- 42,52 ---- def asdom(self, dom=xml.dom.minidom): + """Return a DOM representation of this Node.""" return self._dom_node(dom) + def pformat(self, indent=' ', level=0): + """Return an indented pseudo-XML representation, for test purposes.""" + raise NotImplementedError + def walk(self, visitor): """ *************** *** 479,483 **** # ============== ! class document(Root, Element): def __init__(self, reporter, languagecode, *args, **kwargs): --- 487,491 ---- # ============== ! class document(Root, Structural, Element): def __init__(self, reporter, languagecode, *args, **kwargs): *************** *** 543,546 **** --- 551,557 ---- """List of symbol footnote_reference nodes.""" + self.pending = [] + """List of pending elements @@@.""" + self.anonymous_start = 1 """Initial anonymous hyperlink number.""" *************** *** 572,580 **** msgnode += msg else: ! while 1: id = 'id%s' % self.id_start self.id_start += 1 - if not self.ids.has_key(id): - break node['id'] = id self.ids[id] = node --- 583,593 ---- msgnode += msg else: ! if node.has_key('name'): ! id = utils.id(node['name']) ! else: ! id = '' ! while not id or self.ids.has_key(id): id = 'id%s' % self.id_start self.id_start += 1 node['id'] = id self.ids[id] = node *************** *** 697,700 **** --- 710,716 ---- subrefnode['refname'], []).append(subrefnode) + def note_pending(self, pendingnode): + self.pending.append(pendingnode) + # ================ *************** *** 839,858 **** """ ! The "pending" element is used to encapsulate a pending transform: the ! transform, the point at which to apply it, and any data it requires. ! For example, say you want a table of contents in your reStructuredText ! document. The easiest way to specify where to put it is from within the document, with a directive:: .. contents:: ! But the "contents" directive can't do its work until the entire document ! has been parsed (and possibly transformed to some extent). So the directive code leaves a placeholder behind that will trigger the second phase of the its processing, something like this:: ! <pending directive="contents" ...other attributes...> ! ...any directive data... The "pending" node is also appended to `document.pending`, so that a later --- 855,877 ---- """ ! ! The "pending" element is used to encapsulate a pending operation: the ! operation, the point at which to apply it, and any data it requires. Only ! the pending operation's location within the document is stored in the ! public document tree; the operation itself and its data are stored in ! internal instance attributes. ! For example, say you want a table of contents in your reStructuredText ! document. The easiest way to specify where to put it is from within the document, with a directive:: .. contents:: ! But the "contents" directive can't do its work until the entire document ! has been parsed (and possibly transformed to some extent). So the directive code leaves a placeholder behind that will trigger the second phase of the its processing, something like this:: ! <pending ...public attributes...> + internal attributes The "pending" node is also appended to `document.pending`, so that a later *************** *** 860,866 **** """ ! def __init__(self, transform, *children, **attributes): self.transform = transform ! """Contains the transform class...""" --- 879,915 ---- """ ! def __init__(self, transform, stage, details, ! rawsource='', *children, **attributes): ! Element.__init__(self, rawsource, *children, **attributes) ! self.transform = transform ! """The `dps.transforms.Transform` class implementing the pending ! operation.""" ! ! self.stage = stage ! """The stage of processing when the function will be called.""" ! ! self.details = details ! """Detail data required by the pending operation.""" ! ! def pformat(self, indent=' ', level=0): ! internals = [ ! '.. internal attributes:', ! ' pending.transform: %s.%s' % (self.transform.__module__, ! self.transform.__name__), ! ' pending.stage: %r' % self.stage, ! ' pending.details:'] ! details = self.details.items() ! details.sort() ! for key, value in details: ! if isinstance(value, Node): ! internals.append('%7s%s:' % ('', key)) ! internals.extend(['%9s%s' % ('', line) ! for line in value.pformat().splitlines()]) ! else: ! internals.append('%7s%s: %r' % ('', key, value)) ! return (Element.pformat(self, indent, level) ! + ''.join([(' %s%s\n' % (indent * level, line)) ! for line in internals])) |
From: David G. <go...@us...> - 2002-03-16 06:04:33
|
Update of /cvsroot/docstring/dps/dps/readers In directory usw-pr-cvs1:/tmp/cvs-serv26419/dps/dps/readers Modified Files: __init__.py Log Message: - Transform API update. Index: __init__.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/readers/__init__.py,v retrieving revision 1.4 retrieving revision 1.5 diff -C2 -d -r1.4 -r1.5 *** __init__.py 22 Feb 2002 02:01:02 -0000 1.4 --- __init__.py 16 Mar 2002 06:04:30 -0000 1.5 *************** *** 96,100 **** + tuple(self.transforms) + universal.last_reader_transforms): ! xclass().transform(self.document) def newdocument(self, languagecode=None): --- 96,100 ---- + tuple(self.transforms) + universal.last_reader_transforms): ! xclass(self.document).transform() def newdocument(self, languagecode=None): *************** *** 103,106 **** --- 103,107 ---- languagecode=(languagecode or self.languagecode), reporter=self.reporter) + document['source'] = self.source return document |
From: David G. <go...@us...> - 2002-03-16 06:03:31
|
Update of /cvsroot/docstring/dps/dps/transforms In directory usw-pr-cvs1:/tmp/cvs-serv26057/dps/dps/transforms Added Files: components.py Log Message: Document component transforms: table of contents generation, footnote/citation placement, etc. --- NEW FILE: components.py --- #! /usr/bin/env python """ :Authors: David Goodger, Ueli Schlaepfer :Contact: go...@us... :Revision: $Revision: 1.1 $ :Date: $Date: 2002/03/16 06:03:29 $ :Copyright: This module has been placed in the public domain. Transforms related to document components. - `Contents`: Used to build a table of contents. """ __docformat__ = 'reStructuredText' __all__ = ['build_contents', 'Contents'] import re from dps import nodes, utils from dps.transforms import TransformError, Transform class Contents(Transform): """ This transform generates a table of contents from the entire document tree or from a single branch. It locates "section" elements and builds them into a nested bullet list, which is placed within a "topic". A title is either explicitly specified, taken from the appropriate language module, or omitted (local table of contents). The depth may be specified. Two-way references between the table of contents and section titles are generated (requires Writer support). This transform requires a startnode, which which contains generation options and provides the location for the generated table of contents (the startnode is replaced by the table of contents "topic"). """ def transform(self): topic = nodes.topic(CLASS='contents') title = self.startnode.details['title'] if self.startnode.details.has_key('local'): startnode = self.startnode.parent # @@@ generate an error if the startnode (directive) not at # section/document top-level? Drag it up until it is? while not isinstance(startnode, nodes.Structural): startnode = startnode.parent if not title: title = [] else: startnode = self.doctree if not title: title = nodes.title('', self.language.labels['contents']) contents = self.build_contents(startnode) if len(contents): topic += title topic += contents self.startnode.parent.replace(self.startnode, topic) else: self.startnode.parent.remove(self.startnode) def build_contents(self, node, level=0): level += 1 sections = [] i = len(node) - 1 while i >= 0 and isinstance(node[i], nodes.section): sections.append(node[i]) i -= 1 sections.reverse() entries = [] for section in sections: title = section[0] reference = nodes.reference('', '', refid=section['id'], *title.getchildren()) entry = nodes.paragraph('', '', reference) item = nodes.list_item('', entry) itemid = self.doctree.set_id(item) title['refid'] = itemid if (not self.startnode.details.has_key('depth')) \ or level < self.startnode.details['depth']: subsects = self.build_contents(section, level) item += subsects entries.append(item) if entries: entries = nodes.bullet_list('', *entries) return entries |
From: David G. <go...@us...> - 2002-03-16 06:02:02
|
Update of /cvsroot/docstring/dps/dps/transforms In directory usw-pr-cvs1:/tmp/cvs-serv25511/dps/dps/transforms Modified Files: __init__.py Log Message: Updated the transform API with a "startnode" parameter; changed calling conventions. Index: __init__.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/transforms/__init__.py,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** __init__.py 6 Feb 2002 02:51:25 -0000 1.3 --- __init__.py 16 Mar 2002 06:02:00 -0000 1.4 *************** *** 38,47 **** class Transform: ! def transform(self, doctree): ! """Override to transform the document tree.""" ! raise NotImplementedError('subclass must override this method') - def setup_transform(self, doctree): - """Initial setup, used by `self.transform()`.""" self.doctree = doctree self.language = languages.getlanguage(doctree.languagecode) --- 38,62 ---- class Transform: ! """ ! Docutils transform component abstract base class. ! """ ! ! def __init__(self, doctree, startnode=None): ! """ ! Initial setup for in-place document transforms. ! """ self.doctree = doctree + """The document tree to transform.""" + + self.startnode = startnode + """Node from which to begin the transform. For many transforms which + apply to the document as a whole, `startnode` is not set (i.e. its + value is `None`).""" + self.language = languages.getlanguage(doctree.languagecode) + """Language module local to this document.""" + + def transform(self): + """Override to transform the document tree.""" + raise NotImplementedError('subclass must override this method') |
From: David G. <go...@us...> - 2002-03-16 06:00:24
|
Update of /cvsroot/docstring/dps/dps/transforms In directory usw-pr-cvs1:/tmp/cvs-serv24809/dps/dps/transforms Modified Files: frontmatter.py Log Message: - API update. Index: frontmatter.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/transforms/frontmatter.py,v retrieving revision 1.6 retrieving revision 1.7 diff -C2 -d -r1.6 -r1.7 *** frontmatter.py 4 Mar 2002 04:45:49 -0000 1.6 --- frontmatter.py 16 Mar 2002 06:00:17 -0000 1.7 *************** *** 104,109 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) if self.promote_document_title(): self.promote_document_subtitle() --- 104,108 ---- """ ! def transform(self): if self.promote_document_title(): self.promote_document_subtitle() *************** *** 148,152 **** index = doctree.findnonclass(nodes.PreBibliographic) if index is None or len(doctree) > (index + 1) or \ ! not isinstance(doctree[index], nodes.section): return None, None else: --- 147,151 ---- index = doctree.findnonclass(nodes.PreBibliographic) if index is None or len(doctree) > (index + 1) or \ ! not isinstance(doctree[index], nodes.section): return None, None else: *************** *** 225,233 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) ! self.process_docinfo() ! ! def process_docinfo(self): doctree = self.doctree index = doctree.findnonclass(nodes.PreBibliographic) --- 224,228 ---- """ ! def transform(self): doctree = self.doctree index = doctree.findnonclass(nodes.PreBibliographic) |
From: David G. <go...@us...> - 2002-03-16 05:59:40
|
Update of /cvsroot/docstring/dps/dps/transforms In directory usw-pr-cvs1:/tmp/cvs-serv24481/dps/dps/transforms Modified Files: references.py Log Message: - API update. Index: references.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/transforms/references.py,v retrieving revision 1.11 retrieving revision 1.12 diff -C2 -d -r1.11 -r1.12 *** references.py 13 Mar 2002 02:47:35 -0000 1.11 --- references.py 16 Mar 2002 05:59:38 -0000 1.12 *************** *** 119,124 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) self.resolve_anonymous() self.resolve_chained_targets() --- 119,123 ---- """ ! def transform(self): self.resolve_anonymous() self.resolve_chained_targets() *************** *** 251,255 **** """ ! Copy reference attributes up a length of hyperlink target chain. "Chained targets" are multiple adjacent internal hyperlink targets which --- 250,254 ---- """ ! Copy reference attributes up the length of a hyperlink target chain. "Chained targets" are multiple adjacent internal hyperlink targets which *************** *** 392,399 **** ] ! def transform(self, doctree): ! self.setup_transform(doctree) self.autofootnote_labels = [] ! startnum = doctree.autofootnote_start self.number_footnotes() self.number_footnote_references(startnum) --- 391,397 ---- ] ! def transform(self): self.autofootnote_labels = [] ! startnum = self.doctree.autofootnote_start self.number_footnotes() self.number_footnote_references(startnum) *************** *** 512,520 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) ! self.do_substitutions() ! ! def do_substitutions(self): defs = self.doctree.substitution_defs for refname, refs in self.doctree.substitution_refs.items(): --- 510,514 ---- """ ! def transform(self): defs = self.doctree.substitution_defs for refname, refs in self.doctree.substitution_refs.items(): |
From: David G. <go...@us...> - 2002-03-16 05:58:57
|
Update of /cvsroot/docstring/dps/dps/transforms In directory usw-pr-cvs1:/tmp/cvs-serv24264/dps/dps/transforms Modified Files: universal.py Log Message: - Added "Pending" transforms. - API update. Index: universal.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/transforms/universal.py,v retrieving revision 1.3 retrieving revision 1.4 diff -C2 -d -r1.3 -r1.4 *** universal.py 11 Mar 2002 03:38:56 -0000 1.3 --- universal.py 16 Mar 2002 05:58:54 -0000 1.4 *************** *** 27,39 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) ! if len(doctree.messages) > 0: section = nodes.section(CLASS='system-messages') # @@@ get this from the language module? section += nodes.title('', 'Docutils System Messages') ! section += doctree.messages.getchildren() ! doctree.messages[:] = [] ! doctree += section --- 27,38 ---- """ ! def transform(self): ! if len(self.doctree.messages) > 0: section = nodes.section(CLASS='system-messages') # @@@ get this from the language module? section += nodes.title('', 'Docutils System Messages') ! section += self.doctree.messages.getchildren() ! self.doctree.messages[:] = [] ! self.doctree += section *************** *** 44,64 **** """ ! def transform(self, doctree): ! self.setup_transform(doctree) ! doctree += doctree.messages.getchildren() test_transforms = (TestMessages,) ! """Tuple of universal transforms to apply to the raw doctree when testing.""" ! first_reader_transforms = () ! """Tuple of universal transforms to apply before any other Reader transforms.""" ! last_reader_transforms = (Messages,) ! """Tuple of universal transforms to apply after all other Reader transforms.""" ! first_writer_transforms = () ! """Tuple of universal transforms to apply before any other Writer transforms.""" ! last_writer_transforms = () ! """Tuple of universal transforms to apply after all other Writer transforms.""" --- 43,101 ---- """ ! def transform(self): ! self.doctree += self.doctree.messages.getchildren() ! ! ! class Pending(Transform): ! ! """ ! Execute pending transforms. ! """ ! ! stage = None ! """The stage of processing applicable to this transform; match with ! `nodes.pending.stage`. Possible values include 'first_reader', ! 'last_reader', 'first_writer', and 'last_writer'. Override in ! subclasses.""" ! ! def transform(self): ! for pending in self.doctree.pending: ! if pending.stage == self.stage: ! pending.transform(self.doctree, pending).transform() ! ! ! class FirstReaderPending(Pending): ! ! stage = 'first_reader' ! ! ! class LastReaderPending(Pending): ! ! stage = 'last_reader' ! ! ! class FirstWriterPending(Pending): ! ! stage = 'first_writer' ! ! ! class LastWriterPending(Pending): ! ! stage = 'last_writer' ! test_transforms = (TestMessages,) ! """Universal transforms to apply to the raw doctree when testing.""" ! first_reader_transforms = (FirstReaderPending,) ! """Universal transforms to apply before any other Reader transforms.""" ! last_reader_transforms = (LastReaderPending, Messages) ! """Universal transforms to apply after all other Reader transforms.""" ! first_writer_transforms = (FirstWriterPending,) ! """Universal transforms to apply before any other Writer transforms.""" ! last_writer_transforms = (LastWriterPending,) ! """Universal transforms to apply after all other Writer transforms.""" |
From: David G. <go...@us...> - 2002-03-16 05:57:34
|
Update of /cvsroot/docstring/dps/dps In directory usw-pr-cvs1:/tmp/cvs-serv23805/dps/dps Modified Files: utils.py Log Message: - Added name -> id conversion, ``id()``. - Updated extension attribute support. Index: utils.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/utils.py,v retrieving revision 1.18 retrieving revision 1.19 diff -C2 -d -r1.18 -r1.19 *** utils.py 13 Mar 2002 02:43:47 -0000 1.18 --- utils.py 16 Mar 2002 05:57:30 -0000 1.19 *************** *** 11,15 **** """ ! import sys import nodes --- 11,15 ---- """ ! import sys, re import nodes *************** *** 184,188 **** class ExtensionAttributeError(Exception): pass ! class BadAttributeLineError(ExtensionAttributeError): pass class BadAttributeDataError(ExtensionAttributeError): pass class DuplicateAttributeError(ExtensionAttributeError): pass --- 184,188 ---- class ExtensionAttributeError(Exception): pass ! class BadAttributeError(ExtensionAttributeError): pass class BadAttributeDataError(ExtensionAttributeError): pass class DuplicateAttributeError(ExtensionAttributeError): pass *************** *** 204,208 **** function). - `DuplicateAttributeError` for duplicate attributes. ! - `BadAttributeError` for input lines not enclosed in brackets. - `BadAttributeDataError` for invalid attribute data (missing name, missing data, bad quotes, etc.). --- 204,208 ---- function). - `DuplicateAttributeError` for duplicate attributes. ! - `BadAttributeError` for invalid fields. - `BadAttributeDataError` for invalid attribute data (missing name, missing data, bad quotes, etc.). *************** *** 221,225 **** :Exceptions: ! - `BadAttributeError` for input lines not enclosed in brackets.? - `BadAttributeDataError` for invalid attribute data (missing name, missing data, bad quotes, etc.). --- 221,225 ---- :Exceptions: ! - `BadAttributeError` for invalid fields. - `BadAttributeDataError` for invalid attribute data (missing name, missing data, bad quotes, etc.). *************** *** 232,241 **** name = field[0].astext().lower() body = field[1] ! if len(body) != 1 or not isinstance(body[0], nodes.paragraph) \ or len(body[0]) != 1 or not isinstance(body[0][0], nodes.Text): raise BadAttributeDataError( ! 'extension attribute field body may consist of\n' 'a single paragraph only (attribute "%s")' % name) ! data = body[0][0].astext() attlist.append((name, data)) return attlist --- 232,244 ---- name = field[0].astext().lower() body = field[1] ! if len(body) == 0: ! data = None ! elif len(body) > 1 or not isinstance(body[0], nodes.paragraph) \ or len(body[0]) != 1 or not isinstance(body[0][0], nodes.Text): raise BadAttributeDataError( ! 'extension attribute field body may contain\n' 'a single paragraph only (attribute "%s")' % name) ! else: ! data = body[0][0].astext() attlist.append((name, data)) return attlist *************** *** 264,269 **** try: attributes[name] = convertor(value) ! except ValueError, detail: ! raise ValueError('(attribute "%s") %s' % (name, detail)) return attributes --- 267,273 ---- try: attributes[name] = convertor(value) ! except (ValueError, TypeError), detail: ! raise detail.__class__('(attribute "%s", value "%r") %s' ! % (name, value, detail)) return attributes *************** *** 320,323 **** --- 324,369 ---- """Return a case- and whitespace-normalized name.""" return ' '.join(name.lower().split()) + + def id(string): + """ + Convert `string` into an identifier and return it. + + Docutils identifiers will conform to the regular expression + ``[a-z][-a-z0-9]*``. For CSS compatibility, identifiers (the "class" and + "id" attributes) should have no underscores, colons, or periods. Hyphens + may be used. + + - The `HTML 4.01 spec`_ defines identifiers based on SGML tokens: + + ID and NAME tokens must begin with a letter ([A-Za-z]) and may be + followed by any number of letters, digits ([0-9]), hyphens ("-"), + underscores ("_"), colons (":"), and periods ("."). + + - However the `CSS1 spec`_ defines identifiers based on the "name" token, + a tighter interpretation ("flex" tokenizer notation; "latin1" and + "escape" 8-bit characters have been replaced with entities):: + + unicode \\[0-9a-f]{1,4} + latin1 [¡-ÿ] + escape {unicode}|\\[ -~¡-ÿ] + nmchar [-a-z0-9]|{latin1}|{escape} + name {nmchar}+ + + The CSS1 "nmchar" rule does not include underscores ("_"), colons (":"), + or periods ("."), therefore "class" and "id" attributes should not contain + these characters. They should be replaced with hyphens ("-"). Combined + with HTML's requirements (the first character must be a letter; no + "unicode", "latin1", or "escape" characters), this results in the + ``[a-z][-a-z0-9]*`` pattern. + + .. _HTML 4.01 spec: http://www.w3.org/TR/html401 + .. _CSS1 spec: http://www.w3.org/TR/REC-CSS1 + """ + id = non_id_chars.sub('-', normname(string)) + id = non_id_at_ends.sub('', id) + return str(id) + + non_id_chars = re.compile('[^a-z0-9]+') + non_id_at_ends = re.compile('^[-0-9]+|-+$') def newdocument(languagecode='en', warninglevel=2, errorlevel=4, |
From: David G. <go...@us...> - 2002-03-16 05:53:55
|
Update of /cvsroot/docstring/dps/dps/writers In directory usw-pr-cvs1:/tmp/cvs-serv22569/dps/dps/writers Modified Files: __init__.py Log Message: transform API update Index: __init__.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/writers/__init__.py,v retrieving revision 1.6 retrieving revision 1.7 diff -C2 -d -r1.6 -r1.7 *** __init__.py 7 Mar 2002 04:00:40 -0000 1.6 --- __init__.py 16 Mar 2002 05:53:52 -0000 1.7 *************** *** 61,65 **** + tuple(self.transforms) + universal.last_writer_transforms): ! xclass().transform(self.document) def translate(self): --- 61,65 ---- + tuple(self.transforms) + universal.last_writer_transforms): ! xclass(self.document).transform() def translate(self): *************** *** 83,87 **** (c) `None`, which implies `sys.stdout`. """ ! output = output.encode('raw-unicode-escape') # @@@ temporary? if hasattr(self.destination, 'write'): destination.write(output) --- 83,87 ---- (c) `None`, which implies `sys.stdout`. """ ! output = output.encode('raw-unicode-escape') # @@@ temporary if hasattr(self.destination, 'write'): destination.write(output) |
From: David G. <go...@us...> - 2002-03-16 05:53:25
|
Update of /cvsroot/docstring/dps/dps/writers In directory usw-pr-cvs1:/tmp/cvs-serv22392/dps/dps/writers Modified Files: html.py Log Message: - Added credits, ToC two-way references. Index: html.py =================================================================== RCS file: /cvsroot/docstring/dps/dps/writers/html.py,v retrieving revision 1.13 retrieving revision 1.14 diff -C2 -d -r1.13 -r1.14 *** html.py 13 Mar 2002 02:40:42 -0000 1.13 --- html.py 16 Mar 2002 05:53:22 -0000 1.14 *************** *** 20,23 **** --- 20,24 ---- + import time from dps import writers, nodes, languages *************** *** 251,255 **** def depart_docinfo_item(self): ! self.body.append('</P></TD>\n</TR>\n') def visit_doctest_block(self, node): --- 252,256 ---- def depart_docinfo_item(self): ! self.body.append('</P>\n</TD></TR>') def visit_doctest_block(self, node): *************** *** 264,267 **** --- 265,272 ---- def depart_document(self, node): self.body.append('</DIV>\n') + self.body.append( + '<P CLASS="credits">HTML generated from <CODE>%s</CODE> on %s ' + 'by <A HREF="http://docutils.sourceforge.net/">Docutils</A>.' + '</P>\n' % (node['source'], time.strftime('%Y-%m-%d'))) def visit_emphasis(self, node): *************** *** 676,681 **** if isinstance(node.parent, nodes.topic): self.body.append( ! self.starttag(node, 'H6', '', CLASS='topic-title')) ! self.context.append('</H6>\n') elif self.sectionlevel == 0: self.head.append('<TITLE>%s</TITLE>\n' --- 681,686 ---- if isinstance(node.parent, nodes.topic): self.body.append( ! self.starttag(node, 'P', '', CLASS='topic-title')) ! self.context.append('</P>\n') elif self.sectionlevel == 0: self.head.append('<TITLE>%s</TITLE>\n' *************** *** 686,690 **** self.body.append( self.starttag(node, 'H%s' % self.sectionlevel, '')) ! self.context.append('</H%s>\n' % self.sectionlevel) def depart_title(self, node): --- 691,699 ---- self.body.append( self.starttag(node, 'H%s' % self.sectionlevel, '')) ! context = '' ! if node.hasattr('refid'): ! self.body.append('<A HREF="#%s">' % node['refid']) ! context = '</A>' ! self.context.append('%s</H%s>\n' % (context, self.sectionlevel)) def depart_title(self, node): |
From: David G. <go...@us...> - 2002-03-16 05:50:47
|
Update of /cvsroot/docstring/dps/spec In directory usw-pr-cvs1:/tmp/cvs-serv21457/dps/spec Modified Files: gpdi.dtd Log Message: - Added "refid" attribute to "title" for two-way Tables of Contents. - Made "topic" title optional. - Removed "directive" attribute from "pending": it will use internal attributes. Index: gpdi.dtd =================================================================== RCS file: /cvsroot/docstring/dps/spec/gpdi.dtd,v retrieving revision 1.38 retrieving revision 1.39 diff -C2 -d -r1.38 -r1.39 *** gpdi.dtd 11 Mar 2002 03:26:52 -0000 1.38 --- gpdi.dtd 16 Mar 2002 05:50:44 -0000 1.39 *************** *** 183,187 **** <!ELEMENT title %text.model;> ! <!ATTLIST title %basic.atts;> <!ELEMENT subtitle %text.model;> --- 183,189 ---- <!ELEMENT title %text.model;> ! <!ATTLIST title ! %basic.atts; ! %refid.att;> <!ELEMENT subtitle %text.model;> *************** *** 236,240 **** <!ATTLIST section %basic.atts;> ! <!ELEMENT topic (title, (%body.elements;)+)> <!ATTLIST topic %basic.atts;> --- 238,242 ---- <!ATTLIST section %basic.atts;> ! <!ELEMENT topic (title?, (%body.elements;)+)> <!ATTLIST topic %basic.atts;> *************** *** 396,403 **** %fixedspace.att;> ! <!ELEMENT pending (field_list)> ! <!ATTLIST pending ! %basic.atts; ! directive CDATA #REQUIRED> <!ELEMENT figure (image, ((caption, legend?) | legend) > --- 398,403 ---- %fixedspace.att;> ! <!ELEMENT pending EMPTY> ! <!ATTLIST pending %basic.atts;> <!ELEMENT figure (image, ((caption, legend?) | legend) > |