docstring-checkins

[David G. <go...@us...>]

Update of /cvsroot/docstring/dps/spec
In directory usw-pr-cvs1:/tmp/cvs-serv13589/dps/spec

Modified Files:
	pep-0256.txt 
Log Message:
Overhauled.

Index: pep-0256.txt
===================================================================
RCS file: /cvsroot/docstring/dps/spec/pep-0256.txt,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -d -r1.2 -r1.3
*** pep-0256.txt	4 Aug 2001 15:44:49 -0000	1.2
--- pep-0256.txt	28 Mar 2002 04:36:11 -0000	1.3
***************
*** 3,7 ****
  Version: $Revision$
  Last-Modified: $Date$
! Author: dgo...@bi... (David Goodger)
  Discussions-To: do...@py...
  Status: Draft
--- 3,7 ----
  Version: $Revision$
  Last-Modified: $Date$
! Author: go...@us... (David Goodger)
  Discussions-To: do...@py...
  Status: Draft
***************
*** 13,38 ****
  Abstract
  
!     Python modules, functions, classes and methods have a string
!     attribute called __doc__.  If the first expression inside their
!     definition is a literal string, that string is assigned to the
!     __doc__ attribute, called a documentation string or docstring.  It
!     is often used to summarize the interface of the module, function,
!     class or method.
  
-     There is no standard format (markup) for docstrings, nor are there
-     standard tools for extracting docstrings and transforming them
-     into useful structured formats (e.g., HTML, DocBook, TeX).  Those
-     tools that do exist are for the most part unmaintained and unused.
      The issues surrounding docstring processing have been contentious
!     and difficult to resolve.
! 
!     This PEP proposes a Docstring Processing System (DPS) framework.
!     It separates out the components (program and conceptual), enabling
!     the resolution of individual issues either through consensus (one
!     solution) or through divergence (many).  It promotes standard
!     interfaces which will allow a variety of plug-in components (input
!     parsers and output formatters) to be used.
  
!     This PEP presents the concepts of a DPS framework independently of
      implementation details.
  
--- 13,33 ----
  Abstract
  
!     Python lends itself to inline documentation.  With its built-in
!     docstring syntax, a limited form of Literate Programming [1]_ is
!     easy to do in Python.  However, there are no satisfactory standard
!     tools for extracting and processing Python docstrings.  The lack
!     of a standard toolset is a significant gap in Python's
!     infrastructure; this PEP aims to fill the gap.
  
      The issues surrounding docstring processing have been contentious
!     and difficult to resolve.  This PEP proposes a generic Docstring
!     Processing System (DPS) framework, which separates out the
!     components (program and conceptual), enabling the resolution of
!     individual issues either through consensus (one solution) or
!     through divergence (many).  It promotes standard interfaces which
!     will allow a variety of plug-in components (input context readers,
!     markup parsers, and output format writers) to be used.
  
!     The concepts of a DPS framework are presented independently of
      implementation details.
  
***************
*** 40,167 ****
  Rationale
  
-     Python lends itself to inline documentation.  With its built-in
-     docstring syntax, a limited form of Literate Programming [1] is
-     easy to do in Python.  However, there are no satisfactory standard
-     tools for extracting and processing Python docstrings.  The lack
-     of a standard toolset is a significant gap in Python's
-     infrastructure; this PEP aims to fill the gap.
- 
      There are standard inline documentation systems for some other
!     languages.  For example, Perl has POD (plain old documentation)
!     and Java has Javadoc, but neither of these mesh with the Pythonic
!     way.  POD is very explicit, but takes after Perl in terms of
!     readability.  Javadoc is HTML-centric; except for '@field' tags,
!     raw HTML is used for markup.  There are also general tools such as
!     Autoduck and Web (Tangle & Weave), useful for multiple languages.
  
!     There have been many attempts to write autodocumentation systems
      for Python (not an exhaustive list):
  
!     - Marc-Andre Lemburg's doc.py [2]
  
!     - Daniel Larsson's pythondoc & gendoc [3]
  
!     - Doug Hellmann's HappyDoc [4]
  
!     - Laurence Tratt's Crystal [5]
  
!     - Ka-Ping Yee's htmldoc & pydoc [6] (pydoc.py is now part of the
        Python standard library; see below)
  
!     - Tony Ibbs' docutils [7]
  
!     - Edward Loper's STminus formalization and related efforts [8]
  
      These systems, each with different goals, have had varying degrees
      of success.  A problem with many of the above systems was
!     over-ambition.  They provided a self-contained set of components:
!     a docstring extraction system, an input parser, an internal
!     processing system and one or more output formatters.  Inevitably,
!     one or more components had serious shortcomings, preventing the
!     system from being adopted as a standard tool.
! 
!     Throughout the existence of the Python Documentation Special
!     Interest Group (Doc-SIG) [9], consensus on a single standard
!     docstring format has never been reached.  A lightweight, implicit
!     markup has been sought, for the following reasons (among others):
! 
!     1. Docstrings written within Python code are available from within
!        the interactive interpreter, and can be 'print'ed.  Thus the
!        use of plaintext for easy readability.
! 
!     2. Programmers want to add structure to their docstrings, without
!        sacrificing raw docstring readability.  Unadorned plaintext
!        cannot be transformed ('up-translated') into useful structured
!        formats.
! 
!     3. Explicit markup (like XML or TeX) has been widely considered
!        unreadable by the uninitiated.
! 
!     4. Implicit markup is aesthetically compatible with the clean and
!        minimalist Python syntax.
! 
!     Early on, variants of Setext (Structure Enhanced Text) [10],
!     including Digital Creation's StructuredText [11], were proposed
!     for Python docstring formatting.  Hereafter we will collectively
!     call these variants 'STexts'.  Although used by some (including in
!     most of the above-listed autodocumentation tools), these markup
!     schemes have failed to become standard because:
! 
!     - STexts have been incomplete: lacking 'essential' constructs that
!       people want to use in their docstrings, STexts are rendered less
!       than ideal.  Note that these 'essential' constructs are not
!       universal; everyone has their own requirements.
! 
!     - STexts have been sometimes surprising: bits of text are marked
!       up unexpectedly, leading to user frustration.
! 
!     - SText implementations have been buggy.
! 
!     - Some STexts have have had no formal specification except for the
!       implementation itself.  A buggy implementation meant a buggy
!       spec, and vice-versa.
! 
!     - There has been no mechanism to get around the SText markup rules
!       when a markup character is used in a non-markup context.
! 
!     Recognizing the deficiencies of STexts, some people have proposed
!     using explicit markup of some kind.  There have been proposals for
!     using XML, HTML, TeX, POD, and Javadoc at one time or another.
!     Proponents of STexts have vigorously opposed these proposals, and
!     the debates have continued off and on for at least five years.
  
      It has become clear (to this author, at least) that the "all or
!     nothing" approach cannot succeed, since no all-encompassing
!     proposal could possibly be agreed upon by all interested parties.
!     A modular component approach, where components may be multiply
!     implemented, is the only chance at success.  By separating out the
!     issues, we can form consensus more easily (smaller fights ;-), and
!     accept divergence more readily.
  
      Each of the components of a docstring processing system should be
      developed independently.  A 'best of breed' system should be
!     chosen and/or developed and eventually included in Python's
!     standard library.
  
  
! Pydoc & Other Existing Systems
  
!     Pydoc is part of the Python 2.1 standard library.  It extracts and
!     displays docstrings from within the Python interactive
!     interpreter, from the shell command line, and from a GUI window
!     into a web browser (HTML).  In the case of GUI/HTML, except for
!     some heuristic hyperlinking of identifier names, no formatting of
!     the docstrings is done.  They are presented within <p><small><tt>
!     tags to avoid unwanted line wrapping.  Unfortunately, the result
!     is not pretty.
  
      The functionality proposed in this PEP could be added to or used
!     by pydoc when serving HTML pages.  However, the proposed docstring
!     processing system's functionality is much more than pydoc needs
!     (in its current form).  Either an independent tool will be
!     developed (which pydoc may or may not use), or pydoc could be
!     expanded to encompass this functionality and *become* the
!     docstring processing system (or one such system).  That decision
!     is beyond the scope of this PEP.
  
      Similarly for other existing docstring processing systems, their
--- 35,119 ----
  Rationale
  
      There are standard inline documentation systems for some other
!     languages.  For example, Perl has POD [2]_ and Java has Javadoc
!     [3]_, but neither of these mesh with the Pythonic way.  POD syntax
!     is very explicit, but takes after Perl in terms of readability.
!     Javadoc is HTML-centric; except for '@field' tags, raw HTML is
!     used for markup.  There are also general tools such as Autoduck
!     [4]_ and Web (Tangle & Weave) [5]_, useful for multiple languages.
  
!     There have been many attempts to write auto-documentation systems
      for Python (not an exhaustive list):
  
!     - Marc-Andre Lemburg's doc.py [6]_
  
!     - Daniel Larsson's pythondoc & gendoc [7]_
  
!     - Doug Hellmann's HappyDoc [8]_
  
!     - Laurence Tratt's Crystal [9]_
  
!     - Ka-Ping Yee's htmldoc & pydoc [10]_ (pydoc.py is now part of the
        Python standard library; see below)
  
!     - Tony Ibbs' docutils [11]_
  
!     - Edward Loper's STminus formalization and related efforts [12]_
  
      These systems, each with different goals, have had varying degrees
      of success.  A problem with many of the above systems was
!     over-ambition combined with inflexibility.  They provided a
!     self-contained set of components: a docstring extraction system,
!     a markup parser, an internal processing system and one or more
!     output format writers.  Inevitably, one or more aspects of each
!     system had serious shortcomings, and they were not easily extended
!     or modified, preventing them from being adopted as standard tools.
  
      It has become clear (to this author, at least) that the "all or
!     nothing" approach cannot succeed, since no monolithic
!     self-contained system could possibly be agreed upon by all
!     interested parties.  A modular component approach designed for
!     extension, where components may be multiply implemented, may be
!     the only chance for success.  By separating out the issues, we can
!     form consensus more easily (smaller fights ;-), and accept
!     divergence more readily.
  
      Each of the components of a docstring processing system should be
      developed independently.  A 'best of breed' system should be
!     chosen, either merged from existing systems, and/or developed
!     anew.  This system should be included in Python's standard
!     library.
  
  
! PyDoc & Other Existing Systems
  
!     PyDoc became part of the Python standard library as of release
!     2.1.  It extracts and displays docstrings from within the Python
!     interactive interpreter, from the shell command line, and from a
!     GUI window into a web browser (HTML).  Although a very useful
!     tool, PyDoc has several deficiencies, including:
! 
!     - In the case of the GUI/HTML, except for some heuristic
!       hyperlinking of identifier names, no formatting of the
!       docstrings is done.  They are presented within <p><small><tt>
!       tags to avoid unwanted line wrapping.  Unfortunately, the result
!       is not attractive.
! 
!     - PyDoc extracts docstrings and structural information (class
!       identifiers, method signatures, etc.) from imported module
!       objects.  There are security issues involved with importing
!       untrusted code.  Also, information from the source is lost when
!       importing, such as comments, "additional docstrings" (string
!       literals in non-docstring contexts; see PEP 258 [13]_), and the
!       order of definitions.
  
      The functionality proposed in this PEP could be added to or used
!     by PyDoc when serving HTML pages.  The proposed docstring
!     processing system's functionality is much more than PyDoc needs in
!     its current form.  Either an independent tool will be developed
!     (which PyDoc may or may not use), or PyDoc could be expanded to
!     encompass this functionality and *become* the docstring processing
!     system (or one such system).  That decision is beyond the scope of
!     this PEP.
  
      Similarly for other existing docstring processing systems, their
***************
*** 183,190 ****
         - First line is a one-line synopsis.
  
!        PEP 257, Docstring Conventions [12], documents these issues.
  
!     2. Docstring processing system generic implementation details.
!        Documents issues such as:
  
         - High-level spec: what a DPS does.
--- 135,143 ----
         - First line is a one-line synopsis.
  
!        PEP 257, Docstring Conventions [14]_, documents some of these
!        issues.
  
!     2. Docstring processing system design specification.  Documents
!        issues such as:
  
         - High-level spec: what a DPS does.
***************
*** 192,257 ****
         - Command-line interface for executable script.
  
!        - System Python API
  
         - Docstring extraction rules.
  
!        - Input parser API.
  
!        - Intermediate internal data structure: output from input
!          parser, input to output formatter.
  
!        - Output formatter API.
  
!        - Output management.
  
         These issues are applicable to any docstring processing system
!        implementation.  PEP 258, DPS Generic Implementation Details
!        [13], documents these issues.
  
      3. Docstring processing system implementation.
  
!     4. Input markup specifications: docstring syntax.
  
      5. Input parser implementations.
  
!     6. Output formats (HTML, XML, TeX, DocBook, info, etc.).
! 
!     7. Output formatter implementations.
! 
!     Components 1, 2, and 3 will be the subject of individual companion
!     PEPs, although they may be merged into this PEP once consensus is
!     reached.  If there is only one implementation, PEPs for components
!     2 & 3 can be combined.  Multiple PEPs will be necessary for each
!     of components 4, 5, 6, and 7.  An alternative to the PEP mechanism
!     may be used instead, since these are not directly related to the
!     Python language.
! 
!     The following diagram shows an overview of the framework.
!     Interfaces are indicated by double-borders.  The ASCII diagram is
!     very wide; please turn off line wrapping to view it:
  
  
!                                                     +========================+
!                                                     | Command-Line Interface |
!                                                     +========================+
!                                                     | Executable Script      |
!                                                     +------------------------+
!                                                                 |
!                                                                 | calls
!                                                                 v
!                                                     +===========================================+ returns   +---------+
!                                                     | System Python API                         |==========>| output  |
!                               +--------+            +===========================================+           | objects |
!            _    writes        | Python |      reads | Docstring Processing System               |           +---------+
!           / \  ==============>| module |<===========|                                           |
!           \_/                 +--------+            | input      | transformation, | output     |            +--------+
!            |             +-------------+    follows | docstring  | integration,    | object     | writes     | output |
!          --+--  consults | docstring   |<-----------| extraction | linking         | management |===========>| files  |
!            |   --------->| conventions |            +============+=====+=====+=====+============+            +--------+
!           / \            +-------------+            | parser API       |     |    formatter API |
!          /   \           +-------------+            +===========+======+     +======+===========+            +--------+
!         author  consults | markup      | implements | input     | intermediate      | output    | implements | output |
!                --------->| syntax spec |<-----------| parser    | data structure    | formatter |----------->| format |
!                          +-------------+            +-----------+-------------------+-----------+            +--------+
  
  
--- 145,190 ----
         - Command-line interface for executable script.
  
!        - System Python API.
  
         - Docstring extraction rules.
  
!        - Readers, which encapsulate the input context .
  
!        - Parsers.
  
!        - Document tree: the intermediate internal data structure.  The
!          output of the Parser and Reader, and the input to the Writer
!          all share the same data structure.
  
!        - Transforms, which modify the document tree.
! 
!        - Writers for output formats.
! 
!        - Distributors, which handle output management (one file, many
!          files, or objects in memory).
  
         These issues are applicable to any docstring processing system
!        implementation.  PEP 258, Docutils Design Specification [13 ]_,
!        documents these issues.
  
      3. Docstring processing system implementation.
  
!     4. Input markup specifications: docstring syntax.  PEP 2xx,
!        reStructuredText Standard Docstring Format [15]_, proposes a
!        standard syntax.
  
      5. Input parser implementations.
  
!     6. Input context readers ("modes": Python source code, PEP,
!        standalone text file, email, etc.) and implementations.
  
+     7. Output formats (HTML, XML, TeX, DocBook, info, etc.) and writer
+        implementations.
  
!     Components 1, 2/3, and 4/5 are the subject of individual companion
!     PEPs.  If there is another implementation of the framework or
!     syntax/parser, additional PEPs may be required.  Multiple
!     implementations of each of components 6 and 7 will be required;
!     the PEP mechanism may be overkill for these components.
  
  
***************
*** 259,263 ****
  
      A SourceForge project has been set up for this work at
!     http://docstring.sf.net.
  
  
--- 192,196 ----
  
      A SourceForge project has been set up for this work at
!     http://docstring.sourceforge.net/.
  
  
***************
*** 266,298 ****
      [1] http://www.literateprogramming.com/
  
!     [2] http://www.lemburg.com/files/python/SoftwareDescriptions.html#doc.py
  
!     [3] http://starship.python.net/crew/danilo/pythondoc/
  
!     [4] http://happydoc.sf.net/
  
!     [5] http://www.btinternet.com/~tratt/comp/python/crystal/index.html
  
!     [6] http://www.lfw.org/python/
  
!     [7] http://homepage.ntlworld.com/tibsnjoan/docutils/
  
!     [8] http://www.cis.upenn.edu/~edloper/pydoc/
  
!     [9] http://www.python.org/sigs/doc-sig/
  
!     [10] http://www.bsdi.com/setext/
  
!     [11] http://dev.zope.org/Members/jim/StructuredTextWiki/FrontPage/
  
!     [12] PEP 257, Docstring Conventions, Goodger, Van Rossum
!          http://www.python.org/peps/pep-0257.html
  
!     [13] PEP 258, DPS Generic Implementation Details, Goodger
           http://www.python.org/peps/pep-0258.html
  
!     [14] PEP 216, Docstring Format, Zadka
!          http://www.python.org/peps/pep-0216.html
  
  
  
--- 199,235 ----
      [1] http://www.literateprogramming.com/
  
!     [2] Perl "Plain Old Documentation"
!         http://www.perldoc.com/perl5.6/pod/perlpod.html
  
!     [3] http://java.sun.com/j2se/javadoc/
  
!     [4] http://www.helpmaster.com/hlp-developmentaids-autoduck.htm
  
!     [5] http://www-cs-faculty.stanford.edu/~knuth/cweb.html
  
!     [6] http://www.lemburg.com/files/python/SoftwareDescriptions.html#doc.py
  
!     [7] http://starship.python.net/crew/danilo/pythondoc/
  
!     [8] http://happydoc.sourceforge.net/
  
!     [9] http://www.btinternet.com/~tratt/comp/python/crystal/
  
!     [10] http://www.python.org/doc/current/lib/module-pydoc.html
  
!     [11] http://homepage.ntlworld.com/tibsnjoan/docutils/
  
!     [12] http://www.cis.upenn.edu/~edloper/pydoc/
  
!     [13] PEP 258, Docutils Design Specification, Goodger
           http://www.python.org/peps/pep-0258.html
  
!     [14] PEP 257, Docstring Conventions, Goodger, Van Rossum
!          http://www.python.org/peps/pep-0257.html
  
+     [15] PEP 2xx, reStructuredText Standard Docstring Format, Goodger
+          http://www.python.org/peps/pep-02xx.html
+ 
+     [16] http://www.python.org/sigs/doc-sig/
  
  
***************
*** 304,313 ****
  Acknowledgements
  
!     This document borrows text from PEP 216, Docstring Format, by
!     Moshe Zadka [14].  It is intended as a reorganization of PEP 216
!     and its approach.
! 
!     This document also borrows ideas from the archives of the Python
!     Doc-SIG.  Thanks to all members past & present.
  
  
--- 241,246 ----
  Acknowledgements
  
!     This document borrows ideas from the archives of the Python
!     Doc-SIG [16]_.  Thanks to all members past & present.