epydoc-devel Mailing List for Python API documentation generation tool (Page 6)
Brought to you by:
edloper
You can subscribe to this list here.
2004 |
Jan
|
Feb
|
Mar
|
Apr
(12) |
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
(2) |
Nov
|
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2005 |
Jan
(4) |
Feb
|
Mar
|
Apr
(3) |
May
(1) |
Jun
(1) |
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2006 |
Jan
(4) |
Feb
|
Mar
(13) |
Apr
|
May
|
Jun
(1) |
Jul
(16) |
Aug
(3) |
Sep
|
Oct
(1) |
Nov
|
Dec
|
2007 |
Jan
(6) |
Feb
(36) |
Mar
(4) |
Apr
(4) |
May
(28) |
Jun
(6) |
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
(8) |
Dec
(4) |
2008 |
Jan
(5) |
Feb
(20) |
Mar
(11) |
Apr
(7) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(3) |
Oct
|
Nov
|
Dec
|
2009 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
(5) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2012 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2014 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Edward L. <ed...@gr...> - 2007-02-20 16:17:14
|
On Feb 18, 2007, at 7:04 PM, Daniele Varrazzo wrote: > A simple naming scheme could be: > > - prefix each uppercase and underscore with an underscore > - prefix each name with a letter (let's say N, for "name") > > The mapping should be injective even considering non-case-sensitive > comparisons. Examples: > > MyName -> N_My_Name > _my_name -> N__my__name > __init__ -> N____init____ I'd say to go ahead with this, for anchors only, but let's try to keep the mangled names as readable as possible. First, I don't think adding a prefix is necessary, because (1) we don't generate many other anchors anyway, and (2) when we do generate them, they all currently contain "-". We could continue that with any new anchors, thereby avoiding any conflicts. Second, I don't like using underscore for escaping, because the result looks like it could be a different identifier. If we're going to have a character to designate upcase, why not have it be something that doesn't occur in normal names, so it's clear that it's not part of the name? Here's what we're allowed to use in xhtml for an id: Id ::= (Letter | '_' | ':') (NameChar)* NameChar ::= Letter | Digit | '.' | '-' | '_' | ':' | CombiningChar | Extender So how about using ":" to mark upcase letters? We could then leave lower-case letters and underscores untouched. Example cases: > original -> transformed Example url > > ---------------------------------------------------------------------- > - > MyName -> :My:Name http://foo/bar/SomeClass.html#:My:Name > _my_name -> _my_name http://foo/bar/SomeClass.html#_my_name > __init__ -> __init__ http://foo/bar/SomeClass.html#__init__ > MY_CONST -> :M:Y_:C:O:N:S:T http://foo/bar/ > SomeClass.html#:M:Y_:C:O:N:S:T Recall that this mangling is *only* used for anchors of functions, methods, & variables -- not for classes. So in the common case, where functions, methods, and vars are written in all lower case, the mangling will be an identity function. If we wanted to make the MY_CONST look better, at the expense of complexity, we could use these rule instead: - If name is all-caps, return name surrounded by ":" - Otherwise, prefix each capital letter w/ ":" Making the examples: > MyName -> :My:Name http://foo/bar/SomeClass.html#:My:Name > _my_name -> _my_name http://foo/bar/SomeClass.html#_my_name > __init__ -> __init__ http://foo/bar/SomeClass.html#__init__ > MY_CONST -> :MY_CONST: http://foo/bar/SomeClass.html#:MY_CONST: -Edward |
From: Paul P. <pog...@gm...> - 2007-02-19 22:24:07
|
Edward Loper wrote: > On Feb 18, 2007, at 7:04 PM, Daniele Varrazzo wrote: > > Edward, would you like to keep the current URL fragments with their > > small > > shortcomings or do you prefer to mangle fragments? I prefer to > > decide it > > before releasing the beta package, which is about ready. > > > > A simple naming scheme could be: > > > > - prefix each uppercase and underscore with an underscore > > - prefix each name with a letter (let's say N, for "name") > > Would this be done just for anchors, or for filenames as well? I > guess this depends on whether we are assuming that files are written > to a case-insensitive filesystem. I don't mind changing anchors too > much, but would rather not have the filenames become like N_My_Class- > class.html, if we can avoid it. I guess perhaps there could be a > command-line switch to control whether the name mangling applies to > just the anchors or to the entire URL -- perhaps something like "-- > case-insensitive-filesystem" to reflect its intended usage. > > Thoughts? Yes, please. Too much mangling is ugly. And in this case it is only for completeness sake, since for 99% of projects it will not matter. So, please make it optional (as proposed) or disabled at all. Paul |
From: Daniele V. <pi...@de...> - 2007-02-19 16:52:04
|
> On Feb 18, 2007, at 7:04 PM, Daniele Varrazzo wrote: >> Edward, would you like to keep the current URL fragments with their >> small >> shortcomings or do you prefer to mangle fragments? I prefer to >> decide it >> before releasing the beta package, which is about ready. >> >> A simple naming scheme could be: >> >> - prefix each uppercase and underscore with an underscore >> - prefix each name with a letter (let's say N, for "name") > > Would this be done just for anchors, or for filenames as well? I > guess this depends on whether we are assuming that files are written > to a case-insensitive filesystem. I think what most frequently happens is a class with the same name of a module (case sensitive or not) and they are already distinct files. Sincerely i haven't though about a module containing two classes "myclass" and "MyClass", which may be an issue in a non case sensitive system. Conversely that guy won't have a package containing "MyModule" and "mymodule" because they require two files with the same name (on case-insensitive system). Such a case couldn't even be used on Windows. All the reasoning about object labelling was only about anchors, which currently happen to be invalid (or ambiguous). I don't think we have a compelling reason to change the file names. > I don't mind changing anchors too > much, but would rather not have the filenames become like N_My_Class- > class.html, if we can avoid it. I guess perhaps there could be a > command-line switch to control whether the name mangling applies to > just the anchors or to the entire URL -- perhaps something like "-- > case-insensitive-filesystem" to reflect its intended usage. I'd avoid letting the users decide it, and avoid adding a command switch to control the behaviour. I think Epydoc should generate the same names on system with different respect for file name case. -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Edward L. <ed...@gr...> - 2007-02-19 16:31:47
|
On Feb 18, 2007, at 7:04 PM, Daniele Varrazzo wrote: > Edward, would you like to keep the current URL fragments with their > small > shortcomings or do you prefer to mangle fragments? I prefer to > decide it > before releasing the beta package, which is about ready. > > A simple naming scheme could be: > > - prefix each uppercase and underscore with an underscore > - prefix each name with a letter (let's say N, for "name") Would this be done just for anchors, or for filenames as well? I guess this depends on whether we are assuming that files are written to a case-insensitive filesystem. I don't mind changing anchors too much, but would rather not have the filenames become like N_My_Class- class.html, if we can avoid it. I guess perhaps there could be a command-line switch to control whether the name mangling applies to just the anchors or to the entire URL -- perhaps something like "-- case-insensitive-filesystem" to reflect its intended usage. Thoughts? -Edward |
From: Daniele V. <pi...@de...> - 2007-02-19 00:05:21
|
Hello, i just added Epydoc a system to reference to external API. I know i should have stopped adding features, but there is a pair of points i'd like to not ignore and implementing this module helped me to understand what can be done. The HTML pages generated by Epydoc currently suffer of a pair of issues: - URL fragments are not case sensitive, while Python names are: this may lead to ambiguous URLs (bug #1659522) - XHTML "name" attribute for anchors are actually copied to the "id" attribute, and XML id's can't start with an underscore (cfr. http://www.w3.org/TR/xhtml1/#C_8) so we should decide whether to keep the 1-1 identity from Python names to URL fragments (which creates non strictly valid URLs) or decide to drop such identity and create a function to mangle the names into valid fragments. With the provided tool, people could refer to documented objects without messing up with the URL details. We may then choose to generate URL's with uglier fragment but XML compliant. External API references can be used both in docstrings parsed by Epydoc (to refer to objects generated in other documentations) and in stand-alone documents to be transformed in HTML using the provided command line wrapper. It may also be used to refer to API documentation generated by other tools such Doxygen. While the redirect.html kinda solves the problem of long URLs, it still requires long names to access the object and doesn't solve the naming problems raised above. Furthermore uses Javascript to perform its job. What i like less of the current implementation are the command line options (described below). Suggestions about a better way to specify the external API options are welcome. The package has been tested with docutils 0.3.7 and 2007-01-28 snapshot. --------------------------------------------- Edward, would you like to keep the current URL fragments with their small shortcomings or do you prefer to mangle fragments? I prefer to decide it before releasing the beta package, which is about ready. A simple naming scheme could be: - prefix each uppercase and underscore with an underscore - prefix each name with a letter (let's say N, for "name") The mapping should be injective even considering non-case-sensitive comparisons. Examples: MyName -> N_My_Name _my_name -> N__my__name __init__ -> N____init____ If you want, i'll implement it and test it is used everywhere in the HTML writer. --------------------------------------------- Follows a more detailed description of the system, copied from the module docstring. This module allows a Docutils_ document to refer to elements defined in external API documentation. It is possible to refer to many external API from the same document. Each API documentation is assigned a new interpreted text role: using such interpreted text, an user can specify an object name inside an API documentation. The system will convert such text into an url and generate a reference to it. For example, if the API ``db`` is defined, being a database package, then a certain method may be referred as:: :db:`Connection.cursor()` To define a new API, an *index file* must be provided. This file contains a mapping from the object name to the URL part required to resolve such object. Index file ---------- Each line in the the index file describes an object. Each line contains the fully qualified name of the object and the URL at which the documentation is located. The fields are separated by a ``<tab>`` character. The URL's in the file are relative from the documentation root: the system can be configured to add a prefix in front of each returned URL. Allowed names ------------- When a name is used in an API text role, it is split over any *separator*. The separators defined are '``.``', '``::``', '``->``'. All the text from the first noise char (neither a separator nor alphanumeric or '``_``') is discarded. The same algorithm is applied when the index file is read. First the sequence of name parts is looked for in the provided index file. If no matching name is found, a partial match against the trailing part of the names in the index is performed. If no object is found, or if the trailing part of the name may refer to many objects, a warning is issued and no reference is created. Configuration ------------- This module provides the class `ApiLinkReader` a replacement for the Docutils standalone reader. Such reader specifies the settings required for the API canonical roles configuration. The same command line options are exposed by Epydoc. The script ``apirst2html.py`` is a frontend for the `ApiLinkReader` reader. API Linking Options:: --external-api=NAME Define a new API document. A new interpreted text role NAME will be added. --external-api-file=NAME:FILENAME Use records in FILENAME to resolve objects in the API named NAME. --external-api-root=NAME:STRING Use STRING as prefix for the URL generated from the API NAME. .. _Docutils: http://docutils.sourceforge.net/ -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Edward L. <ed...@gr...> - 2007-02-17 02:45:21
|
Daniele Varrazzo wrote: > Hello, > > now that many variables such as __version__ define modules metadata, wouldn't > it be better to drop them from being documented as variables when they are > recognized as tag alias? This would be fine with me. But if the value is an unexpected type, then epydoc should still list it under variables. -Edward |
From: Daniele V. <pi...@de...> - 2007-02-17 01:50:50
|
Hello, now that many variables such as __version__ define modules metadata, wouldn't it be better to drop them from being documented as variables when they are recognized as tag alias? -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Edward L. <ed...@gr...> - 2007-02-13 20:54:42
|
Daniele brought up the issue of improving epydoc's pretty-printing of object values. As of svn revision 1477, I've made most of the improvements that were suggested. In particular: * I refactored the pyval_repr() code, so that now syntax-highlighting is done in a central place (epydoc.markup.pyval_repr), and doesn't need to be repeated for each output format. In particular, APIDoc.pyval_repr() now returns a subclass of ParsedEpydocDocstring. * The new code does pretty-printing of lists, dicts, strings, tuples, sets, frozensets, and regexp objects. This pretty-printing should be faster than using the pprint module, in that it short-circuits as soon as the the output contains max_lines lines. It also does automatic line-wrapping. * str() is used instead of repr() for floats, ints, longs, and complex numbers. Other types could be added to this list, if desired. * When the pretty-printed representation is constructed, a score is calculated that evaluates "how useful" the representation is. If it's too low, then the parse-based representation is used instead (assuming it's available). In particular, if the string returned by an object's __repr__ has the form <.* at 0xaddr>, then epydoc will tend to use the parse-based repr instead. -Edward |
From: Daniele V. <pi...@de...> - 2007-02-11 00:34:46
|
Hello, in r1448 i added a fallback rule for the DocIndex.find() method: if a class name is looked for, and such class has not been imported into the namespace it is referred from, then look for it into the documented modules. The class is returned only if its name is not ambiguous: if there exist more classes with the same name, a warning is emitted. The implemented check is efficient: about a single non recursive scan of all the documented modules to create a map from names to classes, and a dict lookup for each name that would have failed with the previous find() implementation. The rationale behind this rule is that in Python you don't have to know an object class in order to use it, as long as you receive an instance and you trust the caller he sent you a duck with the right number of legs. So people ends up writing: class Bar: def shootProtons(self, foo): """Frobnicate the world :Parameters: `foo` : `Foo` the required frobnicator """ foo.shoot() It often happens that Foo is defined in another module. If Foo were not imported in the module defining Bar, Epydoc would have failed linking `Foo` to the Foo class documentation. To overcome this it is possible to import the object in the namespace: from foomodule import Foo but if foomodule happens to use objects defined in the barmodule, there would be a cross modules import doomed to fail. Another alternative is to import the containing module: import foomodule but then the class should be referred as `foomodule.Foo` and, in case of refactoring all the references were to be updated; Furthermore the modules would be paired even if the code wouldn't require it. Yet another alternative would be to import all the classes in the __init__.py of the main package, but often you don't *need* them there, and this would block lazy imports. In a more generic sense, Epydoc should not ask the source to be adapted to be handled: it's fine to adapt docstrings, but code shouldn't be affected. -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Daniele V. <pi...@de...> - 2007-02-10 22:33:57
|
>> - docutils :python: directive to allow examples to appear as colored >> Python code > > That would be fine. Is there a reason not to use doctest block? They > already get colored. The "python" directive has been added into the trunk. It uses the recently refactored HTMLDoctestColorizer to perform its job. An example: .. python:: # This is some Python code def foo(): pass class Foo: def __init__(self): pass -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Daniele V. <pi...@de...> - 2007-02-10 19:35:47
|
Hello, i just updated the Epydoc trunk to drop detail boxes for function, properties and variables if all the details available are expressed in the summary. When the summary is extracted from the docstring, an extra flag reports if the docstring is longer than what returned in the summary (the test depends on the used markup), in which case the extra box is required. The extra box is also required if there is any field attached to the object (except the type/return type, which fit on the summary line), or if the Python value has a long representation. I also changed what pyval_repr() returns: now it integrates both parsed and introspected representations (the test used to be carried by the writers). The value representation is now more consistent and the function is a good entry point to refine the representation of Python values. -- Daniele Varrazzo - Develer S.r.l. http://www.develer.com |
From: Edward L. <ed...@gr...> - 2007-02-09 18:18:28
|
On Feb 6, 2007, at 8:14 AM, Daniele Varrazzo wrote: >> I fiddled with the HTML output and the default Epydoc css trying to >> optimize the vertical space. A result can be seen at >> http://www.develer.com/~piro/api-compact/epydoc.apidoc.APIDoc- >> class.html >> [...] Sorry, I guess I missed this message the first time around. This looks like an improvement on the current code, and I see no problems with merging the compat branch into the trunk. I think that the reason I used <dl> rather than <div>+css is that when I started writing epydoc, css was not yet supported by several common browsers; so I wanted to generate HTML code that would display well on older browsers. Now that several years have passed, I don't think that's an issue any more -- all modern browsers should support at least basic css. So feel free to update the html writer to generate output that can be customized more easily w/ css. As for the question of how compact the default output should be, I agree that the current output has a pretty large amount of whitespace in the output, but the api-compact page you pointed at seems a little *too* tight to me. Perhaps we could change the default to be somewhat intermediate between the current spacing and your 'compact' spacing, and provide stylesheets with both more-compact and less- compact variants? -Edward |
From: Edward L. <ed...@gr...> - 2007-02-07 02:07:59
|
Daniele Varrazzo wrote: > I am trying to add a directive to parse a docstring such as the following, > generating a box with colored syntax:: [...] I believe this should do what you want: def run(self): self.assert_has_content() text = '\n'.join(self.content) node = docutils.nodes.doctest_block(text, text) return [ node ] (The class doctest_block inherits its constructor from FixedTextElement. The first arg of the constructor gives the rawsource, and the second gives the text content.) You should only be using nested_parse if the directive contains something that is formatted using rst syntax. -Edward |
From: Daniele V. <dan...@gm...> - 2007-02-07 01:52:08
|
Hello, >> - docutils :python: directive to allow examples to appear as colored >> Python code > > That would be fine. Is there a reason not to use doctest block? They > already get colored. I am trying to add a directive to parse a docstring such as the following, generating a box with colored syntax:: class Foo: """A class like nothing else. This is a test: .. code:: python def foo(): '''Hello world''' for i in range(10): print i """ i'd like to generate a doctest block, so that the visitors would already work. No luck up to now: i tried to follow the example in [1], but it fails. The directive i added is:: class CodeDirective(Directive): required_arguments = 0 optional_arguments = 1 has_content = True def run(self): self.assert_has_content() text = '\n'.join(self.content) node = docutils.nodes.doctest_block(rawsource=text) self.state.nested_parse(self.content, self.content_offset, node) return [ node ] directives.register_directive('code', CodeDirective) but trying to parse the above snippet i receive errors "Unexpected indentation". It seems that "nested_parse" parses the snippet:: def Foo(): '''Hello world''' for i in range(10): print i as it were ordinary text, not a block, so indentations don't make sense for it. It should really be parsed as a literal block. I checked the docutils.parsers.rst.states module, but didn't find any promising method to "parse" a literal block (for which, i know, there's not much to parse) Any hint? Thanks -- Daniele [1] http://docutils.sourceforge.net/docs/howto/rst-directives.html#admonitions |
From: Daniele V. <dan...@gm...> - 2007-02-06 13:15:29
|
Edward Loper ha scritto: > Daniele Varrazzo wrote: >> what about setting a laundry list of things we want to be fixed to roll >> out a beta version of Epydoc 3.0? > > Sounds good. One item is to go through the bug list & respond to all > the bug reports -- it looks like you've already been making some good > progress on that front. > > Unfortunately, I probably won't have much time to work on epydoc until > the end of February, since I'm trying to finish up my phd proposal. But > I'm happy letting you take the lead in putting together the beta and/or > 3.0 release, if you're up for it. > >> Here is a laundry list of things i'd like to discuss [...] > > All of these sound like good features. I don't see any of them as being > necessary for the 3.0 release, but I'd be happy to see any of them added. I also thought that saying "i want the features x and y" and "i want 3.0 rolled out" means going nowhere :) so, many "bright ideas" are probably perfect... for the release 3.1. Getting back to 3.0: >> - pretty printing (see also SF #1628044). How to represent an object >> value? I don't expect this being a showstopper for 3.0, and best result can be achieved incrementally. >> - parsing some "#:" comments into functions body as they were part of >> the function docstring. [...] > > This is a little bit problematic, because it would be hard to tell [...] 3.1 :) >> Some tags (note, bug, warning, todo) are really best placed next the >> code to be documented; precondition and postcondition would be nice >> close to an assert etc. > > True, but I'm not sure how to reconcile this with the ability to use > "#:" comments to describe variables. 3.1! >> - using some module __variable__ as a tag (only the ones widely used: >> __version__ above all, but we may be friendly with __author__ etc; >> people has already been warned not to abuse of such variables, though >> i couldn't find a reference) > > This would be fine. It would be nice if this could be customized, sort > of how @field names can currently be added with @deffield. I'll start with a map variable name -> tag, then think about its customization. >> - docutils :python: directive to allow examples to appear as colored >> Python code > > That would be fine. Is there a reason not to use doctest block? They > already get colored. I thought doctest blocks require >>> and ... things in front of the lines. I already wanted to leverage markup.doctest: i'll check if there is some refactoring to perform in order to add the directive. >> - :api: role to allow docutils documents to refer to Epydoc generated >> API. See for example >> http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/scripts/ext2html.py >> > > This would need some sort of configuration to tell epydoc where that > :api: role should point to. Would this be done in the config > file/command line? Probably a command line switch for a wrapper around rst2html >> - more compact html code, for which the exp-compact-html branch is >> open. In some personal audits i had, somebody complained about the >> signal/noise ratio in HTML output. > > Can you point me at an example page that demonstrates the changes made > by the branch? Mmm.. i sent you a couple of mail that got unnoticed back on september 2006. Anyway... Quoting such message: > I fiddled with the HTML output and the default Epydoc css trying to > optimize the vertical space. A result can be seen at > http://www.develer.com/~piro/api-compact/epydoc.apidoc.APIDoc-class.html > > I'd like your opinion about the supposed problem and the proposed layout. > > Notice that i had to update not only the css, but also some html code. > Actually it looks to me kinda "old", with many alignment tricks that > nowadays can be better accomplished using CSS. For example, there are > many definitions list with empty definitions, i guess only used to > increase the left margin. I cleaned up them, minimizing the number of > warnings issued by Tidy, and using CSS to reposition them. > > For example i replaced > > <dl><dt></dt><dd> > <p><strong>Warning:</strong> > This method uses undocumented data structures inside of > <code>profile_stats</code>. > </p> > </dd></dl> > > with > > <div class="fields"> > <p><strong>Warning:</strong> > This method uses undocumented data structures inside of > <code>profile_stats</code>. > </p> > </div> > > (attached, the full patch up to now) > > If you prefer to keep the pages appearance as is (wide), then we could > provide an optional "compact" CSS. Anyway, to allow wider CSS > positioning freedom, some html cleanup may be required (using more > divs and spans instead of p's and tables, for example) The API i generated then are available at: http://www.develer.com/~piro/api/epydoc.apidoc.APIDoc-class.html http://www.develer.com/~piro/api-compact/epydoc.apidoc.APIDoc-class.html >> Together with setting a goal in terms of features, i'd love to QA Epydoc >> against the most prominent OS projects, such TurboGears (i just read >> they are having documentation issues - cfr. >> http://groups.google.it/group/turbogears/browse_thread/thread/ae0e52ff3f9fde60?hl=en >> for example). > > This would be excellent, and *very* much appreciated. In fact, I think > it might be a good idea to set up some scripts on a server somewhere to > automatically run epydoc on various projects, to at least make sure that > it doesn't crash. I have a server I could probably put this on (I don't > think there'd be enough space on the sourceforge server -- there's a > 100mb quota). This would be something like the python buildbot, but > simpler of course. :) The society i'm currently working for is an enthusiast sustainer of open source software and will be glad to offer web space to host such results. >> A list of beasts to test against may include: >> >> - WxPython >> - Django >> - Turbogears >> - Twisted >> - SqlAlchemy >> - docutils > > Sounds great. Of course the python stdlib should be included too. :) Of course, the Python lib and Epydoc itself are in the list :) > Once we do the beta release, we should make a point of encouraging > people to try it out on their project & get back to us about what they > think of the results. My plan is now: - attack the remaining bugs. I'll put the older ones in pending state asking the reporters to check out the trunk and verify if they magically disappeared: if not, the bug will be fixed. - Open a 3.1 group in the feature request tracker and move there what doesn't stop us to release a good 3.0. In the tracker i will add items from: - the TODO list in epydoc.__init__.py - my laundry list items (together with the comments you did about them) and categorize them and the currently open requests splitting between 3.0 and 3.1. - Close the 3.0 requests and declare "beta" - Spend some time parsing other open source projects and publishing the results, contributing back a cfg file to let them generate the docs by themselves - If no big issue raises from such phase, go 3.0 - Enjoy - Attacking the request list for 3.1... Regards, Daniele |
From: Daniele V. <dan...@gm...> - 2007-02-06 02:55:51
|
Saint Germain ha scritto: >> 1. can be cargo-culted from the LaTeX writer, but this probably would >> lead to more maintenance burden. Probably a nicer work would be to >> refactor the LaTeX writer creating a base class implementing the >> strategy (e.g. write the start; for each class: write it; write the >> end...) of creating a document from the parsed documentation (as >> "document" i mean something that can be read from the beginning to the >> end, which is very different from the hypertext created by the html >> writer) but delegating the details of how to write the leaves (such a >> paragraph or a string) to concrete subclasses (cfr. the strategy >> pattern). Such base class may be subclasses into concrete writers for >> LaTeX, Docbook, plain text (which is currently less maintained than >> the other writers), and going on with single page html, reST... > > Seems reasonable. > Does that mean that currently the LaTeX and html writers are completely > independant ? Yes, they are. They actually create very different output: html creates an hypertext with many index page summarizing different facets of a package (the modules, the classes, all the names...) and each module creates a matching html page, with links to other page with annotated and colored code and some javascript thrown in. The LaTeX writer (of which i don't have a good insight: i rarely used it and worked on its source) also creates many files, but tied together into a document suitable for printing. There are some common services that both LaTeX and html could benefit, but i can see much more similarities between two writers of linear documents such LaTeX and Docbook would be. > And how is currently built the LaTeX writer ? Do you just throw the > LaTeX markups one after another in a procedural way ? I'll try to answer, but i don't know if i correctly understood your question. The writer (both LaTeX and others) receives a DocIndex instance holding the whole content of the analysis Epydoc carried on the code which has been fed to it. It's the writer responsibility to decide what to use of the whole informations bulk and how: the writer is a viewer of the model in the DocIndex instance. The LaTeX document generation is directed by the write() method, which creates a top-level file and then iterates over the DocIndex node running the proper generation function for each class and module it meets. In this "generation strategy" there is almost no LaTeXisms (about only the generated ".tex" files extensions smell like LaTeX). The functions called by the write() method on selected detail decide what to write about such detail; so for a "class" there is an iteration on its methods, and for each of them a proper write_something() function is called. Apart from sporadic snippets of LaTeX code you can find here and there, the navigation in the DocIndex nodes basically the same you would carry on to generate a Docbook output (assuming you want to put the same information in such documents of course). The leaves function called during the index navigation are the ones where you will find most of the \stuff{\like\this} you can expect in a LaTeX generator, which should be replaced by <stuff><like/><this/></stuff> to create a Docbook document. > Well I would have started even without a refactoring, just for the fun > of it but it's better to wait and see if you want to refactor first... Then, please, start and have a good time while coding :) Please, don't wait a refactoring movement pouring from above. Mainly because there's no point in creating a base class for a single subclass: it would be programming in the vacuum. Instead trying to adapt the LaTeX writer into a Docbook writer would give you a precise idea of what is common to both writers and what is specific to each one. I'd effectively proceed this way: - copy the current LatexWriter class into a LinearDocumentWriter class. - create an empty subclass DocbookWriter(LinearDocumentWriter) - walk the code from the entry point write(): each time you stumble into a latex output string, translate such output into Docbook idiom, but put the updated strings into the DocbookWriter. You will end up with a base class dispatching action into a concrete subclass, which is actually an implementation of the strategy design pattern See for example LatexWriter.write_class() method: it's about independent from the output format, except for a single statement: # Label our current location. out(' \\label{%s}\n' % self.label(doc)) You may replace it with a call: self.write_current_location(out, self.label(doc)) and write a matching implementation in the Docbook subclass: def write_current_location(self, out, label): # pretending i know Docbook markup... out(' <a name="%s" />\n' % label) Where to dispose all that messy latex strings? Probably another concrete subclass would be the ideal bin :) Hope this helps. Feel free to write if you need help. Regards, Daniele |
From: Saint G. <sai...@gm...> - 2007-02-06 02:04:51
|
On Mon, 5 Feb 2007 17:41:43 +0100, "Daniele Varrazzo" <dan...@gm...> wrote : > > I'm currently an active user of Python and XML Docbook and would > > like to have my Python documentation generated in XML Docbook. > > > > Epydoc seems to be a really fine documentation generator and I would > > like to help you add an XML Docbook output. > A sketch of the epydoc structure and where you may want to put your > hands if you want to implement is the following:: > > Markups Writers > > epytext | | LaTeX > javadoc | -> build -> | HTML > plaintext | | plaintext > restructuredtext | > > Each markup is implemented by a ParsedDocstring subclass. Writers are > more free-form beasts: there is no base class for writers, whose > interface is a single function called by the cli.py module after > options have been parsed and a docindex generated. Ok I understand. > The ParsedDocstring interface offers methods to retrieve a docstring > in the target output format: currently methods to_plaintext(), > to_html(), to_latex() methods are exposed. The concrete subclasses are > responsible to implement such methods (while a fallback must be > provided by the base class). > > Writing a Docbook generator should require the following steps: > 1. add a new writer implementing the sequences of calls to observe the > built set of documents; > 2. add a new method ParsedDocstring.to_docbook() implementing a > default behavior, which typically is to return a a few more than the > text version; > 3. implement the details of converting the specific markups into > Docbook format. > > 2. should be easy. You may wrap the to_plaintext() output into > <programlisting> tags, i guess; see ParsedDocstring.to_html() for an > example. Ok it's a start > 1. can be cargo-culted from the LaTeX writer, but this probably would > lead to more maintenance burden. Probably a nicer work would be to > refactor the LaTeX writer creating a base class implementing the > strategy (e.g. write the start; for each class: write it; write the > end...) of creating a document from the parsed documentation (as > "document" i mean something that can be read from the beginning to the > end, which is very different from the hypertext created by the html > writer) but delegating the details of how to write the leaves (such a > paragraph or a string) to concrete subclasses (cfr. the strategy > pattern). Such base class may be subclasses into concrete writers for > LaTeX, Docbook, plain text (which is currently less maintained than > the other writers), and going on with single page html, reST... Seems reasonable. Does that mean that currently the LaTeX and html writers are completely independant ? And how is currently built the LaTeX writer ? Do you just throw the LaTeX markups one after another in a procedural way ? > The step 3. could require more knowledge of the single markups. On the > pro side it can be accomplished gradually, because there is always a > fallback that would appear as monotype text. The current to_html() > implementation would help you of course. That step could be quite long but rather easy : Docbook markup are really clear and there are no subtles/magics as with LaTeX. > I'd ask Edward if he would welcome the refactoring described in step > 2. I think that creating a Docbook writer the naive way would lead to > harder maintenance, inconsistencies between formats and to too much > code duplication. I of course agree. I can help a few hours (let's say 3-5) per week at most. > Let me now if i can help you, but i'd like to hear Ed's advice first. Well I would have started even without a refactoring, just for the fun of it but it's better to wait and see if you want to refactor first... Regards, |
From: Edward L. <ed...@gr...> - 2007-02-06 02:01:04
|
On 2/1/07, Saint Germain <sai...@gm...> wrote: >> I know Python, XML Docbook and LaTeX and I'm a heavy user of the >> dblatex project (Docbook to LaTeX, and it's written in Python !) : >> http://dblatex.sourceforge.net >> >> Would you like me to add such a feature and if yes can someone help to >> guide me through Epydoc ? If you haven't already, you should read through the front page of the epydoc API docs: http://epydoc.sourceforge.net/api/ It provides a brief overview of how epydoc is structured. Daniele Varrazzo wrote: > If you want to create a Docbook generator, the best starting point is > largely the LaTeX generator. Agreed. And feel free to suggest changes to the LaTeX generator -- it doesn't get nearly as much use as the HTML generator, and so it isn't as well developed. Daniele's description of how you should go about adding a new docwriter should be enough to get you started, but if you get stuck or have further questions, you can certainly email us. Daniele Varrazzo wrote: > 1. can be cargo-culted from the LaTeX writer, but this probably would > lead to more maintenance burden. Probably a nicer work would be to > refactor the LaTeX writer [...] > > I'd ask Edward if he would welcome the refactoring described in step > 2. I think that creating a Docbook writer the naive way would lead to > harder maintenance, inconsistencies between formats and to too much > code duplication. I think this type of refactoring would be a good idea. I expect the LaTeX and Docbook outputs to be structured very similarly. And there may be other output formats that would also share similar structure. -Edward |
From: Edward L. <ed...@gr...> - 2007-02-06 01:50:59
|
Daniele Varrazzo wrote: > what about setting a laundry list of things we want to be fixed to roll > out a beta version of Epydoc 3.0? Sounds good. One item is to go through the bug list & respond to all the bug reports -- it looks like you've already been making some good progress on that front. Unfortunately, I probably won't have much time to work on epydoc until the end of February, since I'm trying to finish up my phd proposal. But I'm happy letting you take the lead in putting together the beta and/or 3.0 release, if you're up for it. > Here is a laundry list of things i'd like to discuss [...] All of these sound like good features. I don't see any of them as being necessary for the 3.0 release, but I'd be happy to see any of them added. > - exclusion list (SF #1209634) would help big project to adopt Epydoc > (just committed in the trunk a first version) This definitely seems like a good thing. > - pretty printing (see also SF #1628044). How to represent an object > value? > - repr() is fine only for int and for classes that specifically > return a parsable representation, such as datetime. Even floats > (such as 0.1) often have an ugly representation. Yeah -- people are very inconsistent about how they use repr/str. > - fall back on the parsed version if available and the repr() result > matches an ugly pattern looks like "<something>"? Sometimes the parsed version isn't very trustworthy.. E.g., if they create a variable with a list, and then append to it, etc. But you may be right that this may be the best way to go when you get "<something>". It's almost certainly the best way if you get "<xyz instance at 0xaddr>". > - fall back on str()ingification? Yeah, it might be good to use str() for some values. > - prettyprinting of lists, dicts etc? More clever truncation of long > representations? Seems like good ideas. > - if a variable value fits on a single row, omit the detail box? Sure. > - ??? One issue/question is that right now a good amount of this is handled in specific docwriter modules.. E.g., the docwriter.html code to colorize regexps. Should this be abstracted away somehow and handled more centrally? E.g., the centralized code could generate an abstract structure saying how to color the regexp, and the individual writers would just translate that to the appropriate markup? > - parsing some "#:" comments into functions body as they were part of > the function docstring. It would keep some additional information > close to the code implementing it, making the task to keep the API > consistent with the code easier. Es. This is a little bit problematic, because it would be hard to tell the difference between "#:" comments that are supposed to describe the enclosing object vs "#:" comments that are supposed to describe a variable assignment. E.g., I think people would do things like: def __init__(self, foo): """This is a constructor of some sort.""" ...there may be other code #: :warning: the current implementation only allows #: positive numbers val = floor(log(num) / log(10)) + 1 ... or: class A: "..." ... other code #: :warning: ... x = 123 Where it's unclear whether whether that warning applies to the constructor function or to the 'val' variable. > Some tags (note, bug, warning, todo) are really best placed next the > code to be documented; precondition and postcondition would be nice > close to an assert etc. True, but I'm not sure how to reconcile this with the ability to use "#:" comments to describe variables. > - using some module __variable__ as a tag (only the ones widely used: > __version__ above all, but we may be friendly with __author__ etc; > people has already been warned not to abuse of such variables, though > i couldn't find a reference) This would be fine. It would be nice if this could be customized, sort of how @field names can currently be added with @deffield. > - docutils :python: directive to allow examples to appear as colored > Python code That would be fine. Is there a reason not to use doctest block? They already get colored. > - :api: role to allow docutils documents to refer to Epydoc generated > API. See for example > http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/scripts/ext2html.py This would need some sort of configuration to tell epydoc where that :api: role should point to. Would this be done in the config file/command line? > - more compact html code, for which the exp-compact-html branch is > open. In some personal audits i had, somebody complained about the > signal/noise ratio in HTML output. Can you point me at an example page that demonstrates the changes made by the branch? > Together with setting a goal in terms of features, i'd love to QA Epydoc > against the most prominent OS projects, such TurboGears (i just read > they are having documentation issues - cfr. > http://groups.google.it/group/turbogears/browse_thread/thread/ae0e52ff3f9fde60?hl=en > for example). This would be excellent, and *very* much appreciated. In fact, I think it might be a good idea to set up some scripts on a server somewhere to automatically run epydoc on various projects, to at least make sure that it doesn't crash. I have a server I could probably put this on (I don't think there'd be enough space on the sourceforge server -- there's a 100mb quota). This would be something like the python buildbot, but simpler of course. :) > I was very sad when i read (just a couple of weeks ago, but the issues > were older) how Twisted guys were disappointed by Epydoc no more > fulfilling their needs, up to decide to write their own API > generator (http://codespeak.net/~mwh/pydoctor/). That's too bad. :-/ Hopefully once 3.0 comes back, we can win back some of the projects that moved on to other tools. > I'd like to check Epydoc against a list of packages, including some not > specifically thought to be inspected by Epydoc. The goal is: > - Epydoc must not crash (modulo the most magic corners, which now can > be removed from the list of modules to be introspected) > - The generation result shall be presented to the packages developers, > both to gather feedback about Epydoc output and to advertise Epydoc > to the community. Eventually making people happy to write > documentation... > > A list of beasts to test against may include: > > - WxPython > - Django > - Turbogears > - Twisted > - SqlAlchemy > - docutils Sounds great. Of course the python stdlib should be included too. :) Once we do the beta release, we should make a point of encouraging people to try it out on their project & get back to us about what they think of the results. -Edward |
From: Daniele V. <dan...@gm...> - 2007-02-05 16:42:44
|
On 2/1/07, Saint Germain <sai...@gm...> wrote: > Hello, > > I'm currently an active user of Python and XML Docbook and would > like to have my Python documentation generated in XML Docbook. > > Epydoc seems to be a really fine documentation generator and I would > like to help you add an XML Docbook output. > > I know Python, XML Docbook and LaTeX and I'm a heavy user of the > dblatex project (Docbook to LaTeX, and it's written in Python !) : > http://dblatex.sourceforge.net > > Would you like me to add such a feature and if yes can someone help to > guide me through Epydoc ? > > If the LaTeX output is OK, I can just analyse this part and convert it > in XML Docbook ? If you want to create a Docbook generator, the best starting point is largely the LaTeX generator. Honestly i've not been using the latter generator as often as the HTML generator, but it is stable enough to generate at least documentation for Epydoc itself. Furthermore most of the hard work (parsing the distinct pieces and merging stuff together) is carried out before even reaching the docwriter. A sketch of the epydoc structure and where you may want to put your hands if you want to implement is the following:: Markups Writers epytext | | LaTeX javadoc | -> build -> | HTML plaintext | | plaintext restructuredtext | Each markup is implemented by a ParsedDocstring subclass. Writers are more free-form beasts: there is no base class for writers, whose interface is a single function called by the cli.py module after options have been parsed and a docindex generated. The ParsedDocstring interface offers methods to retrieve a docstring in the target output format: currently methods to_plaintext(), to_html(), to_latex() methods are exposed. The concrete subclasses are responsible to implement such methods (while a fallback must be provided by the base class). Writing a Docbook generator should require the following steps: 1. add a new writer implementing the sequences of calls to observe the built set of documents; 2. add a new method ParsedDocstring.to_docbook() implementing a default behavior, which typically is to return a a few more than the text version; 3. implement the details of converting the specific markups into Docbook format. 2. should be easy. You may wrap the to_plaintext() output into <programlisting> tags, i guess; see ParsedDocstring.to_html() for an example. 1. can be cargo-culted from the LaTeX writer, but this probably would lead to more maintenance burden. Probably a nicer work would be to refactor the LaTeX writer creating a base class implementing the strategy (e.g. write the start; for each class: write it; write the end...) of creating a document from the parsed documentation (as "document" i mean something that can be read from the beginning to the end, which is very different from the hypertext created by the html writer) but delegating the details of how to write the leaves (such a paragraph or a string) to concrete subclasses (cfr. the strategy pattern). Such base class may be subclasses into concrete writers for LaTeX, Docbook, plain text (which is currently less maintained than the other writers), and going on with single page html, reST... The step 3. could require more knowledge of the single markups. On the pro side it can be accomplished gradually, because there is always a fallback that would appear as monotype text. The current to_html() implementation would help you of course. I'd ask Edward if he would welcome the refactoring described in step 2. I think that creating a Docbook writer the naive way would lead to harder maintenance, inconsistencies between formats and to too much code duplication. Let me now if i can help you, but i'd like to hear Ed's advice first. Good luck! Daniele |
From: Daniele V. <dan...@gm...> - 2007-02-05 03:52:33
|
ulimit -c unlimited... Hello Edward, what about setting a laundry list of things we want to be fixed to roll out a beta version of Epydoc 3.0? I think the greatest part of the work is done: Epydoc is already something that does things no other Python API generator does - by far (mostly thanks to all your efforts into the parsing/introspecting system). Here is a laundry list of things i'd like to discuss AND to code if this would concretely bring us near to the release. Feel free to add/delete/update items (all the __methods__ have been implemented :) - exclusion list (SF #1209634) would help big project to adopt Epydoc (just committed in the trunk a first version) - pretty printing (see also SF #1628044). How to represent an object value? - repr() is fine only for int and for classes that specifically return a parsable representation, such as datetime. Even floats (such as 0.1) often have an ugly representation. - fall back on the parsed version if available and the repr() result matches an ugly pattern looks like "<something>"? - fall back on str()ingification? - prettyprinting of lists, dicts etc? More clever truncation of long representations? - if a variable value fits on a single row, omit the detail box? - ??? - parsing some "#:" comments into functions body as they were part of the function docstring. It would keep some additional information close to the code implementing it, making the task to keep the API consistent with the code easier. Es. def getDigitsNumber(num): """Return the number of digits of a number.""" ...there may be other code #: :warning: the current implementation only allows #: positive numbers return floor(log(num) / log(10)) + 1 Some tags (note, bug, warning, todo) are really best placed next the code to be documented; precondition and postcondition would be nice close to an assert etc. - using some module __variable__ as a tag (only the ones widely used: __version__ above all, but we may be friendly with __author__ etc; people has already been warned not to abuse of such variables, though i couldn't find a reference) - docutils :python: directive to allow examples to appear as colored Python code - :api: role to allow docutils documents to refer to Epydoc generated API. See for example http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/scripts/ext2html.py - more compact html code, for which the exp-compact-html branch is open. In some personal audits i had, somebody complained about the signal/noise ratio in HTML output. Together with setting a goal in terms of features, i'd love to QA Epydoc against the most prominent OS projects, such TurboGears (i just read they are having documentation issues - cfr. http://groups.google.it/group/turbogears/browse_thread/thread/ae0e52ff3f9fde60?hl=en for example). I was very sad when i read (just a couple of weeks ago, but the issues were older) how Twisted guys were disappointed by Epydoc no more fulfilling their needs, up to decide to write their own API generator (http://codespeak.net/~mwh/pydoctor/). I'd like to check Epydoc against a list of packages, including some not specifically thought to be inspected by Epydoc. The goal is: - Epydoc must not crash (modulo the most magic corners, which now can be removed from the list of modules to be introspected) - The generation result shall be presented to the packages developers, both to gather feedback about Epydoc output and to advertise Epydoc to the community. Eventually making people happy to write documentation... A list of beasts to test against may include: - WxPython - Django - Turbogears - Twisted - SqlAlchemy - docutils Daniele ...core dumped |
From: Daniele V. <dan...@gm...> - 2007-02-05 01:16:21
|
Hello Edward, i added to the Epydoc docbuilder the possibility to skip parsing and introspection of submodules whose dotted name matches a given regular expression pattern. While working at the patch, i noticed that it is useful to specify two distinct patterns: one for the modules not to be parsed and one for the modules not to be introspected. The reason is that too much magic modules can wreck havoc in the built documentation state if imported. For example, while i was testing the Twisted package API generation, i have been struggling with an object (SelectReactor) reaching the indexing phase without a canonical_name assigned... until i noticed the very peculiar 'twisted.internet.reactor' module, whose content can be summarized into: del sys.modules['twisted.internet.reactor'] #from twisted.python import log #log.msg("Installing SelectReactor, since unspecified.") from twisted.internet import selectreactor selectreactor.install() This module can be parsed without any problem; if imported instead it tries to saw the branch it is sitting on, doing no good to Epydoc state. In my current working copy, the Twisted API can be completely generated using the command line: epydoc twisted \ --exclude-introspect="\.test|\.internet\.reactor" \ --exclude-parse="\.test" But now there are 4 different options, all dealing with what to elaborate: --introspect-only, --parse-only and the newly added --exclude-introspect and --exclude-parse. I propose to merge them into two options: --no-parse and --no-introspect, both with an optional parameter being the pattern to be excluded. If the pattern is omitted, --no-parse works as --introspect-only and vice-versa. IIRC optparse doesn't natively handle options with an optional parameter, but it is easy to extend it with an option callback [1]: i already did stuff like this, es. in http://tinyurl.com/226mtj I can keep --introspect-only and --parse-only as undocumented options doing the right thing but issuing a warning (we still have time to alter the command line syntax while in alpha phase, don't we?). Regards, Daniele [1] http://docs.python.org/lib/optparse-option-callbacks.html |
From: Saint G. <sai...@gm...> - 2007-02-01 00:26:11
|
Hello, I'm currently an active user of Python and XML Docbook and would like to have my Python documentation generated in XML Docbook. Epydoc seems to be a really fine documentation generator and I would like to help you add an XML Docbook output. I know Python, XML Docbook and LaTeX and I'm a heavy user of the dblatex project (Docbook to LaTeX, and it's written in Python !) : http://dblatex.sourceforge.net Would you like me to add such a feature and if yes can someone help to guide me through Epydoc ? If the LaTeX output is OK, I can just analyse this part and convert it in XML Docbook ? Thanks in advance, |
From: Daniele V. <dan...@gm...> - 2007-01-30 11:36:41
|
On 1/29/07, Paul Pogonyshev <pog...@gm...> wrote: > Can you please apply this patch? Done, thanks Daniele |
From: Paul P. <pog...@gm...> - 2007-01-29 22:47:39
|
Can you please apply this patch? Paul Index: epydoc/src/setup.py =================================================================== --- epydoc/src/setup.py (revision 1427) +++ epydoc/src/setup.py (working copy) @@ -1,4 +1,4 @@ -#!/usr/local/bin/python +#! /usr/bin/env python # # Distutils setup script for the Natural Language # Processing Toolkit |