Re: [Numpy-discussion] generalized_inverse and documentation in general

[Jon W. <wr...@es...>]

Keith Goodman wrote:

>I prfr shrtr nams lke inv eig and sin.
>  
>
Dwn wth vwls!! Srsly thgh:

 >>> from numpy import linalg
 >>> help(linalg.inv)
Help on function inv in module numpy.linalg.linalg:

inv(a)

 >>> ???

While I prefer inverse to inv, I don't really care as long as the word 
"inverse" appears in the docstring and it faithfully promises that it is 
going to try to do an inverse in the matrix linear algebra sense and not 
in the one_over_x or power_minus_one sense or bit inversion (~ x, 
numpy.invert) sense. It would be handy to know which exception which 
will be raised for a singular matrix, what are valid arguments (eg, 
square shape, numbers), what is the type of the output ('<f8' for me) 
and what is the expected cost (O(n^3)). Other handy information - like a 
pointer to pinv and matrix.I and a note that this is implemented by 
"solve"ing Identity = b = A.x. The docs for solve should indicate they 
use an LU factorisation in lapack as this is of interest (ie it is not 
the matlab \ operator and not cholesky).  If you have all that then I 
suspect people might accept any even vaguely sane naming scheme, as they 
don't have to guess or read the source to find out what it actually 
does. The documentation for "invert" is for a generic ufunc in my 
interpreter (python 2.4.3 and numpy 0.9.8) which could cause confusion - 
one might imagine it is going to invert a matrix. The ".I" attribute 
(instead of ".I()") of a matrix implies to me that the inverse is 
already known and is an O(1) lookup - this seems seriously misleading - 
it seems you need to store the temporary yourself if you want to reuse 
the inverse without recomputing it, but maybe I miss some deep magic?

I did buy the book, and it doesn't contain all the information about 
"inv" that I've listed above, and I don't want Travis to spend his time 
putting all that in the book as then it'd be too long for me to print 
out and read ;-) I think numpy would benefit from having a great deal 
more documentation available than it does right now in the form of 
docstrings and doctests - or am I the only person who relies on the 
interpreter and help(thing) as being the ultimate reference?

This is an area that many of us might be able to help with fixing, via 
the wiki for example. Has there been a decision on the numpy wiki 
examples not being converted to doctests? (these examples could usefully 
be linked from the numeric.scipy.org home page). I saw "rundocs" in 
testing.numpytest, which sort of suggests the possibility is there. With 
the wiki being a moving target I can understand that synchronisation is 
an issue, but perhaps there could be a wiki dump to the numpy.doc 
directory each time a new release is made, along with a doctest? 
Catching and fixing misinformation would be as useful as catching actual 
bugs.

Let me know if you are against the "convert examples to doctests" idea 
or if it has already been done. Perhaps this increases the testsuite 
coverage for free...? It would be equally possible place that example 
code into the actual docstrings in numpy, along with more detailed 
explanations which could also be pulled in from the wiki. I don't think 
this would detract much from Travis' book, since that contains a lot of 
information that doesn't belong in docstrings anyway?

Jon

PS: Doubtless someone might do better, but here is what I mean:

copy and paste the ascii (editor) formatted wiki text into a file 
wiki.txt from the wiki example page and get rid of the {{{ python 
formatting that confuses doctest:

$ grep -v "{{{" wiki.txt | grep -v "}}}"  > testnumpywiki.txt

==testwiki.py:==
import doctest
doctest.testfile("testnumpywiki.txt")

$ python testwiki.py > problems.txt

problems.txt is 37kB in size (83 of 1028)

throwing out the blanklines issues via:
doctest.testfile("testnumpywiki.txt", 
optionflags=doctest.NORMALIZE_WHITESPACE)
reduces this to 24kB (62 of 1028).
...

most cases are not important, just needing to be fixed for formatting on 
the wiki or flagged as version dependent, but a few are worth checking 
out the intentions, eg:
**********************************************************************
File "testnumpywiki.txt", line 69, in testnumpywiki.txt
Failed example:
    a[:,b2]
Exception raised:
    Traceback (most recent call last):
      File "c:\python24\lib\doctest.py", line 1243, in __run
        compileflags, 1) in test.globs
      File "<doctest testnumpywiki.txt[18]>", line 1, in ?
        a[:,b2]
    IndexError: arrays used as indices must be of integer type
**********************************************************************
File "testnumpywiki.txt", line 893, in testnumpywiki.txt
Failed example:
    ceil(a)                                                # nearest 
integers greater-than or equal to a
Expected:
    array([-1., -1., -0.,  1.,  2.,  2.])
Got:
    array([-1., -1.,  0.,  1.,  2.,  2.])
**********************************************************************
File "testnumpywiki.txt", line 1162, in testnumpywiki.txt
Failed example:
    cov(T,P)                                # covariance between 
temperature and pressure
Expected:
    3.9541666666666657
Got:
    array([[ 1.97583333,  3.95416667],
           [ 3.95416667,  8.22916667]])
**********************************************************************
File "testnumpywiki.txt", line 2235, in testnumpywiki.txt
Failed example:
    type(a[0])
Expected:
    <type 'int32_arrtype'>
Got:
    <type 'int32scalar'>