Re: [Numpy-discussion] [SciPy-dev] importing madness

[Ed S. <sch...@ft...>]

Robert Kern wrote:

>Fernando Perez wrote:
>
>  
>
>>Anyway, I won't belabor this point any longer.  I'd just like to hear from 
>>others their opinion on this matter, and if a decision is made to go ahead 
>>with the overwriting, at least I think the rationale for it should be well 
>>justified (and be more than "it's convenient").  The fact that over the last 
>>few weeks we've had several surprised questions on this is, to me, an 
>>indicator that I'm not the one uncomfortable with this decision.
>>    
>>
>
>I haven't followed this discussion in great detail, but I believe the current
>situation is this:
>
>1) If you use numpy.dft and numpy.linalg directly, you will always get the numpy
>versions no matter what else is installed.
>
>2) If you want to optionally use optimized scipy versions if they are available
>and regular numpy versions otherwise, then you use the functions exposed in
>numpy.dual. You do so at your own risk.
>
>3) pkgload() exists to support the loading of subpackages. It does not reach
>into numpy.dft or numpy.linalg at all. It is not relevant to this issue.
>
>4) There are some places in numpy that use numpy.dual.
>
>I think we can address all of your concerns by changing #4.
>  
>
I've been battling for half an hour trying to import scipy.linalg.  Is
this one of Fernando's scary predictions coming true?  I get:

>>> from scipy import linalg
>>> dir(linalg)
['Heigenvalues', 'Heigenvectors', 'LinAlgError', 'ScipyTest',
'__builtins__', '__doc__', '__file__', '__name__', '__path__',
'cholesky', 'cholesky_decomposition', 'det', 'determinant', 'eig',
'eigenvalues', 'eigenvectors', 'eigh', 'eigvals', 'eigvalsh',
'generalized_inverse', 'inv', 'inverse', 'lapack_lite', 'linalg',
'linear_least_squares', 'lstsq', 'pinv', 'singular_value_decomposition',
'solve', 'solve_linear_equations', 'svd', 'test']
>>> linalg.__file__
'/home/schofield/Tools/lib/python2.4/site-packages/numpy/linalg/__init__.pyc'

This is the linalg package from numpy, not scipy.  It's missing my
favourite pinv2 function.  What is going on?!  I've just cleaned
everything out and built from the latest SVN revisions.  Using
pkgload('linalg') doesn't seem to help:

>>> import scipy
>>> scipy.pkgload('linalg')
>>> linalg.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
NameError: name 'linalg' is not defined
>>> scipy.linalg.__file__
'/home/schofield/Tools/lib/python2.4/site-packages/numpy/linalg/__init__.pyc'

The only thing that helps is calling pkgload() with no arguments:

>>> import scipy
>>> scipy.pkgload()
Overwriting lib=<module 'scipy.lib' from
'/home/schofield/Tools/lib/python2.4/site-packages/scipy/lib/__init__.pyc'>
from
/home/schofield/Tools/lib/python2.4/site-packages/scipy/lib/__init__.pyc
(was <module 'numpy.lib' from
'/home/schofield/Tools/lib/python2.4/site-packages/numpy/lib/__init__.pyc'>
from
/home/schofield/Tools/lib/python2.4/site-packages/numpy/lib/__init__.pyc)
>>> scipy.linalg.__file__
'/home/schofield/Tools/lib/python2.4/site-packages/scipy/linalg/__init__.pyc'

Unless I'm doing something very stupid, there seem to be multiple
sources of evil here.  First, numpy's linalg package is available from
the scipy namespace, which seems like a recipe for Ed's madness, since
he can't find his pinv2() function.  Second, scipy.pkgload('linalg')
silently fails to make any visible difference.  This is probably a
simple bug.  Third, ...,  there is no third.  But bad things usually
come in threes.


-- Ed