numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Travis O. <oli...@ie...>]

Christopher Barker wrote:
> HI all,
>
> File 
> "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", 
> line 2814, in SetDataBuffer
>     return _core_.Image_SetDataBuffer(*args, **kwargs)
> TypeError: non-character array cannot be interpreted as character buffer
>
> If I try to pass in a numpy array, while it works great with a 
> numarray array.

This error sounds like wx is using the *wrong* buffer protocol.  Don't 
use bf_getcharbuffer as it is of uncertain utility.  It is slated for 
removal from Python 3000.  It was meant to be used as a way to determine 
buffers that were supposed to contain characters (not arbitrary data).

Just use bf_getreadbuffer and bf_getwritebuffer  from tp_as_buffer.

More support for the buffer protocol all the way around is a good idea. 
  NumPy has always supported it very well (just make sure to use it 
correctly).   FYI, I'm going to write a PEP to get the array protocol 
placed as an add-on to the buffer protocol for Python 2.6

-Travis

[Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Christopher B. <Chr...@no...>]

HI all,

Robin Dunn has been working on adding better support for dumping data 
directly to wxPython from the num* packages. I've been talking to him 
about the new array interface, and he might well support it 
(particularly if one of us contributes code), but in the meantime, he's 
got a number of things working with python buffers. For instance:

wx.Image.SetDataBuffer(dataBuffer)

That sets the data for a wxImage to the buffer handed in. This isn't as 
nice as the array protocol, as it has no way of checking anything other 
than if the length of the buffer is correct, but it is a good way to 
maximize performance for this sort of thing.

he's now working on adding methods for creating wx.Bitmaps directly from 
buffers. In the process if testing some of this, I discovered that 
numarray (which Robin is testing with) works fine, but numpy does not. I 
get:

File "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", 
line 2814, in SetDataBuffer
     return _core_.Image_SetDataBuffer(*args, **kwargs)
TypeError: non-character array cannot be interpreted as character buffer

If I try to pass in a numpy array, while it works great with a numarray 
array.

While I'm a great advocate of the new array protocol, it seems 
supporting the buffer protocol also would be a good idea. I've enclosed 
some simple test code. It works with numarray, but not numpy 1.0b4

Tested with Python 2.4.3, wxPython 2.6.3.0, Linux fedora core4

-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] request for new array method: arr.abs()

[Christopher B. <Chr...@no...>]

Travis Oliphant wrote:
> I think I prefer to just add the float, bool, object, unicode, str names 
> to the "non-imported" numpy
> name-space.

which mean you get it with:

import numpy as N
N.float

but not with

from numpy import *

?

If that's what you mean, then I'm all for it!

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] 1.0b4 problem continuted from 1.0b3

[Travis O. <oli...@ie...>]

kor...@id... wrote:
> On 8/25/06, Travis Oliphant <oli...@ie...> wrote:
>   
>> kor...@id... wrote:
>>     
>>> Message: 4
>>> Date: Thu, 24 Aug 2006 14:17:44 -0600
>>> From: Travis Oliphant <oli...@ee...>
>>> Subject: Re: [Numpy-discussion] (no subject)
>>> To: Discussion of Numerical Python
>>>         <num...@li...>
>>> Message-ID: <44E...@ee...>
>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>
>>> kor...@id... wrote:
>>>
>>>
>>>
>>> You have a module built against an older version of NumPy.  What modules
>>> are being loaded?  Perhaps it is matplotlib or SciPy
>>>
>>>       
>> You need to re-build matplotlib.  They should be producing a binary that
>> is compatible with 1.0b2 (I'm being careful to make sure future releases
>> are binary compatible with 1.0b2).
>>
>> Also, make sure that you remove the build directory under numpy if you
>> have previously built a version of numpy prior to 1.0b2.
>>     

You have to download the SVN version of matplotlib.  The released 
version does not support 1.0b2 and above yet.

-Travis

[Numpy-discussion] 1.0b4 problem continuted from 1.0b3

[<kor...@id...>]

On 8/25/06, Travis Oliphant <oli...@ie...> wrote:
> kor...@id... wrote:
> > Message: 4
> > Date: Thu, 24 Aug 2006 14:17:44 -0600
> > From: Travis Oliphant <oli...@ee...>
> > Subject: Re: [Numpy-discussion] (no subject)
> > To: Discussion of Numerical Python
> >         <num...@li...>
> > Message-ID: <44E...@ee...>
> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> >
> > kor...@id... wrote:
> >
> >
> >
> > You have a module built against an older version of NumPy.  What modules
> > are being loaded?  Perhaps it is matplotlib or SciPy
> >
>
> You need to re-build matplotlib.  They should be producing a binary that
> is compatible with 1.0b2 (I'm being careful to make sure future releases
> are binary compatible with 1.0b2).
>
> Also, make sure that you remove the build directory under numpy if you
> have previously built a version of numpy prior to 1.0b2.
>
> -Travis
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

Travis I have recompiled everything.  I removed sci py numpy and matplotlib.

I installed the numpy 1.0b4 win32exe, and then installed scipi 0.5 and
then the latest matplotlib 0.87.4

I recieved this error at first, which is a matplot lib error,

C:\Lameness>c:\python23\python templatewindow.py
Traceback (most recent call last):
  File "templatewindow.py", line 7, in ?
    import wxmpl
  File "c:\python23\lib\site-packages\wxmpl.py", line 25, in ?
    import matplotlib.numerix as Numeric
  File "C:\PYTHON23\Lib\site-packages\matplotlib\numerix\__init__.py",
line 74,
in ?
    Matrix = matrix
NameError: name 'matrix' is not defined
,
and then switched matplotlib to use numeric, and i recieve this error once
again

Overwriting info=<function info at 0x02097C30> from scipy.misc.helpmod
(was <fun
ction info at 0x020777F0> from numpy.lib.utils)
Overwriting who=<function who at 0x020971B0> from scipy.misc.common (was
<functi
on who at 0x020776F0> from numpy.lib.utils)
Overwriting source=<function source at 0x02097EB0> from scipy.misc.helpmod
(was
<function source at 0x02077830> from numpy.lib.utils)
RuntimeError: module compiled against version 1000000 of C-API but this
version
of numpy is 1000002
Fatal Python error: numpy.core.multiarray failed to import... exiting.


abnormal program termination


i googled the error and also found this thread but have not found a solution

http://www.mail-archive.com/num...@li.../msg01700.html

any help?
thanks
-Kenny

[Numpy-discussion] weave using numeric or numpy?

[Carlos P. <car...@ya...>]

Hi all!
I'm rewriting some swig-based extensions that implement intensive inner loops dealing with numeric/numpy arrays. The intention is to build these extensions by means of weave inline, ext_module, ext_function, etc. I'm not sure about how to point weave to my numpy instalation. By default it tries to include "Numeric/arrayobject.h" and fails if you hack things to get that resolved to numpy arrayobject.h (for example, it complaints that PyArray_SBYTE is undefined). Anyway, even if I managed myself to force weave to compile against numpy/arrayobject.h, I'd still not be sure about the "runtime" that will be chosen. I'm very confused at this point, no library flags are provided at compile/link time, so how is the runtime selected between numpy, Numeric (or even numarray)?
Thank you in advance.
Best regards,
Carlos

 		
---------------------------------
 Preguntá. Respondé. Descubrí.
 Todo lo que querías saber, y lo que ni imaginabas,
 está en Yahoo! Respuestas (Beta).
 Probalo ya! 

Re: [Numpy-discussion] Numeric/numpy incompatibility

[Travis O. <oli...@ie...>]

Andrew Straw wrote:
> The following code indicates there is a problem adding a numpy scalar
> type to a Numeric array. Is this expected behavior or is there a bug
> somewhere?
>   
There was a bug in the __array_struct__ attribute of array flags wherein 
the NOTSWAPPED flag was not being set as it should be.

This is fixed in SVN.

-Travis

Re: [Numpy-discussion] numpy1.04b4: undefined symbol: PyUnicodeUCS2_FromUnicode. error No _WIN32

[rex <re...@no...>]

Travis Oliphant <oli...@ie...> [2006-08-28 12:42]:
> rex wrote:
> > ImportError: /usr/lib/python2.4/site-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_FromUnicode
> >
> >
> >   
> 
> This error usually means that NumPy was built and linked against a 
> Python build where unicode strings were 2-bytes per character but you 
> are trying to import it on a Python build where unicode strings are 
> 4-bytes per character.   Perhaps you have changed your build of Python 
> and did not remove the build directory of NumPy.
> 
> Try
> 
> rm -fr build
> 
> in the numpy directory (where you run setup.py) and build again.

Ah! THANK YOU! 

Python 2.4.2 (#1, May  2 2006, 08:13:46)
[GCC 4.1.0 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.test()
  Found 5 tests for numpy.distutils.misc_util
  Found 3 tests for numpy.lib.getlimits
  Found 31 tests for numpy.core.numerictypes
  Found 32 tests for numpy.linalg
  Found 13 tests for numpy.core.umath
  Found 4 tests for numpy.core.scalarmath
  Found 8 tests for numpy.lib.arraysetops
  Found 42 tests for numpy.lib.type_check
  Found 155 tests for numpy.core.multiarray
  Found 3 tests for numpy.fft.helper
  Found 36 tests for numpy.core.ma
  Found 10 tests for numpy.lib.twodim_base
  Found 10 tests for numpy.core.defmatrix
  Found 1 tests for numpy.lib.ufunclike
  Found 4 tests for numpy.ctypeslib
  Found 39 tests for numpy.lib.function_base
  Found 1 tests for numpy.lib.polynomial
  Found 8 tests for numpy.core.records
  Found 26 tests for numpy.core.numeric
  Found 4 tests for numpy.lib.index_tricks
  Found 46 tests for numpy.lib.shape_base
  Found 0 tests for __main__

----------------------------------------------------------------------
Ran 481 tests in 1.956s

OK

Now on to doing it again with MKL...
>From the numpy directory:
rm -fr build
cp site_mkl.cfg site.cfg

where site_mkl.cfg is:
-----------------------------------------------------------------------
[DEFAULT]
library_dirs=/opt/intel/mkl/8.1/lib/32
include_dirs=/opt/intel/mkl/8.1/include

[blas_opt]
libraries=libmkl.so,libmkl_p3.so,libmkl_vml_p3.so,libmkl_ia32.a,libguide.so,libmkl_def.so
#libraries=whatever_the_mkl_blas_lib_is,mkl_ia32,mkl,guide

[lapack_opt]
libraries=libmkl_lapack32.so,libmkl_lapack.a,
#libraries=mkl_lapack,mkl_lapack32,mkl_ia32,mkl,guide
----------------------------------------------------------------------
python setup.py install >& inst.log

Looks OK, so in another window:

python
Python 2.4.2 (#1, May  2 2006, 08:13:46)
[GCC 4.1.0 (SUSE Linux)] on linux2
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 39, in ?
    import linalg
  File "/usr/lib/python2.4/site-packages/numpy/linalg/__init__.py", line 4, in ?
    from linalg import *
  File "/usr/lib/python2.4/site-packages/numpy/linalg/linalg.py", line 25, in ?
    from numpy.linalg import lapack_lite
ImportError: libmkl_lapack32.so: cannot open shared object file: No such file or directory
>>>

Oops!

^d

export INCLUDE=/opt/intel/mkl/8.1/include:$INCLUDE
export LD_LIBRARY_PATH=/opt/intel/mkl/8.1/lib/32:$LD_LIBRARY_PATH
python
Python 2.4.2 (#1, May  2 2006, 08:13:46)
[GCC 4.1.0 (SUSE Linux)] on linux2
>>> import numpy
>>> numpy.test()
  Found 5 tests for numpy.distutils.misc_util
  Found 3 tests for numpy.lib.getlimits
  Found 31 tests for numpy.core.numerictypes
  Found 32 tests for numpy.linalg
  Found 13 tests for numpy.core.umath
  Found 4 tests for numpy.core.scalarmath
  Found 8 tests for numpy.lib.arraysetops
  Found 42 tests for numpy.lib.type_check
  Found 155 tests for numpy.core.multiarray
  Found 3 tests for numpy.fft.helper
  Found 36 tests for numpy.core.ma
  Found 10 tests for numpy.lib.twodim_base
  Found 10 tests for numpy.core.defmatrix
  Found 1 tests for numpy.lib.ufunclike
  Found 4 tests for numpy.ctypeslib
  Found 39 tests for numpy.lib.function_base
  Found 1 tests for numpy.lib.polynomial
  Found 8 tests for numpy.core.records
  Found 26 tests for numpy.core.numeric
  Found 4 tests for numpy.lib.index_tricks
  Found 46 tests for numpy.lib.shape_base
  Found 0 tests for __main__

----------------------------------------------------------------------
Ran 481 tests in 2.152s

OK

Now off to build SciPy.

Thanks again!

-rex

[Numpy-discussion] Numeric/numpy incompatibility

[Andrew S. <str...@as...>]

The following code indicates there is a problem adding a numpy scalar
type to a Numeric array. Is this expected behavior or is there a bug
somewhere?

This bit me in the context of updating some of my code to numpy, while
part of it still uses Numeric.

import Numeric
import numpy

print 'Numeric.__version__',Numeric.__version__
print 'numpy.__version__',numpy.__version__

a = Numeric.zeros( (10,2), Numeric.Float )
b = numpy.float64(23.39)
a[0,1] = a[0,1] + b
assert a[0,1]==b

Re: [Numpy-discussion] request for new array method: arr.abs()

[Travis O. <oli...@ie...>]

Christopher Barker wrote:
> Travis Oliphant wrote:
>
>   
>> Instead, I like better the idea of adding abs, round, max, and min to 
>> the "non-import-*" namespace of numpy.
>>     
>
> Another I'd like is the built-in data types. I always use:
>
> import numpy as N
>
> so then I do:
>
> a = zeros(shape, float)
> or
> a = zeros(shape, N.float_)
>
> but for non-built-in types, I can't do the former.
>
> The underscore is minor but why not just have:
>
> float = float
>
> in numpy.py?
>
> (and of course, the others)
>   

I think I prefer to just add the float, bool, object, unicode, str names 
to the "non-imported" numpy
name-space.

-Travis

Re: [Numpy-discussion] numpy1.04b4: undefined symbol: PyUnicodeUCS2_FromUnicode. error No _WIN32

[Travis O. <oli...@ie...>]

rex wrote:
> Numpy builds, but fails to run with the error message:
>
>   
>> python
>>     
> Python 2.4.2 (#1, Apr 24 2006, 18:13:30)
> [GCC 4.1.0 (SUSE 10.1 Linux)] on linux2
>   
>>>> import numpy
>>>>         
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 35, in ?
>     import core
>   File "/usr/lib/python2.4/site-packages/numpy/core/__init__.py", line 5, in ?
>     import multiarray
> ImportError: /usr/lib/python2.4/site-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_FromUnicode
>
>
>   

This error usually means that NumPy was built and linked against a 
Python build where unicode strings were 2-bytes per character but you 
are trying to import it on a Python build where unicode strings are 
4-bytes per character.   Perhaps you have changed your build of Python 
and did not remove the build directory of NumPy.

Try

rm -fr build

in the numpy directory (where you run setup.py) and build again.


You can tell how many bytes-per-unicode character your system is built 
with by looking at the output of sys.maxunicode

Re: [Numpy-discussion] request for new array method: arr.abs()

[Christopher B. <Chr...@no...>]

Travis Oliphant wrote:

> Instead, I like better the idea of adding abs, round, max, and min to 
> the "non-import-*" namespace of numpy.

Another I'd like is the built-in data types. I always use:

import numpy as N

so then I do:

a = zeros(shape, float)
or
a = zeros(shape, N.float_)

but for non-built-in types, I can't do the former.

The underscore is minor but why not just have:

float = float

in numpy.py?

(and of course, the others)

-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] numpy1.04b4: undefined symbol: PyUnicodeUCS2_FromUnicode. error No _WIN32

[rex <re...@no...>]

Numpy builds, but fails to run with the error message:

> python
Python 2.4.2 (#1, Apr 24 2006, 18:13:30)
[GCC 4.1.0 (SUSE 10.1 Linux)] on linux2
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/site-packages/numpy/__init__.py", line 35, in ?
    import core
  File "/usr/lib/python2.4/site-packages/numpy/core/__init__.py", line 5, in ?
    import multiarray
ImportError: /usr/lib/python2.4/site-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_FromUnicode


Build was without BLAS or LAPACK. Results were the same when Intel MKL
was used.

python setup.py install >& inst.log
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2_3078
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in /usr/local/lib
  libraries mkl,vml,guide not found in /usr/lib
  NOT AVAILABLE

[...]

running install
running build
running config_fc
running build_src
building py_modules sources
building extension "numpy.core.multiarray" sources
  adding 'build/src.linux-i686-2.4/numpy/core/config.h' to sources.
executing numpy/core/code_generators/generate_array_api.py
  adding 'build/src.linux-i686-2.4/numpy/core/__multiarray_api.h' to sources.
  adding 'build/src.linux-i686-2.4/numpy/core/src' to include_dirs.
numpy.core - nothing done with h_files= ['build/src.linux-i686-2.4/numpy/core/src/scalartypes
.inc', 'build/src.linux-i686-2.4/numpy/core/src/arraytypes.inc', 'build/src.linux-i686-2.4/nu
mpy/core/config.h', 'build/src.linux-i686-2.4/numpy/core/__multiarray_api.h']
building extension "numpy.core.umath" sources
  adding 'build/src.linux-i686-2.4/numpy/core/config.h' to sources.
executing numpy/core/code_generators/generate_ufunc_api.py
  adding 'build/src.linux-i686-2.4/numpy/core/__ufunc_api.h' to sources.
  adding 'build/src.linux-i686-2.4/numpy/core/src' to include_dirs.
numpy.core - nothing done with h_files= ['build/src.linux-i686-2.4/numpy/core/src/scalartypes
.inc', 'build/src.linux-i686-2.4/numpy/core/src/arraytypes.inc', 'build/src.linux-i686-2.4/nu
mpy/core/config.h', 'build/src.linux-i686-2.4/numpy/core/__ufunc_api.h']
building extension "numpy.core._sort" sources
  adding 'build/src.linux-i686-2.4/numpy/core/config.h' to sources.
executing numpy/core/code_generators/generate_array_api.py
  adding 'build/src.linux-i686-2.4/numpy/core/__multiarray_api.h' to sources.
numpy.core - nothing done with h_files= ['build/src.linux-i686-2.4/numpy/core/config.h', 'bui
ld/src.linux-i686-2.4/numpy/core/__multiarray_api.h']
building extension "numpy.core.scalarmath" sources
  adding 'build/src.linux-i686-2.4/numpy/core/config.h' to sources.
executing numpy/core/code_generators/generate_array_api.py
  adding 'build/src.linux-i686-2.4/numpy/core/__multiarray_api.h' to sources.
executing numpy/core/code_generators/generate_ufunc_api.py
  adding 'build/src.linux-i686-2.4/numpy/core/__ufunc_api.h' to sources.
numpy.core - nothing done with h_files= ['build/src.linux-i686-2.4/numpy/core/config.h', 'bui
ld/src.linux-i686-2.4/numpy/core/__multiarray_api.h', 'build/src.linux-i686-2.4/numpy/core/__
ufunc_api.h']
building extension "numpy.core._dotblas" sources
building extension "numpy.lib._compiled_base" sources
building extension "numpy.numarray._capi" sources
building extension "numpy.fft.fftpack_lite" sources
building extension "numpy.linalg.lapack_lite" sources
### Warning:  Using unoptimized lapack ###
  adding 'numpy/linalg/lapack_litemodule.c' to sources.
  adding 'numpy/linalg/zlapack_lite.c' to sources.
  adding 'numpy/linalg/dlapack_lite.c' to sources.
  adding 'numpy/linalg/blas_lite.c' to sources.
  adding 'numpy/linalg/dlamch.c' to sources.
  adding 'numpy/linalg/f2c_lite.c' to sources.
building extension "numpy.random.mtrand" sources
Could not locate executable f95
customize GnuFCompiler
customize GnuFCompiler
customize GnuFCompiler using config

*******************************************************************************************
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC

compile options: '-Inumpy/core/src -Inumpy/core/include -I/usr/include/python2.4 -c'
gcc: _configtest.c
_configtest.c:7:2: error: #error No _WIN32
_configtest.c:7:2: error: #error No _WIN32
failure.
removing: _configtest.c _configtest.o
*******************************************************************************************

building data_files sources

[...]

changing mode of /usr/bin/f2py to 755
running install_data
copying build/src.linux-i686-2.4/numpy/core/__multiarray_api.h -> /usr/lib/python2.4/site-pac
kages/numpy/core/include/numpy
copying build/src.linux-i686-2.4/numpy/core/multiarray_api.txt -> /usr/lib/python2.4/site-pac
kages/numpy/core/include/numpy
copying build/src.linux-i686-2.4/numpy/core/__ufunc_api.h -> /usr/lib/python2.4/site-packages
/numpy/core/include/numpy
copying build/src.linux-i686-2.4/numpy/core/ufunc_api.txt -> /usr/lib/python2.4/site-packages
/numpy/core/include/numpy

Any pointers would be much appreciated. This isn't the first time I've
spent days trying to get SciPy built under SUSE... :(

-rex

Re: [Numpy-discussion] C Api newbie question

[Matt K. <mat...@ho...>]

>>Matt Knox wrote:>>> Hi there. I'm in the unfortunate situation of trying =
to track down a >> memory error in someone elses code, and to make matters =
worse I don't >> really know jack squat about C programming. The problem se=
ems to arise >> when several numpy arrays are created from C arrays in the =
C api and >> returned to python, and then trying to print out or cast to a =
string >> the resulting array. I think the problem may be happening due to =
the >> following chunk of code:>> {>>      PyObject* temp =3D PyArray_Simpl=
eNewFromData(1, &numobjs, typeNum, >> dbValues);>>      PyObject* temp2 =3D=
 PyArray_FromArray((PyArrayObject*)temp, >> ((PyArrayObject*)temp)->descr, =
DEFAULT_FLAGS | ENSURECOPY);>>      Py_DECREF(temp);>>      PyDict_SetItemS=
tring(returnVal, "data", temp2);>>      Py_DECREF(temp2);>> }>>  >> Lets as=
sume that all my other inputs up this point are fine and that >> numobjs, t=
ypeNum, and dbValues are fine. Is their anything obviously >> wrong with th=
e above chunk of code? or does it appear ok? Ultimately >> the dictionary "=
returnVal" is returned by the function this code came >> from, and everythi=
ng else is discarded. Any help is very greatly >> appreciated. Thanks in ad=
vance,
> You didn't indicate what kind of trouble you are having.>> First of all, =
this is kind of odd style.  Why is a new array created > from a data-pointe=
r and then copied using PyArray_FromArray (the > ENSURECOPY flag will give =
you a copy)?  Using>> temp2 =3D PyArray_Copy(temp)>> seems simpler.   This =
will also avoid the reference-count problem that > is currently happening i=
n the PyArray_FromArray call on the descr > structure.    Any array-creatio=
n function that takes a descr structure > "steals" a reference to it, so yo=
u need to increment the reference count > if you are passing an unowned ref=
erence to a ->descr structure.>> -Travis
=20
Sorry. Yeah, the problem was the interpreter crashing on exit, which aftery=
our response definitely seems like it was a reference count issue. Ichanged=
 the PyArray_FromArray call to be PyArray_Copy and it seems to workfine. Th=
ank you very much!
=20
Love the numpy stuff (when I can stay in the python world and not mess with=
the C stuff :) ). Keep up the great work!
=20
- Matt
_________________________________________________________________
Be one of the first to try Windows Live Mail.
http://ideas.live.com/programpage.aspx?versionId=3D5d21c51a-b161-4314-9b0e-=
4911fb2b2e6d=

Re: [Numpy-discussion] memory corruption bug

[Bill B. <wb...@gm...>]

Nope, that's the way python works in general for any type other than
basic scalar types.
>>> a = [1,2,3,4]
>>> b = a
>>> b[1] = 99
>>> print a
[1, 99, 3, 4]
>>> print b
[1, 99, 3, 4]

Also the issue never comes up for types like tuples or strings because
they aren't mutable.

--bb

On 8/28/06, Sven Schreiber <sve...@gm...> wrote:
> Charles R Harris schrieb:
> > +1. I too suspect that what you have here is a reference/copy problem.
> > The only thing that is local to the class is the reference (pointer),
> > the data is global.
> >
> > Chuck
>
> Ok, so you guys were right, turns out that my problem was caused by the
> fact that a local assignment like x = y is also by reference only, which
> I wasn't really aware of. (Of course, it's explained in Travis' book...)
> So that behavior is different from standard python assignments, isn't it?
>
> Sorry for the noise.
>
> -Sven
>

Re: [Numpy-discussion] memory corruption bug

[Sven S. <sve...@gm...>]

Charles R Harris schrieb:
> +1. I too suspect that what you have here is a reference/copy problem.
> The only thing that is local to the class is the reference (pointer),
> the data is global.
> 
> Chuck

Ok, so you guys were right, turns out that my problem was caused by the
fact that a local assignment like x = y is also by reference only, which
I wasn't really aware of. (Of course, it's explained in Travis' book...)
So that behavior is different from standard python assignments, isn't it?

Sorry for the noise.

-Sven

Re: [Numpy-discussion] Optimizing mean(axis=0) on a 3D array

[Martin S. <nu...@ms...>]

Martin Spacek wrote:

> 
> Actually, your original version is just as fast as the take() version. 
> Both are about 9X faster than numpy.mean() on my system. I prefer the 
> take() version because you only have to pass a single argument to 
> mean_accum()

I forgot to mention that all my indices are, for now, sorted. I just 
tried shuffling them (as you did), but I still get the same 9x 
improvement in speed, so I don't know why you only get a 4x improvement 
on your system.

Martin

Re: [Numpy-discussion] Optimizing mean(axis=0) on a 3D array

[Martin S. <nu...@ms...>]

Tim Hochberg wrote:

> I'm actually surprised that the take version is faster than my original 
> version since it makes a big  ol' copy. I guess this is an indication 
> that indexing is more expensive than I realize. That's why nothing beats 
> measuring!

Actually, your original version is just as fast as the take() version. 
Both are about 9X faster than numpy.mean() on my system. I prefer the 
take() version because you only have to pass a single argument to 
mean_accum()

Martin

Re: [Numpy-discussion] bad generator behaviour with sum

[Travis O. <oli...@ie...>]

Tom Denniston wrote:
> I was thinking about this in the context of Giudo's comments at scipy
> 2006 that much of the language is moving away from lists toward
> iterators.  He gave the keys of a dict as an example.
>
> Numpy treats iterators, generators, etc as 0x0 PyObjects rather than
> lazy generators of n dimensional data.  I guess my question for Travis
> (any others much more expert than I in numpy) is is this intentional
> or is it something that was never implemented because of the obvious
> subtlties of defiing the correct semantics to make this work.
>
>   
It's not intentional, it's just that iterators came later and I did not 
try to figure out how to "do the right thing" in the array function.    
Thanks to Tim Hochberg, there is a separate  fromiter function that 
creates arrays from iterators.

> Personally i find it no big deal to use array(list(iter)) in the 1d
> case and the list function combined with a list comprehension for the
> 2d case.  I usually know how many dimensions i expect so i find this
> easy and i know about this peculiar behavior.  I find, however, that
> this behavior is very suprising and confusing to the new user and i
> don't usually have a good justification for it to answer them.
>   
The problem is that NumPy arrays need to know both how big they are and 
what data-type they are.  With iterators you have to basically construct 
the whole thing before you can even interrogate that question.   
Iterators were not part of the language when Numeric (from which NumPy 
got it's code base) was created.
> The ideal semantics, in my mind, would be if an iterator of iterators
> of iterators, etc was no different in numpy than a list of lists of
> lists, etc.  But I have no doubt that there are subtleties i am not
> considering. Has anyone more familiar than I with the bowels of numpy
> thought about this problem and see reasons why this is a bad idea or
> just prohibitively difficult to implement?
>   
It's been discussed before and ideas have been considered.   Right now, 
the fromiter function carries the load.   Whether or not to bring that 
functionality into the array function itself has been met with hesitancy 
because of how bulky the array function already is.

-Travis

Re: [Numpy-discussion] odd import behavior

[Travis O. <oli...@ie...>]

Matthew Auger wrote:
> I recently installed python2.5c1, numpy-1.0b3, and matplotlib-0.87.4. I 
> was getting an error when importing pylab that led me to this curious 
> behavior:
>   

matplotlib-0.87.4 is *not* compatible with 1.0b2 and above.   A new 
version needs to be released to work with NumPy 1.0

The SVN version of matplotlib works fine with NumPy 1.0

-Travis

Re: [Numpy-discussion] Constant array

[Charles R H. <cha...@gm...>]

Hi Carlos,

On 8/27/06, Carlos Pita <car...@ya...> wrote:
>
> Hi all!
> Is there a more efficient way of creating a constant K-valued array of
> size N than:
> zeros(N) + K
> ?
>

Maybe something like this:

In [12]: a = empty((3,3), dtype=int)

In [13]: a.fill(11)

In [14]: a
Out[14]:
array([[11, 11, 11],
       [11, 11, 11],
       [11, 11, 11]])

I haven't timed it, so don't know how fast it is. Looking at this makes me
think fill should return the array so that one could do something like: a =
empty((3,3), dtype=int).fill(10)

Chuck

[Numpy-discussion] Constant array

[Carlos P. <car...@ya...>]

Hi all!
Is there a more efficient way of creating a constant K-valued array of size N than:
zeros(N) + K
?
Thank you in advance.
Regards, Carlos

 		
---------------------------------
 Preguntá. Respondé. Descubrí.
 Todo lo que querías saber, y lo que ni imaginabas,
 está en Yahoo! Respuestas (Beta).
 Probalo ya! 

Re: [Numpy-discussion] bad generator behaviour with sum

[Tim H. <tim...@ie...>]

Tom Denniston wrote:
> I was thinking about this in the context of Giudo's comments at scipy
> 2006 that much of the language is moving away from lists toward
> iterators.  He gave the keys of a dict as an example.
>
> Numpy treats iterators, generators, etc as 0x0 PyObjects rather than
> lazy generators of n dimensional data.  I guess my question for Travis
> (any others much more expert than I in numpy) is is this intentional
> or is it something that was never implemented because of the obvious
> subtlties of defiing the correct semantics to make this work.
>   
More the latter than the former.

> Personally i find it no big deal to use array(list(iter)) in the 1d
> case and the list function combined with a list comprehension for the
> 2d case. 
There is a relatively new function fromiter, that materialized the last 
time this discussion came up that covers the above case. For example:
    numpy.fromiter((i*i for i in range(10)), int)

>  I usually know how many dimensions i expect so i find this
> easy and i know about this peculiar behavior.  I find, however, that
> this behavior is very suprising and confusing to the new user and i
> don't usually have a good justification for it to answer them.
>
> The ideal semantics, in my mind, would be if an iterator of iterators
> of iterators, etc was no different in numpy than a list of lists of
> lists, etc.  But I have no doubt that there are subtleties i am not
> considering. Has anyone more familiar than I with the bowels of numpy
> thought about this problem and see reasons why this is a bad idea or
> just prohibitively difficult to implement?
>   
There was some discussion about this several months ago and I even set 
out to implement it. I realized after not too long however that a 
complete solution, as you describe above, was going to be difficult and 
that I only really cared about the 1D case anyway, so punted and 
implemented fromiter instead. As I recall, there are two issues that 
complicate the general case:

   1. You need to specify the type or you gain no advantage over just
      instantiating the list. This is because you need to know the type
      before you allocate space for the array. Normally you do this by
      traversing the structure and looking at the contents. However for
      an iterable, you have to stash the results when you iterate over
      it looking for the type. This means that unless the array type is
      specified up front, you might as well just convert everything to
      lists as far as performance goes.
   2. For 1D arrays you can get away without knowing the shape by doing
      doing successive overallocation of memory, similar to the way list
      and array.array work. This is what fromiter does. I suppose the
      same tactic would work for iterators of iterators, but the
      bookkeeping becomes quite daunting.


Issue 1 is the real killer -- because of that a solution would either 
sometimes (mysteriously for the unitiated) be really inefficient or one 
would be required to specify types for array(iterable). The latter is my 
preference, but I'm beginning to think it would actually be better to 
always have to specify types. It's tempting to take another stab at 
this, in Python this time, and see if I can get a Python level soltuion 
working. However I don't have the time to try it right now.

-tim

> On 8/27/06, Charles R Harris <cha...@gm...> wrote:
>   
>> Hi,
>>
>> The problem seems to arise in the array constructor, which treats the
>> generator as a python object and creates an array containing that object.
>> So, do we want the possibility of an array of generators or should we
>> interpret it as a sort of list? I vote for that latter.
>>
>> Chuck
>>
>>
>> On 8/27/06, Charles R Harris <cha...@gm...> wrote:
>>     
>>> Hi Christopher,
>>>
>>>
>>>
>>> On 8/27/06, Charles R Harris < cha...@gm...> wrote:
>>>       
>>>> Hi,
>>>>
>>>>
>>>>
>>>> On 8/27/06, lis...@ma... <lis...@ma...> wrote:
>>>>         
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>> Hash: SHA1
>>>>>
>>>>> It seems like numpy.sum breaks generator expressions:
>>>>>
>>>>> In [1]: sum(i*i for i in range(10))
>>>>> Out[1]: 285
>>>>>
>>>>> In [2]: from numpy import sum
>>>>>
>>>>> In [3]: sum(i*i for i in range(10))
>>>>> Out[3]: <generator object at 0x10eca58>
>>>>>
>>>>> Is this intentional? If so, how do I get the behaviour that I am
>>>>>           
>> after?
>>     
>>>>
>>>>
>>>>
>>>> In [3]: sum([i*i for i in range(10)])
>>>>
>>>> Out[3]: 285
>>>>
>>>> Chuck
>>>>         
>>>
>>> The numarray.sum also fails to accept a generator as an argument. Because
>>>       
>> python does and the imported sum overwrites it, we should probably check the
>> argument type and make it do the right thing.
>>     
>>> Chuck
>>>
>>>
>>>
>>>       
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Num...@li...
>> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>>
>>
>>     
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>   

Re: [Numpy-discussion] bad generator behaviour with sum

[Robert K. <rob...@gm...>]

lis...@ma... wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Aug 27, 2006, at 4:19 PM, numpy-discussion- 
> re...@li... wrote:
> 
>>> It seems like numpy.sum breaks generator expressions:
>>>
>>> In [1]: sum(i*i for i in range(10))
>>> Out[1]: 285
>>>
>>> In [2]: from numpy import sum
>>>
>>> In [3]: sum(i*i for i in range(10))
>>> Out[3]: <generator object at 0x10eca58>
>>>
>>> Is this intentional? If so, how do I get the behaviour that I am  
>>> after?
>>>
>>
>> In [3]: sum([i*i for i in range(10)])
>> Out[3]: 285
> 
> Well, thats a list comprehension, not a generator expression. I was  
> after the latter because it is more efficient.

Not really. Any numpy functions that would automatically create an array from an 
__len__-less iterator will have to convert it to a list first. That said, some 
cases for numpy.sum() might be handled by passing the argument to 
__builtin__.sum(), but it might be tricky devising a robust rule for when that 
happens. Consequently, I would like to avoid doing so.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] ticket system does not like me ! - seems broken ...

[David M. C. <co...@ph...>]

On Aug 27, 2006, at 17:36 , Sebastian Haase wrote:

> Robert Kern wrote:
>> Sebastian Haase wrote:
>>
>>> (Could you add a web link from one system to the other ?)
>>
>> I'm afraid that I don't understand what you want. The numpy front  
>> page has a
>> link to the scipy front page. If you want a similar one in  
>> reverse, it's a Wiki
>> and you can do it yourself. If you mean something else, what do  
>> you mean?
>>
>
> Sorry for being so unclear -- I just often find myself (by clicking  
> on a
> ticket link) in one system (e.g. the scipy one)  and then I realize  
> that
> what I want is really more related to numpy ...
>
> I just found that the numpy page at
> http://projects.scipy.org/scipy/numpy
> contains the text
> """SciPy developer stuff goes on the sister site,
> http://projects.scipy.org/scipy/scipy/.
> """
>
> Could you add similar text to
> http://projects.scipy.org/scipy/scipy/
> like:
> """Stuff specific to the underlying numerical library (i.e. numpy)
> goes on the sister site, http://projects.scipy.org/scipy/numpy/
> """

It's a wiki; you can add it yourself :-) (if you're logged in, of  
course.)

-- 
|>|\/|<
/------------------------------------------------------------------\
|David M. Cooke              http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...