numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] OS X binary installers

[Chris F. <fon...@gm...>]

I am now distributing Mac binary installers for both numpy and scipy in a
"meta-package" along with a couple other modules (Matplotlib, PyMC). This
will hopefully resolve some of the version conflicts that some have been
experiencing with my builds of numpy and scipy that were not compiled
together. These builds are recent svn checkouts, and I hope to update them
approximately weekly. In addition, now that I have a new dual core Intel Ma=
c
Mini, I am distributing both PPC and Intel versions.

You can download either at http://trichech.us in the OS X downloads section=
.

Chris

--=20
Chris Fonnesbeck + Atlanta, GA + http://trichech.us

[Numpy-discussion] resize()

[<jo...@st...>]

Hi,

I was surprised by the following effect of resize()

>>> from numpy import *                    # 0.9.6
>>> a = array([1,2,3,4])
>>> a.resize(2,2)
>>> a
array([[1, 2],
       [3, 4]])
>>> a.resize(2,3)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: cannot resize an array that has been referenced or is referencing
another array in this way.  Use the resize function

Where exactly is the reference? I just started the interactive python shell,
did nothing else...

On the other hand, restarting python and executing

>>> from numpy import *
>>> a = array([1,2,3,4])
>>> a.resize(2,3)
>>> a
array([[1, 2, 3],
       [4, 0, 0]])

does work...

Why didn't it work for the first case?

Joris


Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

[Numpy-discussion] Discussion with Guido: an idea for scipy'06

[Fernando P. <fpe...@gm...>]

Hi all,

I think the presence of Guido as keynote speaker for scipy'06 is
something we should try to benefit of as much as possible.  I figured
a good way to do that would be to set aside a one hour period, at the
end of the first day (his keynote is that day), to hold a discussion
with him on all aspects of Python which are relevant to us as a group
of users, and which he may contribute feedback to, incorporate into
future versions, etc.

I floated the idea privately by some people and got no negative (and
some positive) feedback, so I'm now putting it out on the lists.  If
you all think it's a waste of time, it's easy to just kill the thing.

Since I know that many on this list may not be able to attend the
conference, but may still have valuable ideas to contribute, I thought
the best way to proceed (assuming people want to do this) would be to
prepare the key points for discussion on a public forum, true to the
spirit of open source collaboration.  For this purpose, I've just
created a stub page in a hurry:

http://scipy.org/SciPy06DiscussionWithGuido

Feel free to contribute to it.  Hopefully there (and on-list) we can
sort out interesting questions, and we can contact Guido a few days
before the conference so he has a chance to read it in advance.

Cheers,

f

ps - I didn't link to this page from anywhere else on the wiki, so
outside of this message it won't be easy to find.  I just didn't feel
comfortable touching the more 'visible' pages, but if this idea
floats,  we should make it easier to find by linking to it on one of
the conference pages.

Re: [Numpy-discussion] Getting 0.9.8 out this week

[<co...@ph...>]

Travis Oliphant <oli...@ie...> writes:

> I'd like to get 0.9.8 of NumPy released by the end of the week.
> There are a few Trac tickets that need to be resolved by then.
>
> In particular #83 suggests returning scalars instead of 1x1 matrices
> from certain reduce-like methods.  Please chime in on your preference.
> I'm waiting to here more feedback before applying the patch.
>
> If you can help out on any other ticket that would be much
> appreciated.

I'd like to fix up #81 (Numpy should be installable with setuptool's
easy_install), but I'm not going to have anytime to work on it before
the weekend.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] Building on Windows

[Stephan T. <st...@si...>]

Lalo wrote:
> Hi,
> 
> I built on Cygwin using the detailed ATLAS+LAPACK build tutorial below. 
> It's great, thanks!

If I've understood you correctly, you built Numpy in Cygwin using the 
_Python_ compiler from Cygwin, not the official Python binary 
distribution. As far as I know, that is currently not supported and 
probably unsafe because the resulting extension modules and Python will 
be linked to incompatible runtime libraries (msvcrt and the Cygwin one).

If you want to build Numpy under Windows without MSVC, currently the 
easiest way is to download MinGW from
http://sourceforge.net/forum/forum.php?forum_id=539405
install it and to put the MinGW/bin folder on the path of a normal 
windows command prompt. You do not need MSYS if you have built ATLAS and 
LAPACK with Cygwin and the -mno-cygwin option, as previously described.
Now run "d:\path\to\offical\python.exe setup.py config 
--compiler=mingw32 build --compiler=mingw32 bdist_wininst" in the Numpy 
root directory in the windows command prompt window (not Cygwin).

Stephan


> 
> My site.cfg is:
> 
> -------------------------------------------------
> [DEFAULT]
> library_dirs = /usr/lib:/usr/local/lib:/usr/lib/python2.4/config
> include_dirs = /usr/include:/usr/local/include
> search_static_first = 0
> 
> [atlas]
> library_dirs = /usr/local/lib/atlas
> # for overriding the names of the atlas libraries
> atlas_libs = lapack, f77blas, cblas, atlas
> -------------------------------------------------
> 
> However, I had to create the following link, not sure why it wouldn't 
> find the library otherwise:
> 
> ln -s /usr/lib/python2.4/config/libpython2.4.dll.a 
> /usr/lib/libpython2.4.dll.a
> 
> Hope this helps,
> Lalo
> 
> 
> ----- Original Message ----
> From: Stephan Tolksdorf <st...@si...>
> To: Albert Strasheim <fu...@gm...>
> Cc: numpy-discussion <num...@li...>
> Sent: Wednesday, May 10, 2006 4:48:16 PM
> Subject: Re: [Numpy-discussion] Building on Windows
> 
> Hi Albert,
> 
> in the following you find an abridged preview version of my
> MSVC+ATLAS+Lapack build tutorial ;-) You probably already know most of
> it, but maybe it helps.
> 
> You'll need a current Cygwin with g77 and MinGW-libraries installed.
> 
> Atlas:
> ======
> Download and extract latest development ATLAS (3.7.11). Comment out line
> 77 in Atlas/CONFIG/probe_SSE3.c. Run "make" and choose the appropriate
> options for your system. Don't activate posix threads (for now).
> Overwrite the compiler and linker flags with flags that include
> "-mno-cygwin". Use the default architecture settings. Atlas and the test
> suite hopefully compile without an error now.
> 
> Lapack:
> =======
> Download and extract www.netlib.org/lapack/lapack.tgz 
> <http://www.netlib.org/lapack/lapack.tgz> and apply the most
> current patch from www.netlib.org/lapack-dev/ 
> <http://www.netlib.org/lapack-dev/>
> Replace lapack/make.inc with lapack/INSTALL/make.inc.LINUX. Append
> "-mno-cygwin" to OPTS, NOOPT and LOADOPTS in make.inc. Add
> ".PHONY: install testing timing" as the last line to lapack/Makefile.
> Run "make install lib" in the lapack root directory in Cygwin.  ("make
> testing timing" and should also work now, but you probably want to use
> your optimised BLAS for that. Some errors in the tests are to be expected.)
> 
> Atlas + Lapack:
> ===============
> Copy the generated lapack_LINUX.a together with "libatlas.a",
> "libcblas.a", "libf77blas.a", "liblapack.a" into a convenient directory.
> 
> In Cygwin execute the following command sequence in that directory to
> get an ATLAS-optimized LAPACK library
> "ar x liblapack.a
> ar r lapack_LINUX.a *.o
> rm *.o
> mv lapack_LINUX.a liblapack.a"
> 
> Now make a copy of all lib*.a's to *.lib's, i.e. duplicate libatlas.a to
> atlas.lib, in order to allow distutils to recognize the libs and at the
> same time provide the correct versions for MSVC.
> 
> Copy libg2c.a and libgcc.a from cygwin/lib/gcc/i686-pc-mingw32/3.4.4 to
> this directory and again make .lib copies.
> 
> Compile and install numpy:
> ==========================
> 
> Put
> "[atlas]
> library_dirs =  d:\path\to\your\BlasDirectory
> atlas_libs = lapack,f77blas,cblas,atlas,g2c,gcc"
> into your site.cfg in the numpy root directory.
> 
> Open an Visual Studio 2003 command prompt and run
> "Path\To\Python.exe setup.py config --compiler=msvc build
> --compiler=msvc bdist_wininst".
> Use the resulting dist/numpy-VERSION.exe installer to install Numpy.
> 
> Testing:
> In a Python console run
> import numpy.testing
> numpy.testing.NumpyTest(numpy).run()
> ... hopefully without an error. Test your code base.
> 
> I'll wikify an extended version in the next days.
> 
> Stephan
> 
> 
> Albert Strasheim wrote:
>  > Hello all,
>  >
>  > It seems that many people are building on Windows without problems, 
> except
>  > for Stephan and myself.
>  >
>  > Let me start by staying that yes, the default build on Windows with MinGW
>  > and Visual Studio works nicely.
>  >
>  > However, is anybody building with ATLAS and finding that experience to be
>  > equally painless? If so, *please* can you tell me how you've 
> organized your
>  > libraries (which libraries? CLAPACK? FLAPACK? .a? .lib? What about 
> ATLAS's
>  > LAPACK functions? What about building ATLAS as DLL?). Also, I'd be very
>  > interested in the contents of your site.cfg.  I've been trying for many
>  > weeks to do some small subset of the above without hacking into the 
> core of
>  > numpy.distutils. So far, no luck.
>  >
>  > Does anybody do debug builds on Windows? Again, please tell me how you do
>  > this, because I would really like to be able to build a debug version of
>  > NumPy for debugging with the MSVS compiler.
>  >
>  > As for compiler warnings, last time I checked, distutils seems to be
>  > suppressing the output from the compiler, except when the build actually
>  > fails. Or am I mistaken?
>  >
>  > Eagerly awaiting Windows build nirvana,
>  >
>  > Albert
>  >
>  >> -----Original Message-----
>  >> From: num...@li... [mailto:numpy-
>  >> dis...@li...] On Behalf Of Tim Hochberg
>  >> Sent: 10 May 2006 23:49
>  >> To: Travis Oliphant
>  >> Cc: Stephan Tolksdorf; numpy-discussion
>  >> Subject: Re: [Numpy-discussion] Building on Windows
>  >>
>  >> Travis Oliphant wrote:
>  >>
>  >>> Stephan Tolksdorf wrote:
>  >>>
>  >>>> Hi,
>  >>>>
>  >>>> there are still some (mostly minor) problems with the Windows build
>  >>>> of Numpy (MinGW/Cygwin/MSVC). I'd be happy to produce patches and
>  >>>> documentation enhancements, but before doing so I'd like to know if
>  >>>> there's interest from one of the core developers to review/commit
>  >>>> these patches afterwards. I'm asking because in the past questions
>  >>>> and suggestions regarding the building process of Numpy (especially
>  >>>> on Windows) often remained unanswered on this list. I realise that
>  >>>> many developers don't use Windows and that the distutils build is a
>  >>>> complex beast, but the current situation seems a bit unsatisfactory
>  >>>> - and I would like to help.
>  >>> I think your assessment is a bit harsh.   I regularly build on MinGW
>  >>> so I know it works there (at least at release time).   I also have
>  >>> applied several patches with the express purpose of getting the build
>  >>> working on MSVC and Cygwin.
>  >>>
>  >>> So, go ahead and let us know what problems you are having.   You are
>  >>> correct that my main build platform is not Windows, but I think
>  >>> several other people do use Windows regularly and we definitely want
>  >>> to support it.
>  >>>
>  >> Indeeed. I build from SVN at least once a week using MSVC and it's been
>  >> compiling warning free and passing all tests for me for some time.
>  >>
>  >> -tim
>  >
>  >
>  >
>  > -------------------------------------------------------
>  > Using Tomcat but need to do more? Need to support web services, security?
>  > Get stuff done quickly with pre-integrated technology to make your 
> job easier
>  > Download IBM WebSphere Application Server v.1.0.1 based on Apache 
> Geronimo
>  > 
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 
> <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
>  > _______________________________________________
>  > Numpy-discussion mailing list
>  > Num...@li...
>  > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  >
>  >
> 
> 
> 
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 
> <http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

RE: [Numpy-discussion] Compiler warnings

[Albert S. <fu...@gm...>]

Hello all

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Stephan Tolksdorf
> Sent: 11 May 2006 19:54
> To: num...@li...
> Subject: [Numpy-discussion] Compiler warnings
> 
> Seems like the setup script currently suppresses quite a lot of
> warnings, not only in MSVC. I created a ticket (#113) with a patch that
> fixes many of the warnings under Visual C. Maybe someone else could have
> a look at the remaining ones.
> 
> @Albert: Do you still have a linking problem with the math functions? I
> haven't seen such a warning...

I sent Travis a description of the problem and he fixed it a while back.

Nice work on fixing the compiler warnings. I'll take a look at what's left
over when your patch is merged.

Regards,

Albert

Re: [Numpy-discussion] Building on Windows

[Lalo <lap...@ya...>]

Hi,

I built on Cygwin using the detailed ATLAS+LAPACK build tutorial below. It's great, thanks!

My site.cfg is:

-------------------------------------------------
[DEFAULT]
library_dirs = /usr/lib:/usr/local/lib:/usr/lib/python2.4/config
include_dirs = /usr/include:/usr/local/include
search_static_first = 0

[atlas]
library_dirs = /usr/local/lib/atlas
# for overriding the names of the atlas libraries
atlas_libs = lapack, f77blas, cblas, atlas
-------------------------------------------------

However, I had to create the following link, not sure why it wouldn't find the library otherwise:

ln -s /usr/lib/python2.4/config/libpython2.4.dll.a /usr/lib/libpython2.4.dll.a

Hope this helps,
Lalo



----- Original Message ----
From: Stephan Tolksdorf <st...@si...>
To: Albert Strasheim <fu...@gm...>
Cc: numpy-discussion <num...@li...>
Sent: Wednesday, May 10, 2006 4:48:16 PM
Subject: Re: [Numpy-discussion] Building on Windows

Hi Albert,

in the following you find an abridged preview version of my 
MSVC+ATLAS+Lapack build tutorial ;-) You probably already know most of 
it, but maybe it helps.

You'll need a current Cygwin with g77 and MinGW-libraries installed.

Atlas:
======
Download and extract latest development ATLAS (3.7.11). Comment out line 
77 in Atlas/CONFIG/probe_SSE3.c. Run "make" and choose the appropriate 
options for your system. Don't activate posix threads (for now). 
Overwrite the compiler and linker flags with flags that include 
"-mno-cygwin". Use the default architecture settings. Atlas and the test 
suite hopefully compile without an error now.

Lapack:
=======
Download and extract www.netlib.org/lapack/lapack.tgz and apply the most 
current patch from www.netlib.org/lapack-dev/
Replace lapack/make.inc with lapack/INSTALL/make.inc.LINUX. Append 
"-mno-cygwin" to OPTS, NOOPT and LOADOPTS in make.inc. Add
".PHONY: install testing timing" as the last line to lapack/Makefile.
Run "make install lib" in the lapack root directory in Cygwin.  ("make 
testing timing" and should also work now, but you probably want to use 
your optimised BLAS for that. Some errors in the tests are to be expected.)

Atlas + Lapack:
===============
Copy the generated lapack_LINUX.a together with "libatlas.a", 
"libcblas.a", "libf77blas.a", "liblapack.a" into a convenient directory.

In Cygwin execute the following command sequence in that directory to 
get an ATLAS-optimized LAPACK library
"ar x liblapack.a
ar r lapack_LINUX.a *.o
rm *.o
mv lapack_LINUX.a liblapack.a"

Now make a copy of all lib*.a's to *.lib's, i.e. duplicate libatlas.a to 
atlas.lib, in order to allow distutils to recognize the libs and at the 
same time provide the correct versions for MSVC.

Copy libg2c.a and libgcc.a from cygwin/lib/gcc/i686-pc-mingw32/3.4.4 to 
this directory and again make .lib copies.

Compile and install numpy:
==========================

Put
"[atlas]
library_dirs =  d:\path\to\your\BlasDirectory
atlas_libs = lapack,f77blas,cblas,atlas,g2c,gcc"
into your site.cfg in the numpy root directory.

Open an Visual Studio 2003 command prompt and run
"Path\To\Python.exe setup.py config --compiler=msvc build 
--compiler=msvc bdist_wininst".
Use the resulting dist/numpy-VERSION.exe installer to install Numpy.

Testing:
In a Python console run
import numpy.testing
numpy.testing.NumpyTest(numpy).run()
... hopefully without an error. Test your code base.

I'll wikify an extended version in the next days.

Stephan


Albert Strasheim wrote:
> Hello all,
> 
> It seems that many people are building on Windows without problems, except
> for Stephan and myself.
> 
> Let me start by staying that yes, the default build on Windows with MinGW
> and Visual Studio works nicely.
> 
> However, is anybody building with ATLAS and finding that experience to be
> equally painless? If so, *please* can you tell me how you've organized your
> libraries (which libraries? CLAPACK? FLAPACK? .a? .lib? What about ATLAS's
> LAPACK functions? What about building ATLAS as DLL?). Also, I'd be very
> interested in the contents of your site.cfg.  I've been trying for many
> weeks to do some small subset of the above without hacking into the core of
> numpy.distutils. So far, no luck.
> 
> Does anybody do debug builds on Windows? Again, please tell me how you do
> this, because I would really like to be able to build a debug version of
> NumPy for debugging with the MSVS compiler.
> 
> As for compiler warnings, last time I checked, distutils seems to be
> suppressing the output from the compiler, except when the build actually
> fails. Or am I mistaken?
> 
> Eagerly awaiting Windows build nirvana,
> 
> Albert
> 
>> -----Original Message-----
>> From: num...@li... [mailto:numpy-
>> dis...@li...] On Behalf Of Tim Hochberg
>> Sent: 10 May 2006 23:49
>> To: Travis Oliphant
>> Cc: Stephan Tolksdorf; numpy-discussion
>> Subject: Re: [Numpy-discussion] Building on Windows
>>
>> Travis Oliphant wrote:
>>
>>> Stephan Tolksdorf wrote:
>>>
>>>> Hi,
>>>>
>>>> there are still some (mostly minor) problems with the Windows build
>>>> of Numpy (MinGW/Cygwin/MSVC). I'd be happy to produce patches and
>>>> documentation enhancements, but before doing so I'd like to know if
>>>> there's interest from one of the core developers to review/commit
>>>> these patches afterwards. I'm asking because in the past questions
>>>> and suggestions regarding the building process of Numpy (especially
>>>> on Windows) often remained unanswered on this list. I realise that
>>>> many developers don't use Windows and that the distutils build is a
>>>> complex beast, but the current situation seems a bit unsatisfactory
>>>> - and I would like to help.
>>> I think your assessment is a bit harsh.   I regularly build on MinGW
>>> so I know it works there (at least at release time).   I also have
>>> applied several patches with the express purpose of getting the build
>>> working on MSVC and Cygwin.
>>>
>>> So, go ahead and let us know what problems you are having.   You are
>>> correct that my main build platform is not Windows, but I think
>>> several other people do use Windows regularly and we definitely want
>>> to support it.
>>>
>> Indeeed. I build from SVN at least once a week using MSVC and it's been
>> compiling warning free and passing all tests for me for some time.
>>
>> -tim
> 
> 
> 
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 



-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] basearray / arraykit

[Fernando P. <fpe...@gm...>]

On 5/11/06, Travis Oliphant <oli...@ee...> wrote:

> I agree in spirit.   Python is growing several ways to do the same thing
> because determining what data-type you are dealing with is useful in
> many contexts.   Unfortunately, there is not a lot of cross-pollination
> between the people.   I've contributed to a couple of threads to try and
> at least raise awareness that data-description is something NumPy has
> been thinking about, also.

Sounds like a good topic for the discussion idea.  I started the wiki for i=
t:

http://scipy.org/SciPy06DiscussionWithGuido

Gotta run now, I'll announce it on list later.

f

Re: [Numpy-discussion] basearray / arraykit

[Travis O. <oli...@ee...>]

Tim Hochberg wrote:

> Travis Oliphant wrote:
>
>> Christopher Barker wrote:
>>
>>> Travis Oliphant wrote:
>>>
>>>> 1) Implement a base-array with no getitem method nor setitem method 
>>>> at all
>>>>
>>>> 2) Implement a sub-class that supports only creation of data-types 
>>>> corresponding to existing Python scalars (Boolean, Long-based 
>>>> integers, Double-based floats, complex and object types).  Then, 
>>>> all array accesses should return the underlying Python objects.
>>>> This sub-class should also only do view-based indexing (basically 
>>>> it's old Numeric behavior inside of NumPy).
>>>
>>>
>>>
>>>> Item 1) should be pushed for inclusion in 2.6  and possibly even 
>>>> something like 2)
>>>
>>>
>>>
>>> + sys.maxint
>>>
>>> Having even this very basic n-d object in the standard lib would be 
>>> a MAJOR boon to python.
>>>
>>
>> I totally agree.   I've been advertising this for at least 8 months, 
>> but nobody is really willing to work on it (or fund it).   At least 
>> we have a summer student who is going to try and get Google 
>> summer-of-code money for it.  If you have any ability to bump up the 
>> ratings of summer of code applications.  Please consider bumping up 
>> his application.
>>
>>> However, as I think about it, one reason I'd really like to see an 
>>> nd-array in python is as a standard way to pass binary data around. 
>>> Right now, I'm working with the GDAL lib for geo-referenced raster 
>>> images, PIL, numpy and wxPython. I'm making a lot of copies to and 
>>> from python strings to pass the binary data back and forth. If all 
>>> these libs used the same nd-array, this would be much more 
>>> efficient. However, that would require non-python data types, most 
>>> notably a byte (or char, whatever) type.
>>>
>> Anything in Python would need to define at least a basic bytes type, 
>> for exactly this purpose.  So, we are on the same page.
>>
>> I'm thinking now that the data-type object now in NumPy would make a 
>> nice addition to Python as well for a standard way to define 
>> data-types.   Then,  Python itself wouldn't have to do anything 
>> useful with non-standard arrays (except pass them around), but it 
>> would at least have a way to describe them.
>
>
> On this front, it's probably at least thinking a bit about whether 
> there is any prospect of harmonizing ctypes type notation and the 
> numpy data-type object. It seems somewhat silly to have (1) 
> array.arrays notation for types ['i'. 'f', etc], (2)  ctypes notation 
> for types [c_int, c_float, etc] and (3) numpy's notation for types 
> [dtype('<i4'),  dtype('<f8')].
>
I agree in spirit.   Python is growing several ways to do the same thing 
because determining what data-type you are dealing with is useful in 
many contexts.   Unfortunately, there is not a lot of cross-pollination 
between the people.   I've contributed to a couple of threads to try and 
at least raise awareness that data-description is something NumPy has 
been thinking about, also.


-Travis

[Numpy-discussion] Compiler warnings

[Stephan T. <st...@si...>]

Seems like the setup script currently suppresses quite a lot of 
warnings, not only in MSVC. I created a ticket (#113) with a patch that 
fixes many of the warnings under Visual C. Maybe someone else could have 
a look at the remaining ones.

@Albert: Do you still have a linking problem with the math functions? I 
haven't seen such a warning...

Ciao,
   Stephan

Re: [Numpy-discussion] NumPy, Numeric or numarray or all of them?

[Sasha <nd...@ma...>]

On 5/11/06, Christopher Barker <Chr...@no...> wrote:

> [...]
> > or to stay with Numeric for a while (e.g. 1 year). There is also idea t=
o support both
> > packages.
>
> That's a reasonable option as well. In fact, the APIs are pretty darn
> similar. Also, keep in mind that if you go exclusively to numpy, the
> newest version of Numeric will still work well with it. The conversion
> of numpy to Numeric arrays is very efficient, thanks to them both using
> the new "array protocol". Same goes for numarray.

I have started using the array protocol recently, and it is very
useful.  It allows you to eliminate compile time dependency on any of
the particular array package from your extension modules.  If your
package already has an array-like object, that object should provide
an __array_struct__ attribute and that will make it acceptable to
asarray method from any of the latest array packages (numpy, numarray,
or Numeric). If you have functions that take array as arguments they
should be modified to accept any object that has __array_struct__.

Re: [Numpy-discussion] basearray / arraykit

[Sasha <nd...@ma...>]

On 5/11/06, Tim Hochberg <tim...@co...> wrote:
> [...]
> On this front, it's probably at least thinking a bit about whether there
> is any prospect of harmonizing ctypes type notation and the numpy
> data-type object. It seems somewhat silly to have (1) array.arrays
> notation for types ['i'. 'f', etc], (2)  ctypes notation for types
> [c_int, c_float, etc] and (3) numpy's notation for types [dtype('<i4'),
> dtype('<f8')].
>

Don't forget (4) the struct module (similar to (1), but not exactly
the same).  Also I am not familiar with Boost.Python, but it must have
some way to reflect C++ types to Python.  If anyone on this list uses
Boost.Python, please think if we can borrow any ideas from there.

Re: [Numpy-discussion] NumPy, Numeric or numarray or all of them?

[Christopher B. <Chr...@no...>]

Mateusz =C5=81oskot wrote:
> I'm a developer contributing to GDAL project (http://www.gdal.org).
> GDAL is a core GeoSpatial Librarry that has had Python bindings
> for a while. Recently, we did get some reports from users that GDAL
> bindings do not work with NumPy package.

Speaking as a long time numpy (Numeric, etc.) user, and a new user of=
=20
GDAL, I had no idea GDAL worked with num* at all! at least not direct=
ly.=20
In fact, I thought I was going to have to write that code myself. Whe=
re=20
do I find docs for this? I'm sure I've just missed something, but I'm=
=20
finding the docs a bit sparse.

On the other hand, I am also finding GDAL to be an excellent library,=
=20
and have so far gotten it to do what I need it to do. So kudos!

> So, now we are facing the question what we should do?
> Should we completely port our project to use NumPy

I say yes. At least for new code. numpy is the way of the future, and=
=20
the more projects commit to it the better.

> or to stay with Numeric for a while (e.g. 1 year). There is also id=
ea to support both
> packages.

That's a reasonable option as well. In fact, the APIs are pretty darn=
=20
similar. Also, keep in mind that if you go exclusively to numpy, the=
=20
newest version of Numeric will still work well with it. The conversio=
n=20
of numpy to Numeric arrays is very efficient, thanks to them both usi=
ng=20
the new "array protocol". Same goes for numarray.

> Is it fair to say we are unlikely to see Numeric releases for new
> Pythons in the future?

I'm guessing someone might do a little maintainance. For the short te=
rm,=20
the current Numeric will probably build just fine against the next=
=20
couple releases of python -- python's good that way!

> Can we consider NumPy as the only package in future?
> Simply, we are wondering which Python library we should develop for
> NumPy, Numeric or numarray to be most generally useful.

Speaking as a very interested observer, but not a developer of any of=
=20
the num* packages:

numpy is the way of the future. As an observer, I think that's pretty=
=20
clear. On the other hand, it is still beta software, so dropping Nume=
ric=20
just yet may not be appropriate. If you don't already support numarra=
y,=20
there is no reason to do so now.

It will do the python numerical community a world of good for all of =
us=20
to get back to a single array package.

Note also that we hope some day to get a simple n-d array object into=
=20
the python standard library. When that happens, that object is all bu=
t=20
guaranteed to be compatible with numpy.

-Chris




--=20
Christopher Barker, Ph.D.
Oceanographer
                                     =09=09
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] basearray / arraykit

[Tim H. <tim...@co...>]

Travis Oliphant wrote:

> Christopher Barker wrote:
>
>> Travis Oliphant wrote:
>>
>>> 1) Implement a base-array with no getitem method nor setitem method 
>>> at all
>>>
>>> 2) Implement a sub-class that supports only creation of data-types 
>>> corresponding to existing Python scalars (Boolean, Long-based 
>>> integers, Double-based floats, complex and object types).  Then, all 
>>> array accesses should return the underlying Python objects.
>>> This sub-class should also only do view-based indexing (basically 
>>> it's old Numeric behavior inside of NumPy).
>>
>>
>>> Item 1) should be pushed for inclusion in 2.6  and possibly even 
>>> something like 2)
>>
>>
>> + sys.maxint
>>
>> Having even this very basic n-d object in the standard lib would be a 
>> MAJOR boon to python.
>>
>
> I totally agree.   I've been advertising this for at least 8 months, 
> but nobody is really willing to work on it (or fund it).   At least we 
> have a summer student who is going to try and get Google 
> summer-of-code money for it.  If you have any ability to bump up the 
> ratings of summer of code applications.  Please consider bumping up 
> his application.
>
>> However, as I think about it, one reason I'd really like to see an 
>> nd-array in python is as a standard way to pass binary data around. 
>> Right now, I'm working with the GDAL lib for geo-referenced raster 
>> images, PIL, numpy and wxPython. I'm making a lot of copies to and 
>> from python strings to pass the binary data back and forth. If all 
>> these libs used the same nd-array, this would be much more efficient. 
>> However, that would require non-python data types, most notably a 
>> byte (or char, whatever) type.
>>
> Anything in Python would need to define at least a basic bytes type, 
> for exactly this purpose.  So, we are on the same page.
>
> I'm thinking now that the data-type object now in NumPy would make a 
> nice addition to Python as well for a standard way to define 
> data-types.   Then,  Python itself wouldn't have to do anything useful 
> with non-standard arrays (except pass them around), but it would at 
> least have a way to describe them.

On this front, it's probably at least thinking a bit about whether there 
is any prospect of harmonizing ctypes type notation and the numpy 
data-type object. It seems somewhat silly to have (1) array.arrays 
notation for types ['i'. 'f', etc], (2)  ctypes notation for types 
[c_int, c_float, etc] and (3) numpy's notation for types [dtype('<i4'),  
dtype('<f8')].

It's not clear that this is possible. I at least don't see any clean 
ways to consolidate these off the top of my head, but it's probably 
worth throwing a few of our collective neurons at anyway.

-tim

Re: [Numpy-discussion] basearray / arraykit

[Travis O. <oli...@ie...>]

Christopher Barker wrote:
> Travis Oliphant wrote:
>> 1) Implement a base-array with no getitem method nor setitem method 
>> at all
>>
>> 2) Implement a sub-class that supports only creation of data-types 
>> corresponding to existing Python scalars (Boolean, Long-based 
>> integers, Double-based floats, complex and object types).  Then, all 
>> array accesses should return the underlying Python objects.
>> This sub-class should also only do view-based indexing (basically 
>> it's old Numeric behavior inside of NumPy).
>
>> Item 1) should be pushed for inclusion in 2.6  and possibly even 
>> something like 2)
>
> + sys.maxint
>
> Having even this very basic n-d object in the standard lib would be a 
> MAJOR boon to python.
>

I totally agree.   I've been advertising this for at least 8 months, but 
nobody is really willing to work on it (or fund it).   At least we have 
a summer student who is going to try and get Google summer-of-code money 
for it.  If you have any ability to bump up the ratings of summer of 
code applications.  Please consider bumping up his application.

> However, as I think about it, one reason I'd really like to see an 
> nd-array in python is as a standard way to pass binary data around. 
> Right now, I'm working with the GDAL lib for geo-referenced raster 
> images, PIL, numpy and wxPython. I'm making a lot of copies to and 
> from python strings to pass the binary data back and forth. If all 
> these libs used the same nd-array, this would be much more efficient. 
> However, that would require non-python data types, most notably a byte 
> (or char, whatever) type.
>
Anything in Python would need to define at least a basic bytes type, for 
exactly this purpose.  So, we are on the same page. 


I'm thinking now that the data-type object now in NumPy would make a 
nice addition to Python as well for a standard way to define 
data-types.   Then,  Python itself wouldn't have to do anything useful 
with non-standard arrays (except pass them around), but it would at 
least have a way to describe them.


-Travis

Re: [Numpy-discussion] Creating small arrays from strings and concatenating with empty arrays

[Christopher B. <Chr...@no...>]

Bill Baxter wrote:
> Two quick questions:
> ---------1------------
> Is there any better way to intialize an array from a string than this:
> 
>    A = asarray(matrix("1 2 3"))

How about:

 >>> import numpy as N
 >>> N.fromstring("1 2 3", sep = " ")
array([1, 2, 3])

or

 >>> N.fromstring("1 2 3", dtype = N.Float, sep = " ")
array([ 1.,  2.,  3.])

If you pass a non-empty "sep" parameter, it parses the string, rather 
than treating is as binary data. fromfile works this way too -- thanks 
Travis!

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] basearray / arraykit

[Christopher B. <Chr...@no...>]

Travis Oliphant wrote:
> 1) Implement a base-array with no getitem method nor setitem method at all
> 
> 2) Implement a sub-class that supports only creation of data-types 
> corresponding to existing Python scalars (Boolean, Long-based integers, 
> Double-based floats, complex and object types).  Then, all array 
> accesses should return the underlying Python objects.
> This sub-class should also only do view-based indexing (basically it's 
> old Numeric behavior inside of NumPy).

> Item 1) should be pushed for inclusion in 2.6  and possibly even 
> something like 2)

+ sys.maxint

Having even this very basic n-d object in the standard lib would be a 
MAJOR boon to python.

However, as I think about it, one reason I'd really like to see an 
nd-array in python is as a standard way to pass binary data around. 
Right now, I'm working with the GDAL lib for geo-referenced raster 
images, PIL, numpy and wxPython. I'm making a lot of copies to and from 
python strings to pass the binary data back and forth. If all these libs 
used the same nd-array, this would be much more efficient. However, that 
would require non-python data types, most notably a byte (or char, 
whatever) type.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] NumPy, Numeric or numarray or all of them?

[Travis O. <oli...@ie...>]

Mateusz ?oskot wrote:
> Hi,
>
> I'm a developer contributing to GDAL project (http://www.gdal.org).
> GDAL is a core GeoSpatial Librarry that has had Python bindings
> for a while. Recently, we did get some reports from users that GDAL
> bindings do not work with NumPy package.
>   

Most packages can be "ported" simply by replacing include 
"Numeric/arrayobject.h" with include "numpy/arrayobject.h" and making 
sure the include files are retrieved from the right place. NumPy was 
designed to make porting from Numeric a breeze.
>
> This situation brings some questions we'd like to ask NumPy Dev Team:
>
> Is it fair to say we are unlikely to see Numeric releases for new
> Pythons in the future?
>   
Yes, that's fair. Nobody is maintaining Numeric.
> Can we consider NumPy as the only package in future?
>   
Yes. That's where development is currently active.
> Simply, we are wondering which Python library we should develop for
> NumPy, Numeric or numarray to be most generally useful.
>   

NumPy is the merging of Numeric and numarray to bring people together. I 
think you should develop for NumPy. In practical terms, it is pretty 
easy to port from Numeric.

> What's the recommended way to go now?
>   

I've ported tens of packages to NumPy from Numeric and have had very 
little trouble. It is not difficult. Most of the time, simply replacing 
the *.h file with the one from numpy works fine. It might be a bit 
trickier to get your headers from the right place. The directory is 
returned by

import numpy.distutils
numpy.distutils.misc_util.get_numpy_include_dirs()

Give it a try it's not very difficult.

-Travis

Re: [Numpy-discussion] Who uses matrix?

[Robert H. <he...@ta...>]

I never use matrices primarily because I am worried about one more  
data type floating around in my code.  That is, data is often read in  
or constructed as lists, and must be converted to an array to do  
anything useful.

Take a simple example of optimal interpolation:  Read in the data (as  
a list?), construct the background error covariance arrays (arrays),  
then do about three lines of linear algebra; e.g.,

         W = dot(linalg.inv(B + O), Bi)  				# weights
         A = dot(self.Di,W).transpose() 				# analysis
         Ea = diag(sqrt(self.Be - dot(W.transpose(), Bi)))	# analysis  
error

Is it worth it to convert the arrays to matrices in order to do this  
handful of calculation?  Almost.  I covet the shorthand .T notation  
in matrix object while getting RSI typing in t-r-a-n-s-p-o-s-e.   
Also, for involved calculations inverse, transpose et. al are long  
enough words such that the line always wraps, resulting in less- 
readable code.

Should I give in?  If there was some shorthand links to inverse and  
transpose methods in the array object, I would definitely stick with  
arrays.

-Rob

p.s. By the way, array broadcasting saves much pain in creating the  
background error covariance arrays.  Yeah for array broadcasting!


-----
Rob Hetland, Assistant Professor
Dept of Oceanography, Texas A&M University
p: 979-458-0096, f: 979-845-6331
e: he...@ta..., w: http://pong.tamu.edu

[Numpy-discussion] NumPy, Numeric or numarray or all of them?

[<ma...@lo...>]

Hi,

I'm a developer contributing to GDAL project (http://www.gdal.org).
GDAL is a core GeoSpatial Librarry that has had Python bindings
for a while. Recently, we did get some reports from users that GDAL
bindings do not work with NumPy package.
We've learned on the NumPy website that it's a new derivation from
Numeric code base.

So, now we are facing the question what we should do?
Should we completely port our project to use NumPy or to stay with
Numeric for a while (e.g. 1 year). There is also idea to support both
packages. Python plays a very important role in GDAL project, so our
concern are quite critical for future development.

This situation brings some questions we'd like to ask NumPy Dev Team:

Is it fair to say we are unlikely to see Numeric releases for new
Pythons in the future?
Can we consider NumPy as the only package in future?
Simply, we are wondering which Python library we should develop for
NumPy, Numeric or numarray to be most generally useful.
What's the recommended way to go now?

We'd appreciate your assistance on this issue.

Best regards
-- 
Mateusz Łoskot
http://mateusz.loskot.net

[Numpy-discussion] Creating small arrays from strings and concatenating with empty arrays

[Bill B. <wb...@gm...>]

Two quick questions:
---------1------------
Is there any better way to intialize an array from a string than this:

    A =3D asarray(matrix("1 2 3"))

Or is that as good as it gets?  I suppose it's not so inefficient.

----------2-------------

A lot of time I'd like to be able to write loops like this:

A =3D array([])
for row  in  function_that_returns_iterable_of_one_d_arrays():
     A =3D vstack((A,row))

But that generates an error the first iteration because the shape of A is
wrong.

I could stick in a reshape in the loop:
A =3D array([])
for row  in  function_that_returns_iterable_of_one_d_arrays():
     if not A.shape[0]: A.reshape(0,row.shape[0])
     A =3D vstack((A,row))

Meh, but I don't really like looking at 'if's in loops when I know they're
really only going to be true once the first time.

In Matlab the empty matrix [] can be concatenated with anything and just
takes on it's shape.  It's handy for writing code like the above.

Thanks,
--bill

Re: [Numpy-discussion] Building on Windows

[Stephan T. <st...@si...>]

 >> As for compiler warnings, last time I checked, distutils seems to be
 >> suppressing the output from the compiler, except when the build actually
 >> fails. Or am I mistaken?
 >>
 >>
 > Hmm. I hadn't thought about that. It certainly spits out plenty of
 > warnings when the build fails, so I assumed that it was always spitting
 > out warnings. [Fiddle] Ouch! It does indeed seem to supress warnings on
 > a successful compilation. Anyone know a way to stop that off the top of
 > their head?

A quick fix is to throw out the customized spawn by commenting out
line 40 in distutils/ccompiler.py.


Stephan

Re: [Numpy-discussion] Re: array of arbitrary objects

[Tim H. <tim...@co...>]

Albert Strasheim wrote:

>Hello all
>
>  
>
>>-----Original Message-----
>>From: num...@li... [mailto:numpy-
>>dis...@li...] On Behalf Of Robert Kern
>>Sent: 11 May 2006 05:28
>>To: num...@li...
>>Subject: [Numpy-discussion] Re: array of arbitrary objects
>>
>>Ryan Krauss wrote:
>>    
>>
>>>Is it possible with numpy to create arrays of arbitrary objects?
>>>Specifically, I have defined a symbolic string class with operator
>>>overloading for most simple math operations: 'a'*'b' ==> 'a*b'
>>>
>>>Can I create two matrices of these symbolic string objects and
>>>multiply those matrices together?
>>>
>>>(simply doing array([[a,b],[c,d]]) did not work.  the symbolic strings
>>>got cast to regular strings)
>>>      
>>>
>>Use dtype=object .
>>    
>>
>
>How does one go about putting tuples into an object array?
>
>Consider the following example, courtesy of Louis Cordier:
>
>In [1]: import numpy as N
>In [2]: a = [(1,2), (2,3), (3,4)]
>In [3]: len(a)
>Out[3]: 3
>In [5]: N.array(a, 'O')
>Out[5]:
>array([[1, 2],
>       [2, 3],
>       [3, 4]], dtype=object)
>
>instead of something like this:
>
>array([[(1, 2), (2, 3), (3, 4)]], dtype=object)
>  
>
Creating object arrays is always going to be a bit of a pain, however, 
the following approach will work here:

 >>> import numpy as N
 >>> a = [(1,2), (2,3), (3,4)]
 >>> b = N.zeros([len(a)], object)
 >>> b[:] = a
 >>> b
array([(1, 2), (2, 3), (3, 4)], dtype=object)
 >>> type(b[0])
<type 'tuple'>


Regards,

-tim

[Numpy-discussion] Re: array of arbitrary objects

[Robert K. <rob...@gm...>]

Albert Strasheim wrote:

> How does one go about putting tuples into an object array?

Very carefully.

In [2]: a = empty(3, dtype=object)

In [3]: a
Out[3]: array([None, None, None], dtype=object)

In [4]: a[:] = [(1,2), (2,3), (3,4)]

In [5]: a
Out[5]: array([(1, 2), (2, 3), (3, 4)], dtype=object)

In [6]: a.shape
Out[6]: (3,)

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

RE: [Numpy-discussion] Re: array of arbitrary objects

[Albert S. <fu...@gm...>]

Hello all

> -----Original Message-----
> From: num...@li... [mailto:numpy-
> dis...@li...] On Behalf Of Robert Kern
> Sent: 11 May 2006 05:28
> To: num...@li...
> Subject: [Numpy-discussion] Re: array of arbitrary objects
> 
> Ryan Krauss wrote:
> > Is it possible with numpy to create arrays of arbitrary objects?
> > Specifically, I have defined a symbolic string class with operator
> > overloading for most simple math operations: 'a'*'b' ==> 'a*b'
> >
> > Can I create two matrices of these symbolic string objects and
> > multiply those matrices together?
> >
> > (simply doing array([[a,b],[c,d]]) did not work.  the symbolic strings
> > got cast to regular strings)
> 
> Use dtype=object .

How does one go about putting tuples into an object array?

Consider the following example, courtesy of Louis Cordier:

In [1]: import numpy as N
In [2]: a = [(1,2), (2,3), (3,4)]
In [3]: len(a)
Out[3]: 3
In [5]: N.array(a, 'O')
Out[5]:
array([[1, 2],
       [2, 3],
       [3, 4]], dtype=object)

instead of something like this:

array([[(1, 2), (2, 3), (3, 4)]], dtype=object)

Cheers,

Albert