numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] tentative numpy tutorial

[Pau G. <pau...@gm...>]

hi all,

motivated by the lack of free documentation for NumPy, with some
friends, we started writing a tutorial, that we would like to see in
scipy.org. After some time, the project have started to loose its
initial impetus. Now, we put the current unfinished version in

http://www.scipy.org/Tentative_NumPy_Tutorial

with the hope that, with the help of other users, it will be possible
to build a useful document for beginners.

We are not Numpy experts nor native English speakers (as you probably
realized ;-), so the current version needs to be reviewed. Moreover,
there are huge holes marked with (TODO) tags waiting for motivated
people to fill them.

In my opinion, the lack of free documentation is preventing many users
to get started with NumPy:
  - Many NumPy features are only documented in the Guide to NumPy.
 - (In my opinion) New users are not going to buy the Guide before
knowing about this features.
So there is a kind of dead lock here.

It would be nice, if NumPy 1.0 could be released with a complete free
tutorial, don't you think so? then click the edit button!

thanks,

pau

Re: [Numpy-discussion] A C++ library and the new array interface: the best approach?

[Albert S. <fu...@gm...>]

Hello all

On Mon, 03 Jul 2006, Travis Oliphant wrote:

> Fernando Perez wrote:
> > Hi all,
> >
> > but my lack of familiarity with all the details of new type creation got me a 
> > bit lost.  I'm sure the information I need is all there, but right now I don't 
> > really see the forest with all the leaves in my face.  I've also read the 
> > various recent threads on ctypes, as well was the one initiated by Joris:
> >
> > http://article.gmane.org/gmane.comp.python.numeric.general/6296
> >
> > So I'd like to know if SWIG is really the best way out in this particular case 
> > (and any advice on taking advantage of the array interface via SWIG would be 
> > appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
> > pyrex in other contexts, but I know it doesn't directly support C++.  However, 
> > since I have access to all the code, perhaps a pure C layer could be used to 
> > bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
> > can be an option?
> 
> I don't have a lot of time to respond right now.  But here are some 
> thoughts.
> 
> I'm not sure I'd use SWIG at this point. 

At least for C code, I would definately agree. If you already know SWIG, 
you might want to try it, but I started without any knowledge of SWIG 
and ctypes when I wanted to wrap a pretty basic library, and after a 
week of puzzling over SWIG, I learned ctypes and wrapped the library in 
a matter of hours.

> I'd probably either
> 
> a) use ctypes to call the C++ library (if it is compiled as a *shared* 
> library) although to be honest I'm not sure if calling a C++ causes any 
> grief. 

If I understand correctly, because each C++ compiler potentially mangles 
C++ identifier names in a different way, there isn't a way for ctypes 
to know what to look for in a shared library.
 
> b) byte-the bullet and use Boost python (which other's who have 
> integrated C++ code with Python seem to really like).  I've never 
> wrapped my mind around Boost python, though, so I'm not much help there.

If there is any way for you to "hide" you C++ code behind an extern "C" 
interface, i.e. write some C-style wrappers on top of your C++ code, 
then you can use ctypes. As luck would have it, this is exactly what the 
libsvm authors did already, which made wrapping libsvm with ctypes 
relatively straightforward.

If you want to achieve things like Python classes deriving from C++ 
classes and vice versa, Boost-Python is what you want to look at. But 
as others have noted, prepare to wait for the compiler when wrapping 
anything of even moderate size.

In Boost-Python world there's Pyste that allows you to generate code. I 
think Pyste uses GCC-XML underneath. I played with Pyste for a day or 
two long ago, but at that stage the code it made differed greatly from 
what I wrote when doing Boost-Python wrapping by hand, so it was a bit 
strange.

In general, I got the feeling that Pyste wasn't flexible enough to wrap 
any kind of C++ library. But maybe you're lucky and your C++ code is 
just right to wrap easily with Pyste.

> c) write a custom Tensor class that either inherits from the ndarray or 
> is a mix-in class with the ndarray as a component.

Don't know much about this option.

In summary (keeping in mind that I'm a bit of a ctypes zealot at 
present), I'd try the "C interface on top of C++ with ctypes" approach, 
and build an ndarray subclass on top these C functions.

Regards,

Albert

Re: [Numpy-discussion] Ctypes support in NumPy

[Albert S. <fu...@gm...>]

Hello all

> > In this case, I think one wants strides[0] (the row stride) to return
> 40.
> >
> 
> Why do you think that?
> 
> All sliced arrays keep the same strides information as their
> "parents".   This is the essence of a "view".  The striding is exactly
> the same as before (the data hasn't moved anywhere), only the starting
> point and the bounds have changed.

Sorry, I was suffering some temporary strides brain damage. :-)

Regards,

Albert

Re: [Numpy-discussion] MA bug or feature?

[Pierre GM <pgm...@ma...>]

Michael,
I wonder whether the Mask class you suggest is not a bit overkill. There 
should be enough tools in the existing MA module to do what we want. And I 
don't wanna think about compatibility the number of changes in the MA code 
that'd be required (but I'm lazy)...

For the sake of consistency and optimization, I still think it could be easier 
(and cleaner) to make `nomask` the default for a MaskedArray without masked 
values. That could for example be implemented by forcing `nomask` at the 
creation of the MaskedArray with an extra 
`if mask and not mask.any(): mask=nomask`, or by using Paul's 
make_mask( flag=1) trick.

Masking some specific values could still be done when mask is nomask with an 
intermediary MA.getmaskarray() step.

On a side note, modifying an existing mask is a delicate matter. Everything's 
OK if you use masks as a way to hide existing data, it's more complex when 
initially you have some holes in your dataset...

Re: [Numpy-discussion] MA bug or feature?

[Michael S. <mic...@gm...>]

On 6/23/06, Paul Dubois <pfd...@gm...> wrote:
> In the original MA in Numeric, I decided that to constantly check for masks
> that didn't actually mask anything was not a good idea. It punishes normal
> use with a very expensive check that is rarely going to be true.
>
> If you are in a setting where you do not want this behavior, but instead
> want masks removed whenever possible, you may wish to wrap or replace things
> like masked_array so that they call make_mask with flag = 1:
>
> y = masked_array(data, make_mask(maskdata, flag=1))
>
> y will have no mask if maskdata is all false.
>

Hi Paul. If this is purely for optimisation, is there perhaps a better
way to do so in which the optimisation is hidden? For example if the
optimisation is in regard to the case in which none of the elements is
masked, an alternative approach may be to make a subclass of ndarray
and cache the result of the any method. e.g.

import numpy as N

def asmask(data):
    if isinstance(data, Mask):
        return data
    else:
        return Mask(data)

class Mask(N.ndarray):
    __array_priority__ = 10.0
    def __new__(subtype, data):
        ret = N.array(data, N.bool_)
        return ret.view(Mask)

    def __init__(self, data):
        self._any = None

    def any(self):
        if self._any is None:
            self._any = N.ndarray.any(self)
        return self._any

    def __setitem__(self, index, value):
        self._any = None
        N.ndarray.__setitem__(self, index, value)

The biggest problem I have with the current setup is the inconsistency
between the behaviour of the array when the mask is nomask vs a
boolarray with all False. Another example of this is when one changes
the mask on an element. This is not possible when the mask is nomask

print N.version.version
ma1 = N.ma.array([1,2,3], mask=[False, False, False])
print ma1.mask
ma1.mask[2] = True
ma2 = N.ma.array([1,2,3], mask=False)
print ma2.mask
ma2.mask[2] = True

----- output
0.9.8
[False False False]
[False False True]
False
Traceback (most recent call last):
  File "D:\eclipse\Mask\src\mask\__init__.py", line 111, in ?
    ma2.mask[2] = True
TypeError: object does not support item assignment

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Robert K. <rob...@gm...>]

Robert Kern wrote:
> Bill Baxter wrote:
>> How does one use this Trac thing again?  I couldn't find a "submit a 
>> bug" link anywhere on the web scipy.org <http://scipy.org> web page.  I 
>> found it once before, but now I can't seem to locate it.
> 
> http://projects.scipy.org/scipy/scipy
> 
> You need to register first. Click the "Register" link in the upper right.

Or rather, http://projects.scipy.org/scipy/numpy

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Robert K. <rob...@gm...>]

Bill Baxter wrote:
> How does one use this Trac thing again?  I couldn't find a "submit a 
> bug" link anywhere on the web scipy.org <http://scipy.org> web page.  I 
> found it once before, but now I can't seem to locate it.

http://projects.scipy.org/scipy/scipy

You need to register first. Click the "Register" link in the upper right.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Bill B. <wb...@gm...>]

I just noticed that array.transpose() takes either a tuple or separate
arguments.  I'm not trying to imply anything, just I was thinking such a
thing had no precedent in numpy, but it turns out it does.


And as an aside, the docstring for .transpose() needs fixing. It says:
>>> a = array([[1,2],[3,4]])
>>> a
array([[1, 2],
       [3, 4]])
>>> a.transpose()
array([[1, 3],
       [3, 4]])

Whoa -- transpose destroys any 2's in your input and turns them into 3's!

How does one use this Trac thing again?  I couldn't find a "submit a bug"
link anywhere on the web scipy.org web page.  I found it once before, but
now I can't seem to locate it.

--bb

[Numpy-discussion] Research study on CVS - you can help and get paid!

[Eric G. <egi...@cs...>]

I am a Computer Science graduate student at the University of  
Illinois at Urbana-Champaign. I need your help in a study I want to  
conduct. I have created a project called LifeSource that visualizes  
CVS repositories. You can find screenshots and info here:

http://social.cs.uiuc.edu/people/gilbert (info on me)
http://social.cs.uiuc.edu/people/gilbert/research.html (info on  
LifeSource, the CVS project)

LifeSource appeared as a Work-in-Progress paper at ACM's CHI 2006 in  
Montreal (http://www.chi2006.org). I applied LifeSource to the  
Azureus and Gaim CVS repositories. I would like to apply the same  
visualization to your
project. Next, I want to see if you as the visualized community find  
it accurate, helpful, disturbing, etc.

In order to participate in the study, you will need to view the  
LifeSource visualization once a week (on the web) for four weeks and  
respond to two short online surveys.

You will receive a $20 gift certificate for your participation.

I can fully explain the process in the near future - just email me  
back. I will provide all of the information of the methodology of the  
study online. (We can set up a blog for study members to leave  
further comments.)

Please let me know what you think. I plan on using the study in a  
long paper to CHI 2007.

Cheers,
Eric

P.S. I apologize for the spam and will send no more.

Re: [Numpy-discussion] Does zero based indexing drive anyone else crazy?

[Keith G. <kwg...@gm...>]

On 7/2/06, Webb Sprague <web...@gm...> wrote:

> Given the long history of python and its ancestry in C (for which zero
> based indexing made lots of sense since it dovetailed with thinking in
> memory offsets in systems programming), there is probably nothing to
> be done now.  I guess I just want to vent, but also to ask if anyone
> has found any way to deal with this issue in their own scientific
> programming.

Aha! Guido himself prefers starting the index at one. Here's a code
snippet from a fun article he wrote about optimizing python code:

    import time
    def timing(f, n, a):
        print f.__name__,
        r = range(n)
        t1 = time.clock()
        for i in r:
            f(a); f(a); f(a); f(a); f(a); f(a); f(a); f(a); f(a); f(a)
        t2 = time.clock()
        print round(t2-t1, 3)

http://www.python.org/doc/essays/list2str/

Notice he chose t1 and t2 instead of t0 and t1.

QED

Re: [Numpy-discussion] A C++ library and the new array interface: the best approach?

[Andrew S. <str...@as...>]

Dear Fernando,

A couple of quick thoughts:

A) Despite my love of Pyrex, it probably isn't the tool for the job --
it doesn't play particularly well with C++, and it would require you to
write custom code for every function wrapped.

B) It sounds like you want something that will more-or-less
automatically generate the python interface from the header files. To my
mind, this suggests SWIG or boost. Boost uses some seriously fancy (and
mind-bending) C++, which can either be a good thing or a bad thing, but
it certainly boosts compile time! There are doubtless other ways. For
example, perhaps you could try using gccxml to auto-generate something
like a Python/ctypes wrapper.

C) The core of whatever you do will probably involve writing code to
view a Tensor as an object that has the __array_struct__ interface. That
way numpy/scipy/whatever can use the data with no copying. So that's a
good place to start and whatever you learn there will probably be useful
regardless of whatever tools you end up with. You'll probably also want
the converse function so that you can view objects exposing
__array_struct__ (numpy arrays) as a Tensor.

Anyhow, that's not much, but perhaps it'll help.

Cheers!
Andrew

Fernando Perez wrote:

>Hi all,
>
>I'd like to ask for a bit of guidance on the best path to follow regarding the 
>integration of an existing C++ library with numpy, given the recent 
>developments on these topics:
>
>1. the new array interface specification
>2. the ctypes discussions
>3. pyrex
>
>Our situation is the following: an existing in-house C++ library defines a 
>templated class (Tensor), in many ways similar to a numpy array, but with its 
>own extra functionality for other things.  There are many functions in this 
>code which take or return Tensors.  I now need to interface this library with 
>a Python code which is fully numpy-based, and hopefully be able to go back and 
>forth between the C++ and python sides without unnecessary copies.
>
>Since the C++ codebase is somewhat large and already exists, and will continue 
>to evolve as a standalone C++ system, something tells me I should just bite 
>the typemap bullet and SWIG the darn thing.  But I've never written anything 
>beyond the most trivial typemaps, and I'm not 100% sure on exactly how to take 
>advantage of the new array interface with SWIG.  I read all the docs under
>
>  - http://svn.scipy.org/svn/PEP
>
>  - http://www.scipy.org/BaseArray
>
>  - http://numeric.scipy.org/array_interface.html
>
>  - the pyrex/array interface wiki pages (the site is not responding right 
>now, so I can't give a URL)
>
>
>but my lack of familiarity with all the details of new type creation got me a 
>bit lost.  I'm sure the information I need is all there, but right now I don't 
>really see the forest with all the leaves in my face.  I've also read the 
>various recent threads on ctypes, as well was the one initiated by Joris:
>
>http://article.gmane.org/gmane.comp.python.numeric.general/6296
>
>
>So I'd like to know if SWIG is really the best way out in this particular case 
>(and any advice on taking advantage of the array interface via SWIG would be 
>appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
>pyrex in other contexts, but I know it doesn't directly support C++.  However, 
>since I have access to all the code, perhaps a pure C layer could be used to 
>bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
>can be an option?
>
>Any advice will be greatly appreciated.
>
>Best,
>
>f
>
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>

[Numpy-discussion] Namespaces in header files [Was: Time for beta1 of NumPy 1.0]

[Lee T. <lt...@ll...>]

On Thu, 29 Jun 2006, Travis Oliphant wrote:

> I think it's time for the first beta-release of NumPy 1.0
>
> I'd like to put it out within 2 weeks.  Please make any comments or
> voice major concerns so that the 1.0 release series can be as stable as
> possible.

One issue I ran across that I have not seen addressed is the namespace of 
arrayobject.h.  I'm not refering to C++ namespaces but prefixing symbols 
to avoid clashes with user's code.

The externals start with PyArray.  But I had symbol redefinition errors 
for byte, MAX_DIMS, and ERR.  That is, I already had defines for MAX_DIMS 
and ERR and a typedef for byte in my code.  When adding a numpy interface 
to my library I had to undef these symbols before including arrayobject.h.

Is there a way to move implemention defines, like ERR, into a separate 
header.  Or if they're part of the API, prefix the symbols?

Lee Taylor

Re: [Numpy-discussion] A C++ library and the new array interface: the best approach?

[Travis O. <oli...@ie...>]

Fernando Perez wrote:
> Hi all,
>
> but my lack of familiarity with all the details of new type creation got me a 
> bit lost.  I'm sure the information I need is all there, but right now I don't 
> really see the forest with all the leaves in my face.  I've also read the 
> various recent threads on ctypes, as well was the one initiated by Joris:
>
> http://article.gmane.org/gmane.comp.python.numeric.general/6296
>
>
> So I'd like to know if SWIG is really the best way out in this particular case 
> (and any advice on taking advantage of the array interface via SWIG would be 
> appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
> pyrex in other contexts, but I know it doesn't directly support C++.  However, 
> since I have access to all the code, perhaps a pure C layer could be used to 
> bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
> can be an option?
>   

I don't have a lot of time to respond right now.  But here are some 
thoughts.

I'm not sure I'd use SWIG at this point. 

I'd probably either

a) use ctypes to call the C++ library (if it is compiled as a *shared* 
library) although to be honest I'm not sure if calling a C++ causes any 
grief. 

b) byte-the bullet and use Boost python (which other's who have 
integrated C++ code with Python seem to really like).  I've never 
wrapped my mind around Boost python, though, so I'm not much help there.

c) write a custom Tensor class that either inherits from the ndarray or 
is a mix-in class with the ndarray as a component.

If it were me, I'd probably be doing a combination of a and c.

I've just finished the chapter in my book that covers and compares f2py, 
weave, pyrex and ctypes integration.   I've been waiting on finishing 
this chapter before sending updates.  You should get the update by the 
end of the week.


Perhaps more later.


-Travis



> Any advice will be greatly appreciated.
>
> Best,
>
> f
>
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Charles R H. <cha...@gm...>]

<snip>

+1 for tuples,

Because dimensions *are* tuples. I think using tuples is a good habit to get
into and agree with Robert Kern's comments about using the new random number
interface instead of the old compatibility functions rand and randn.

Chuck

[Numpy-discussion] A C++ library and the new array interface: the best approach?

[Fernando P. <Fer...@co...>]

Hi all,

I'd like to ask for a bit of guidance on the best path to follow regarding the 
integration of an existing C++ library with numpy, given the recent 
developments on these topics:

1. the new array interface specification
2. the ctypes discussions
3. pyrex

Our situation is the following: an existing in-house C++ library defines a 
templated class (Tensor), in many ways similar to a numpy array, but with its 
own extra functionality for other things.  There are many functions in this 
code which take or return Tensors.  I now need to interface this library with 
a Python code which is fully numpy-based, and hopefully be able to go back and 
forth between the C++ and python sides without unnecessary copies.

Since the C++ codebase is somewhat large and already exists, and will continue 
to evolve as a standalone C++ system, something tells me I should just bite 
the typemap bullet and SWIG the darn thing.  But I've never written anything 
beyond the most trivial typemaps, and I'm not 100% sure on exactly how to take 
advantage of the new array interface with SWIG.  I read all the docs under

  - http://svn.scipy.org/svn/PEP

  - http://www.scipy.org/BaseArray

  - http://numeric.scipy.org/array_interface.html

  - the pyrex/array interface wiki pages (the site is not responding right 
now, so I can't give a URL)


but my lack of familiarity with all the details of new type creation got me a 
bit lost.  I'm sure the information I need is all there, but right now I don't 
really see the forest with all the leaves in my face.  I've also read the 
various recent threads on ctypes, as well was the one initiated by Joris:

http://article.gmane.org/gmane.comp.python.numeric.general/6296


So I'd like to know if SWIG is really the best way out in this particular case 
(and any advice on taking advantage of the array interface via SWIG would be 
appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
pyrex in other contexts, but I know it doesn't directly support C++.  However, 
since I have access to all the code, perhaps a pure C layer could be used to 
bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
can be an option?

Any advice will be greatly appreciated.

Best,

f

Re: [Numpy-discussion] iterate along a ray: linear algebra?

[Torgil S. <tor...@gm...>]

I've done something similar a few years ago (numarray,numeric). I
started roughly at the middle and did 64 points from a reference point
(xc,yc). This point together with a point at the edge of the image
(xp,yp) also defined a reference angle (a0). (ysize,xsize) is the
shape of the intensity image.

I used the following code to calculate points of interest:

na=64
xr,yr=xsize-xc,ysize-yc
a0=arctan2(yp-yr,xp-xc)
if a0<0: a0+=2*pi
ac=arctan2([yr,yr,-yc,-yc],[xr,-xc,-xc,xr])
if numarray:
    ac[ac<0]+=2*pi
else:
    ac=choose(ac<0,(ac,ac+2*pi))
a1,a2,a3,a4=ac
rmaxfn={
    0: lambda a: a<=a1 and xr/cos(a-0.0*pi) or yr/cos(0.5*pi-a),
    1: lambda a: a<=a2 and yr/cos(a-0.5*pi) or xc/cos(1.0*pi-a),
    2: lambda a: a<=a3 and xc/cos(a-1.0*pi) or yc/cos(1.5*pi-a),
    3: lambda a: a<=a4 and yc/cos(a-1.5*pi) or xr/cos(2.0*pi-a)
    }
angles=arange(a0,a0+2*pi,2*pi/na)
if numarray:
    angles[angles>=2*pi]-=2*pi
else:
    angles=choose(angles>=2*pi,(angles,angles-2*pi))
nr=int(ceil(sqrt(max(yc,yr)**2+max(xc,xr)**2)))
crmax=array([int(floor(rmaxfn[floor(a*2/pi)](a))) for a in angles])
cr=outerproduct(ones(na),arange(float(nr)))
ca=outerproduct(angles,ones(nr))
x=cr*cos(ca)+xc
y=cr*sin(ca)+yc

After this I did cubic spline interpolation in the image with these
points and did something useful. I don't know how relevant this is to
you and it doesn't use the linear algebra package but it might give
you some hint.

If you find out a nifty way to do your rays please post on this thread.

Sidenote -- Watch my explicit float argument to arange and even
putting in pi there in one case. There's a discussion on this list
that floats in arange are troublesome


On 6/30/06, stephen emslie <ste...@gm...> wrote:
> I am in the process of implementing an image processing algorithm that
> requires following rays extending outwards from a starting point and
> calculating the intensity derivative at each point. The idea is to find the
> point where the difference in intensity goes beyond a particular threshold.
>
> Specifically I'm examining an image of an eye to find the pupil, and the
> edge of the pupil is a sharp change in intensity.
>
> How does one iterate along a line in a 2d matrix, and is there a better way
> to do this? Is this a problem that linear algebra can help with?
>
> Thanks
> Stephen Emslie
>
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>

Re: [Numpy-discussion] dtype error in cumsum

[Sasha <nd...@ma...>]

It did get through all right. SF is a mess.  In any case what you
observe is clearly a bug.  Looks like uninitialized memory somewhere.
Please file a bug report at
http://projects.scipy.org/scipy/numpy/newticket .

On 7/3/06, Nikhil Padmanabhan <npa...@pr...> wrote:
> (I sent the following from my gmail account, but that does not seem
> to have been accepted by sourceforge -- apologies in advance if it
> shows up twice)
>
> Hi,
>
> (I saw some discussion of this on the mailing list, but was not sure
> if there was a solution reached, so I thought I'd ask again....) If I
> try to use cumsum on a numpy array with the dtype keyword, I get the
> wrong answer if dtype does not match that of the array. More
> specifically,
>
> Python 2.4.3 (#1, Apr  3 2006, 18:07:18)
> [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> ->> import numpy
> ->> numpy.__version__
> '0.9.9.2727'
> ->> a = numpy.arange(10002)
> ->> a.cumsum(dtype='f4')[-1]
> 0.0
> ->> a.cumsum(dtype='f8')[-1]
> 0.0
> ->> a.cumsum()[-1]
> 50015001
> ->> a.sum()
> 50015001
> ->> a.sum(dtype='f8')
> 50015001.0
>
> Note that this only seems to affect cumsum(); sum() works just fine.
> And cumsum works fine for small arrays....
>
> Am I misunderstanding what the dtype keyword in cumsum is, or doing
> something silly? I should not be overflowing any type bounds (I
> think...)
>
> This is with the SVN numpy, on an iBook G4 running OS X 10.4.6
>
> Thanks in advance,
> -- Nikhil
>
> ------------------------------------
> Nikhil Padmanabhan
> npa...@pr...
> http://wwwphy.princeton.edu/~npadmana
> nikhil@(609) 258-4355
>
>
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

[Numpy-discussion] dtype error in cumsum

[Nikhil P. <npa...@Pr...>]

(I sent the following from my gmail account, but that does not seem  
to have been accepted by sourceforge -- apologies in advance if it  
shows up twice)

Hi,

(I saw some discussion of this on the mailing list, but was not sure  
if there was a solution reached, so I thought I'd ask again....) If I  
try to use cumsum on a numpy array with the dtype keyword, I get the  
wrong answer if dtype does not match that of the array. More  
specifically,

Python 2.4.3 (#1, Apr  3 2006, 18:07:18)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
->> import numpy
->> numpy.__version__
'0.9.9.2727'
->> a = numpy.arange(10002)
->> a.cumsum(dtype='f4')[-1]
0.0
->> a.cumsum(dtype='f8')[-1]
0.0
->> a.cumsum()[-1]
50015001
->> a.sum()
50015001
->> a.sum(dtype='f8')
50015001.0

Note that this only seems to affect cumsum(); sum() works just fine.  
And cumsum works fine for small arrays....

Am I misunderstanding what the dtype keyword in cumsum is, or doing  
something silly? I should not be overflowing any type bounds (I  
think...)

This is with the SVN numpy, on an iBook G4 running OS X 10.4.6

Thanks in advance,
-- Nikhil

------------------------------------
Nikhil Padmanabhan
npa...@pr...
http://wwwphy.princeton.edu/~npadmana
nikhil@(609) 258-4355

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Sasha <nd...@ma...>]

On 7/3/06, Sven Schreiber <sve...@gm...> wrote:
> ...
> I was arguing that it's inconvenient exactly because it's inconsistent,
> so I don't see the tradeoff here. (Given that I'm fairly indifferent
> between one or two pairs of parentheses.)
>

It is convenient to be able to write zeros(5) and rand(5,5) instead of
consistent zeros([5]) and rand([5,5]).   Note that inconsistency will
not be fully cured by allowing rand([5,5]) because rand([5,5],
float32) will still not work.

Actually, I can see some value in a random numbers generator that
conforms to the ones/zeros interface because that may promote use of
random data in testing instead of ones/zeros.

How about

     rands(shape, dtype=float, low=default_low(dtype), high=default_high(dtype))

?

Where default_{low,high}(floatN) should probably be {0,1} but for
integer types some other default may make more sense.

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Sven S. <sve...@gm...>]

Sasha schrieb:

> 
> Consistency is already lost because 1d case allows both ones(5) and
> ones([5]) (and even ones((5,)) if anyone can tolerate that
> abomination).  I don't think those who argue for sequence only are
> willing to require ones([5]).

Imho consistency is not lost there precisely because one can use
ones((5)) if one is so inclined. So the double-parentheses habit (or
call it tuple-habit if you like) goes through. In contrast, rand((5,5))
fails, and that is what breaks consistency.

> 
> Remember, "A Foolish Consistency is the Hobgoblin of Little Minds"
> (Ralph Waldo Emerson (1803–1882),  adopted without attribution as a
> section heading in PEP 8 <http://www.python.org/dev/peps/pep-0008>).

A "little mind" is an accurate description of myself in the numpy field.
But I believe that in order to become a success, numpy must take into
account the needs of little minds as well.

> 
> I think the current situation strikes the right balance between
> convenience and consistency.

I was arguing that it's inconvenient exactly because it's inconsistent,
so I don't see the tradeoff here. (Given that I'm fairly indifferent
between one or two pairs of parentheses.)

In any case I promise to shut up about this when 1.0(beta) is out, but I
think Alan is right that under the status quo there will be a constant
stream of the same newbie question that I asked.

cheers,
Sven

Re: [Numpy-discussion] Ctypes support in NumPy

[Travis O. <oli...@ie...>]

Albert Strasheim wrote:
> Stefan and I did some more experiments and it seems like .ctypes.strides
> isn't doing the right thing for subarrays.
>
> For example:
>
> In [52]: x = N.rand(3,4)
> In [57]: [x.ctypes.strides[i] for i in range(x.ndim)]
> Out[57]: [32, 8]
>
> This looks fine. But for this subarray:
>
> In [56]: [x[1:3,1:4].ctypes.strides[i] for i in range(x.ndim)]
> Out[56]: [32, 8]
>
> In this case, I think one wants strides[0] (the row stride) to return 40.
>   

Why do you think that? 

All sliced arrays keep the same strides information as their 
"parents".   This is the essence of a "view".  The striding is exactly 
the same as before (the data hasn't moved anywhere), only the starting 
point and the bounds have changed.
> .ctypes.data already seems to do the right thing:
>
> In [60]: x.ctypes.data
> Out[60]: c_void_p(31685288)
>
> In [61]: x[1:3,1:4].ctypes.data
> Out[61]: c_void_p(31685328)
>
> In [62]: 31685288-31685328
> Out[62]: 40
>
> What would be a good way of dealing with discontiguous arrays? It seems like
> one might want to disable their .ctypes attribute.
>
>   
No, not at all.  Discontiguous arrays are easily handled simply by using 
the strides information to step through the array in each dimension 
instead of "assuming" contiguousness. 

Perhaps there is some confusion about what the strides actually represent.

It's quite easy to write C-code that takes stride information as well 
which will then work with discontiguous arrays.  The benefit of this 
approach is that you avoid copying data when you don't really have to.  
There should be very little performance penalty in most algorithms as 
well as the strides calculation is not much more than adding 1 to the 
current pointer.

-Travis

Re: [Numpy-discussion] None as missing value

[Sven S. <sve...@gm...>]

Pierre GM schrieb:
>> I was also a bit surprised at the following behavior:
>>>> a = numpy.asarray([1,1])
>>>> a
> array([1, 1])
>>>> a[0]=numpy.nan
>>>> a
> array([0, 1])
> 
> Seems to affect only the int_ arrays:
> 
>>>> a = numpy.asarray([1,1], dtype=float_)
>>>> a
> array([1., 1.])
>>>> a[0]=numpy.nan
>>>> a
> array([        nan,  1.        ])
> 

Sure it works with floats. The question is, should there maybe be an
error if the type of the assigned value doesn't match the dtype of the
array, instead of silently doing something unexpected?
Consider the following (with still the same *integer* array a from my
post above):

>>> a[0]=0.1
>>> a
array([0, 1])
>>> a[0]='0'
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
TypeError: an integer is required

This TypeError makes a lot of sense, but why does it only react to
strings, not to floats?

Cheers,
Sven

[Numpy-discussion] dtype conversion errors in cumsum

[Nikhil P. <nik...@gm...>]

Hi,

(I saw some discussion of this on the mailing list, but was not sure  
if there was a solution reached...) If I try to use cumsum on a numpy  
array with the dtype keyword, I get the wrong answer if dtype does  
not match that of the array. More specifically,

Python 2.4.3 (#1, Apr  3 2006, 18:07:18)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
->> import numpy
->> numpy.__version__
'0.9.9.2727'
->> a = numpy.arange(10002)
->> a.cumsum(dtype='f4')[-1]
0.0
->> a.cumsum(dtype='f8')[-1]
0.0
->> a.cumsum()[-1]
50015001
->> a.sum()
50015001
->> a.sum(dtype='f8')
50015001.0

Note that this only seems to affect cumsum(); sum() works just fine.  
And cumsum works fine for small arrays....

Am I misunderstanding what the dtype keyword in cumsum is, or doing  
something silly? I should not be overflowing any type bounds (I  
think...)

This is with the SVN numpy, on an iBook G4 running OS X 10.4.6

Thanks in advance,
-- Nikhil

------------------------------------
Nikhil Padmanabhan
nik...@gm...

Re: [Numpy-discussion] Does zero based indexing drive anyone else crazy?

[Christopher B. <Chr...@no...>]

Webb Sprague wrote:

> it is far more natural to program if the indices are aligned with the
> counts of the elements

I suggest that it's only more natural if that's what you're used to -- 
i.e. you come form other languages that do it that way.

I fairly rarely get bitten by indexing 1, rather than zero, but I save a 
lot of errors that I used to get in MATLAB by the way python does slices:

len(a[i:j]) == j - i

and:

l[:j] + l[j:] == l

or:

r_[a[:i],a[i:]] == a

for numpy arrays.

I suppose you could have one-indexing and the python slicing, but I 
think that would be even more error prone.

>  zero
> based indexing made lots of sense since it dovetailed with thinking in
> memory offsets in systems programming

it also dovetails nicely into using an array to represent a grid of values:

i = (X - MinX) / deltaX
rather than
i = (X - MinX) / deltaX + 1

X = i*deltaX
rather than
X = (i-1)*deltaX

In Fortran, you can choose where you want your array indexing to start, 
and I found myself starting with zero more often than 1, and I was never 
a C programmer.

 > I guess I just want to vent, but also to ask if anyone
> has found any way to deal with this issue in their own scientific
> programming.

You'll get used to it. There are disadvantages either way, but after 
switching from primarily Matlab to primarily Python, I like zero-based 
indexing better. Perhaps you will too.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Args for ones, zeros, rand, eye, ones, empty (possible 1.0 change?)

[Christopher B. <Chr...@no...>]

Tim Hochberg wrote:
> I also find:
> 
>   ones([5, 5], dt)
> 
> clearer than:
> 
>    ones(5, 5, dt)
> 
> or, more dramatic, consider: ones([dx, dy], dt) versus ones(dx, dy, dt). 
> Brrrr!

And I imagine this is EXACTLY why Numeric was originally designed that 
way: when there are more arguments to pass, then a shape is essentially 
a single entity. Also consider this:

MyShape = a.shape

b = ones(MyShape, N.float_)

much better than:

b = ones(*MyShape, N.float_)

At least I think so.

> Kahan is oft quoted by Tim Peters who seems to have one of the better 
> grasps of the fiddly aspects of floating point in the Python community, 
> so I give his views a lot of weight.

I took at class with Kahan at Berkeley (that I barely passed), he is 
definitely someone to listen to.

> NumPy really has at 
>> least two "users"  1) application developers and 2) prototype developers 

FWIW: I came from a heavy MATLAB background, still do far more 
prototyping that really application writing (though I certainly do the 
later), and I want:

>> 1) longish_explanatory names
>> 2) consistent calling syntax
(note that I've edited those a bit...)

And above all:
>> 3) strict name-space control

In my experience that that is a difference between "prototyping" and 
"using Python like Matlab". The sooner that you learn to use Python like 
Python, rather than trying to use it like Matlab, the better off you'll 
be. Python is an excellent language for prototyping, but even for that, 
you're better off working with, rather than against, its features.

As for the current issue:

1) Make everything take a sequence for the size argument.
2) dump the "convenience functions" into a compatibility module.

Is it so hard to write:

import numpy.random.uniform as rand

(though I notice that numpy.random.rand() is the inconsistent 
"convenience function" -- darn)


Sasha wrote:
> Consistency is already lost because 1d case allows both ones(5) and
> ones([5]) (and even ones((5,)) if anyone can tolerate that
> abomination).  I don't think those who argue for sequence only are
> willing to require ones([5]).

But I'd be happy to ban ones(5) (by the way, doesn't Matlab's ones(5) 
produce a 5X5 matrix?). We certainly don't' want to require a list, that 
would totally go against Python's concept of duck typing.



-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...