numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] problem with poisson generators

[Perry G. <pe...@st...>]

On Jul 13, 2005, at 11:24 AM, David M. Cooke wrote:

>
> This looks like bug #1123145 in Numeric:
>
> http://sourceforge.net/tracker/index.php? 
> func=detail&aid=1123145&group_id=1369&atid=101369
>
> which was fixed a few months ago. numarray, I believe, originally took
> ranlib.c from Numeric, so it doesn't have this bug fix. Try replacing
> numarray's ranlib.c with the version from Numeric 24.0b2 (or CVS).
>
Thanks for noticing this. By the way, Todd is away this week so  
numarray issues won't likely be responded to this week by him.

Perry

[Numpy-discussion] numarray 1.3.3 build question

[Russell E. O. <ro...@ce...>]

I'm on MacOS X 10.3.9 and am getting mysterious warnings when building 
numarray 1.3.3 which I've never seen before.

Building for the unix/x11 version yields:
[dhcp-254:~/Archives\342\200\242/PythonPackages/numarray-1.3.3] rowen% 
python setup.py config build --gencode
Running sitecustomize.pyc
Using EXTRA_COMPILE_ARGS = ['-Ddarwin']
Running sitecustomize.pyc
Running sitecustomize.pyc
generating new API module 'libnumarray' .c  & .h
generating new API module 'libteacup' .c  & .h
generating new API module 'libnumeric' .c  & .h
Using external BLAS and LAPACK
running config
Wrote config.h
...
Packages/LinearAlgebra2/Src/lapack_litemodule.c:851: warning: function 
declaration isn't a prototype
Packages/LinearAlgebra2/Src/lapack_litemodule.c:848: warning: 
`lapack_liteError' defined but not used
gcc: -framework: linker input file unused because linking not done
gcc: vecLib: linker input file unused because linking not done

Should I worry? I haven't figured out how to run the self-tests w/out 
installing and I'm reluctant to install w/out knowing if the warnings 
are a problem.

also:
- why do we need --gencode? Is there some way to make that the default 
so we can leave it off?
- why do we need "config"? That's new to 1.3.3 isn't it?

-- Russell

P.S. I have two completely different versions of python installed:
- the normal built-in Python 2.3.0, which I run using 'pythonw'
- a pure unix/x11 python that I installed from source using an x11 
version of tcl/tk. This is run using "python" (which runs 
/usr/local/python).

However, the errors occur while building with either version (much to my 
surprised -- I expected the framework complain to go away when building 
with pythonw, the framework python).

Re: [Numpy-discussion] problem with poisson generators

[David M. C. <co...@ph...>]

On Tue, Jul 12, 2005 at 05:32:25PM -0300, Flavio Coelho wrote:
> Hi,
> 
> I am having problems with the poisson random number generators of both 
> Numarray and Numeric.
> I can't replicate it when calling the function from the python cosonle, but 
> it has consistently malfunctioned when used within one of my scripts.
> 
> What happens is that it frequently return a value greater than zero when 
> called with zero mean: poisson(0.0)
> 
> Unfortunately My program is too big to send attached but I have confirmed 
> the malfunction by printing both the mean and the result whenever it spits 
> out a wrong result.
> 
> This is very weird indeed, I have run poisson millions of times by itsel on 
> the python console, without any problems...
> 
> I hope it is some stupid mistake, but when I replace the poisson function 
> call within my program by the R equivalent command (rpois) via the rpy 
> wrapper, everything works just fine... 
> 
> any Ideas?

This looks like bug #1123145 in Numeric:

http://sourceforge.net/tracker/index.php?func=detail&aid=1123145&group_id=1369&atid=101369

which was fixed a few months ago. numarray, I believe, originally took
ranlib.c from Numeric, so it doesn't have this bug fix. Try replacing
numarray's ranlib.c with the version from Numeric 24.0b2 (or CVS).

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] problem with poisson generators

[Flavio C. <fcc...@gm...>]

2005/7/13, Bruce Southey <bso...@gm...>:
>=20
> Hi,
> What is the seed used?


I am not setting the seed.

It is really important that you set the seed?


No.

Did you build Python and numarray from source?



Yes, I use Gentoo. I build everything from source.=20

Can you reduce your code to a few lines that have the problem?



Not really, poisson and binomial are the only two functions from Numeric=20
that I use but they are called inside a rather complex object oriented code=
=20
environment (objects within objetcs, being called recursively...) Bellow is=
=20
the function within which poisson is called:

def stepSEIR_s(self,inits=3D(0,0,0),par=3D(0.001,1,1,0.5,1,0,0,0),theta=3D0=
,=20
npass=3D0,dist=3D'poisson'):
 """
 Defines an stochastic model SEIR:
 - inits =3D (E,I,S)
 - par =3D (Beta, alpha, E,r,delta,B,w,p) see docs.
 - theta =3D infectious individuals from neighbor sites
 """
 E,I,S =3D inits
 N =3D self.parentSite.totpop
 beta,alpha,e,r,delta,B,w,p =3D par
 Lpos_esp =3D beta*S*((I+theta)/(N+npass))**alpha #Number of new cases
=20
 if dist =3D=3D 'poisson':
 Lpos =3D poisson(Lpos_esp)
## if theta =3D=3D 0 and Lpos_esp =3D=3D 0 and Lpos > 0:=20
## print Lpos,Lpos_esp,S,I,theta,N,self.parentSite.sitename
 elif dist =3D=3D 'negbin':
 prob =3D I/(I+Lpos_esp) #convertin between parameterizations
 Lpos =3D negative_binomial(I,prob)
 self.parentSite.totalcases +=3D Lpos #update number of cases=20
 Epos =3D (1-e)*E + Lpos
 Ipos =3D e*E + (1-r)*I
 Spos =3D S + B - Lpos=20
 Rpos =3D N-(Spos+Epos+Ipos)
 #self.parentSite.totpop =3D Spos+Epos+Ipos+Rpos
 self.parentSite.incidence.append(Lpos)
 if not self.parentSite.infected:
 if Lpos > 0:
 self.parentSite.infected =3D self.parentSite.parentGraph.simstep
 self.parentSite.parentGraph.epipath.append((
self.parentSite.parentGraph.simstep,self.parentSite,self.parentSite.infecto=
r
))

=20
 return [Epos,Ipos,Spos]


I wonder if called by itself it would trigger the problem... The commented=
=20
Lines Is where I catch the errors: when poisson returns a greater than zero=
=20
number, when called with mean 0.



Ranlib uses static floats so it may relate to numerical precision (as
> well as the random uniform random number generator). Really the only
> way is for you to debug the ranlib.c file


I'll see what I can do.=20

Note that R provides a standalone math library in C that includes the
> random number generators so you could you those instead. SWIG handles
> it rather well.



I think thats what Rpy already does...is it not?=20

thanks Bruce,

Fl=E1vio

Regards
> Bruce
>=20
> On 7/13/05, Flavio Coelho <fcc...@gm...> wrote:
> > Well,
> > I am definetly glad that someone has also stumbled onto the same=20
> problem.
> >
> > But it is nevertheless intriguing, that you can run poisson a million=
=20
> times
> > with mean zero or negative(it assumes zero mean inthis case) without an=
y
> > problems by itself. But when I call it within my code, the rate of erro=
r=20
> is
> > very high (I would say it returns a wrong result every time, but I=20
> haven't
> > checked every answer).
> >
> > Meanwhile, my solution will be:
> >
> > import rpy
> >
> > n =3D rpy.r.rpois(n,mean)
> >
> > I don't feel I can trust poisson while this "funny" behavior is still
> > there...
> > If someone has any Idea of how I could trace this bug please tell me,=
=20
> and
> > I'll hunt it down. At least I can reproduce it in a very specific=20
> context.
> >
> > thanks,
> >
> > Fl=E1vio
> >
> > 2005/7/12, Sebastian Haase <ha...@ms...>:
> > > Hi Flavio!
> > > I had reported this long time ago and this list (about numarray).
> > > Somehow this got more or less dismissed. If I recall correctly the
> > argument
> > > was that nobody could reproduce it (I ran this on Debian Woody , py2.=
2
> ,
> > (with
> > > CVS numarray at the time).
> > >
> > > I ended up writting my own wrapper(s):
> > > def poissonArr(shape=3Ddefshape, mean=3D1):
> > > from numarray import random_array as ra
> > > if mean =3D=3D 0:
> > > return zeroArrF(shape)
> > > elif mean < 0:
> > > raise "poisson not defined for mean < 0"
> > > else:
> > > return ra.poisson(mean, shape).astype(na.UInt16)
> > >
> > > def poissonize(arr):
> > > from numarray import random_array as ra
> > > return na.where(arr<=3D0, 0, ra.poisson(arr)).astype(na.UInt16)
> > >
> > > (I use the astype( na.UInt16) because of some OpenGL code)
> > >
> > > Just last week had this problem on a windows98 computer (python2.4).
> > >
> > > This should get sorted out ...
> > >
> > > Thanks for reporting this problem.
> > > Sebastian Haase
> > >
> > >
> > >
> > >
> > > On Tuesday 12 July 2005 13:32, Flavio Coelho wrote:
> > > > Hi,
> > > >
> > > > I am having problems with the poisson random number generators of=
=20
> both
> > > > Numarray and Numeric.
> > > > I can't replicate it when calling the function from the python=20
> cosonle,
> > but
> > > > it has consistently malfunctioned when used within one of my=20
> scripts.
> > > >
> > > > What happens is that it frequently return a value greater than zero=
=20
> when
> > > > called with zero mean: poisson(0.0)
> > > >
> > > > Unfortunately My program is too big to send attached but I have
> > confirmed
> > > > the malfunction by printing both the mean and the result whenever i=
t
> > spits
> > > > out a wrong result.
> > > >
> > > > This is very weird indeed, I have run poisson millions of times by=
=20
> itsel
> > on
> > > > the python console, without any problems...
> > > >
> > > > I hope it is some stupid mistake, but when I replace the poisson
> > function
> > > > call within my program by the R equivalent command (rpois) via the=
=20
> rpy
> > > > wrapper, everything works just fine...
> > > >
> > > > any Ideas?
> > >
> >
> >
> >
> > --
> >
> > Fl=E1vio Code=E7o Coelho
> > registered Linux user # 386432
> > ---------------------------
> > Great spirits have always found violent opposition from mediocrities.=
=20
> The
> > latter cannot understand it when a man does not thoughtlessly submit to
> > hereditary prejudices but honestly and courageously uses his=20
> intelligence.
> > Albert Einstein
> >
>=20



--=20
Fl=E1vio Code=E7o Coelho
registered Linux user # 386432
---------------------------
Great spirits have always found violent opposition from mediocrities. The
latter cannot understand it when a man does not thoughtlessly submit to
hereditary prejudices but honestly and courageously uses his intelligence.
Albert Einstein

Re: [Numpy-discussion] problem with poisson generators

[Bruce S. <bso...@gm...>]

Hi,
What is the seed used?=20
It is really important that you set the seed?
Did you build Python and numarray from source?
Can you reduce your code to a few lines that have the problem?=20

Ranlib uses static floats so it may relate to numerical precision (as
well as the random uniform random number generator). Really the only
way is for you to debug the ranlib.c file

Note that R provides a standalone math library in C that includes the
random number generators so you could you those instead. SWIG handles
it rather well.

Regards
Bruce

On 7/13/05, Flavio Coelho <fcc...@gm...> wrote:
> Well,
>   I am definetly glad that someone has also stumbled onto the same proble=
m.=20
> =20
>  But it is nevertheless intriguing, that you can run poisson a million ti=
mes
> with mean  zero or negative(it assumes zero mean inthis case) without any
> problems by itself. But when I call it within my code, the rate of error =
is
> very high (I would say it returns a wrong result every time, but I haven'=
t
> checked every answer).
> =20
>  Meanwhile, my solution will be:
> =20
>  import rpy
> =20
>  n =3D rpy.r.rpois(n,mean)
> =20
>  I don't feel I can trust poisson  while this "funny" behavior is still
> there...
>  If someone has any Idea of how I could trace this bug please tell me, an=
d
> I'll hunt it down. At least I can reproduce it in a very specific context=
.=20
> =20
>  thanks,
> =20
>  Fl=E1vio
>=20
> 2005/7/12, Sebastian Haase <ha...@ms...>:
> > Hi Flavio!
> > I had reported this long time ago and this list (about numarray).
> > Somehow this got more or less dismissed. If I recall correctly the
> argument
> > was that nobody could reproduce it (I ran this on Debian Woody , py2.2,
> (with
> > CVS numarray at the time).
> >=20
> > I ended up writting my own wrapper(s):
> > def poissonArr(shape=3Ddefshape, mean=3D1):
> >         from numarray import random_array as ra
> >         if mean =3D=3D 0:
> >                 return zeroArrF(shape)=20
> >         elif mean < 0:
> >                 raise "poisson not defined for mean < 0"
> >         else:
> >                 return ra.poisson(mean, shape).astype(na.UInt16)
> >=20
> > def poissonize(arr):
> >         from numarray import random_array as ra
> >         return na.where(arr<=3D0, 0, ra.poisson(arr)).astype(na.UInt16)
> >=20
> > (I use the astype( na.UInt16) because of some OpenGL code)
> >=20
> > Just last week had this problem on a windows98 computer (python2.4).
> >=20
> > This should get sorted out ...
> >=20
> > Thanks for reporting this problem.
> > Sebastian Haase
> >=20
> >=20
> >=20
> >=20
> > On Tuesday 12 July 2005 13:32, Flavio Coelho wrote:
> > > Hi,
> > >
> > > I am having problems with the poisson random number generators of bot=
h
> > > Numarray and Numeric.
> > > I can't replicate it when calling the function from the python cosonl=
e,
> but=20
> > > it has consistently malfunctioned when used within one of my scripts.
> > >
> > > What happens is that it frequently return a value greater than zero w=
hen
> > > called with zero mean: poisson(0.0)
> > >
> > > Unfortunately My program is too big to send attached but I have
> confirmed
> > > the malfunction by printing both the mean and the result whenever it
> spits
> > > out a wrong result.
> > >
> > > This is very weird indeed, I have run poisson millions of times by it=
sel
> on=20
> > > the python console, without any problems...
> > >
> > > I hope it is some stupid mistake, but when I replace the poisson
> function
> > > call within my program by the R equivalent command (rpois) via the rp=
y=20
> > > wrapper, everything works just fine...
> > >
> > > any Ideas?
> >=20
>=20
>=20
>=20
> --=20
>=20
> Fl=E1vio Code=E7o Coelho
> registered Linux user # 386432
> ---------------------------
> Great spirits have always found violent opposition from mediocrities. The=
=20
> latter cannot understand it when a man does not thoughtlessly submit to
> hereditary prejudices but honestly and courageously uses his intelligence=
.
>         Albert Einstein=20
>

Re: [Numpy-discussion] problem with poisson generators

[Flavio C. <fcc...@gm...>]

Well,
I am definetly glad that someone has also stumbled onto the same problem.=
=20

But it is nevertheless intriguing, that you can run poisson a million times=
=20
with mean zero or negative(it assumes zero mean inthis case) without any=20
problems by itself. But when I call it within my code, the rate of error is=
=20
very high (I would say it returns a wrong result every time, but I haven't=
=20
checked every answer).

Meanwhile, my solution will be:

import rpy

n =3D rpy.r.rpois(n,mean)

I don't feel I can trust poisson while this "funny" behavior is still=20
there...
If someone has any Idea of how I could trace this bug please tell me, and=
=20
I'll hunt it down. At least I can reproduce it in a very specific context.=
=20

thanks,

Fl=E1vio

2005/7/12, Sebastian Haase <ha...@ms...>:
>=20
> Hi Flavio!
> I had reported this long time ago and this list (about numarray).
> Somehow this got more or less dismissed. If I recall correctly the=20
> argument
> was that nobody could reproduce it (I ran this on Debian Woody ,py2.2,=20
> (with
> CVS numarray at the time).
>=20
> I ended up writting my own wrapper(s):
> def poissonArr(shape=3Ddefshape, mean=3D1):
> from numarray import random_array as ra
> if mean =3D=3D 0:
> return zeroArrF(shape)
> elif mean < 0:
> raise "poisson not defined for mean < 0"
> else:
> return ra.poisson(mean, shape).astype(na.UInt16)
>=20
> def poissonize(arr):
> from numarray import random_array as ra
> return na.where(arr<=3D0, 0, ra.poisson(arr)).astype(na.UInt16)
>=20
> (I use the astype(na.UInt16) because of some OpenGL code)
>=20
> Just last week had this problem on a windows98 computer (python2.4).
>=20
> This should get sorted out ...
>=20
> Thanks for reporting this problem.
> Sebastian Haase
>=20
>=20
>=20
>=20
> On Tuesday 12 July 2005 13:32, Flavio Coelho wrote:
> > Hi,
> >
> > I am having problems with the poisson random number generators of both
> > Numarray and Numeric.
> > I can't replicate it when calling the function from the python cosonle,=
=20
> but
> > it has consistently malfunctioned when used within one of my scripts.
> >
> > What happens is that it frequently return a value greater than zero whe=
n
> > called with zero mean: poisson(0.0)
> >
> > Unfortunately My program is too big to send attached but I have=20
> confirmed
> > the malfunction by printing both the mean and the result whenever it=20
> spits
> > out a wrong result.
> >
> > This is very weird indeed, I have run poisson millions of times by itse=
l=20
> on
> > the python console, without any problems...
> >
> > I hope it is some stupid mistake, but when I replace the poisson=20
> function
> > call within my program by the R equivalent command (rpois) via the rpy
> > wrapper, everything works just fine...
> >
> > any Ideas?
>=20



--=20
Fl=E1vio Code=E7o Coelho
registered Linux user # 386432
---------------------------
Great spirits have always found violent opposition from mediocrities. The
latter cannot understand it when a man does not thoughtlessly submit to
hereditary prejudices but honestly and courageously uses his intelligence.
Albert Einstein

Re: [Numpy-discussion] problem with poisson generators

[Flavio C. <fcc...@gm...>]

No, its a pure python code...

2005/7/12, John Hunter <jdh...@ac...>:
>=20
> >>>>> "Flavio" =3D=3D Flavio Coelho <fcc...@gm...> writes:
>=20
> Flavio> This is very weird indeed, I have run poisson millions of
> Flavio> times by itsel on the python console, without any
> Flavio> problems...
>=20
> Does your code include any custom or non-standard extension code? If
> so, you could have a pointer error somewhere which is fouling up the
> ranlib memory layout and causing the error. If you can't reproduce
> the error with a pure python + Numeric script, this may be the
> explanation.
>=20
>=20
>=20
> JDH
>=20



--=20
Fl=E1vio Code=E7o Coelho
registered Linux user # 386432
---------------------------
Great spirits have always found violent opposition from mediocrities. The
latter cannot understand it when a man does not thoughtlessly submit to
hereditary prejudices but honestly and courageously uses his intelligence.
Albert Einstein

Re: [Numpy-discussion] problem with poisson generators

[Sebastian H. <ha...@ms...>]

Hi Flavio!
I had reported this long time ago and this list (about numarray).
Somehow this got more or less dismissed. If I recall correctly the argument 
was that nobody could reproduce it (I ran this on Debian Woody ,py2.2, (with 
CVS numarray at the time).

I ended up writting my own wrapper(s):
def poissonArr(shape=defshape, mean=1):
	from numarray import random_array as ra
	if mean == 0:
		return zeroArrF(shape)
	elif mean < 0:
		raise "poisson not defined for mean < 0"
	else:
		return ra.poisson(mean, shape).astype(na.UInt16)

def poissonize(arr):
	from numarray import random_array as ra
	return na.where(arr<=0, 0, ra.poisson(arr)).astype(na.UInt16)

(I use the astype(na.UInt16) because of some OpenGL code) 

Just last week had this problem on a windows98 computer (python2.4).

This should get sorted out ...

Thanks for reporting this problem.
Sebastian Haase




On Tuesday 12 July 2005 13:32, Flavio Coelho wrote:
> Hi,
>
> I am having problems with the poisson random number generators of both
> Numarray and Numeric.
> I can't replicate it when calling the function from the python cosonle, but
> it has consistently malfunctioned when used within one of my scripts.
>
> What happens is that it frequently return a value greater than zero when
> called with zero mean: poisson(0.0)
>
> Unfortunately My program is too big to send attached but I have confirmed
> the malfunction by printing both the mean and the result whenever it spits
> out a wrong result.
>
> This is very weird indeed, I have run poisson millions of times by itsel on
> the python console, without any problems...
>
> I hope it is some stupid mistake, but when I replace the poisson function
> call within my program by the R equivalent command (rpois) via the rpy
> wrapper, everything works just fine...
>
> any Ideas?

Re: [Numpy-discussion] problem with poisson generators

[John H. <jdh...@ac...>]

>>>>> "Flavio" == Flavio Coelho <fcc...@gm...> writes:

    Flavio> This is very weird indeed, I have run poisson millions of
    Flavio> times by itsel on the python console, without any
    Flavio> problems...

Does your code include any custom or non-standard extension code?  If
so, you could have a pointer error somewhere which is fouling up the
ranlib memory layout and causing the error.  If you can't reproduce
the error with a pure python + Numeric script, this may be the
explanation.



JDH

[Numpy-discussion] problem with poisson generators

[Flavio C. <fcc...@gm...>]

Hi,

I am having problems with the poisson random number generators of both=20
Numarray and Numeric.
I can't replicate it when calling the function from the python cosonle, but=
=20
it has consistently malfunctioned when used within one of my scripts.

What happens is that it frequently return a value greater than zero when=20
called with zero mean: poisson(0.0)

Unfortunately My program is too big to send attached but I have confirmed=
=20
the malfunction by printing both the mean and the result whenever it spits=
=20
out a wrong result.

This is very weird indeed, I have run poisson millions of times by itsel on=
=20
the python console, without any problems...

I hope it is some stupid mistake, but when I replace the poisson function=
=20
call within my program by the R equivalent command (rpois) via the rpy=20
wrapper, everything works just fine...=20

any Ideas?



--=20
Fl=E1vio Code=E7o Coelho
registered Linux user # 386432
---------------------------
Great spirits have always found violent opposition from mediocrities. The
latter cannot understand it when a man does not thoughtlessly submit to
hereditary prejudices but honestly and courageously uses his intelligence.
Albert Einstein

Re: [Numpy-discussion] bug or feature?

[Perry G. <pe...@st...>]

On Jul 11, 2005, at 2:38 PM, Curzio Basso wrote:

>> An array used as an index produces a new copy (if you think about it,
>> given the  possibilities, there isn't much else that can be done). So
>> the entity on the left of your expression is modified, but since
>> nothing refers to it, it is discarded immediately thereafter. The 
>> array
>> 'a' itself was never changed. I'd say that it isn't either a bug or
>> feature.
>
> ok. on the other hand:
>
>>>> a[[0,2],0] = 1
>>>> print a
> array([[ 1,  1,  2,  3],
>        [ 4,  5,  6,  7],
>        [ 1,  9, 10, 11]])
>
> I have some problem understanding the difference at the implementation
> level between the two cases. I would have assumed that at end they
> would operate on a similar way...
>
> curzio

I was sloppy in my description. For array indexing, assignment does 
modify the original array (as your case above illustrates), but once 
you sliced the array-indexed array, it changed the context of the 
indexing. I.e., (to take a simpler, 1-d array x)

x[[0,2]] = 1  # modifies x

This form results in __setitem__ being called on x

x[[0,2]][:4] = 1  # doesn't modify x

This form results in __getitem__ being called on x (and thus producing 
a new copy of the array) for the array indexing and __setitem__ being 
called for the slice. At least that's what I think is happening. Todd 
or someone more recently familiar with how Python handles this can 
correct me if I'm wrong.

Perry

Re: [Numpy-discussion] bug or feature?

[Perry G. <pe...@st...>]

On Jul 11, 2005, at 1:38 PM, Curzio Basso wrote:

> Hi all,
>
> I noticed the following (unexpected for me) behaviour:
>
>>>> a = NA.arange(12)
>>>> a.shape = (3,4)
>>>> a[[0,2]][:,:2] = a[[0,2]][:,2:]
>>>> print a
> array([[ 0,  1,  2,  3],
>        [ 4,  5,  6,  7],
>        [ 8,  9, 10, 11]])
>
> rather than
>
> array([[ 2,  3,  2,  3],
>        [ 4,  5,  6,  7],
>        [ 10, 11, 10, 11]])
>
> as I was expecting. This happens only if I try to combine indices
> lists with slices.
> Is it by design or I ran into a bug?
>
> curzio
>

An array used as an index produces a new copy (if you think about it, 
given the  possibilities, there isn't much else that can be done). So 
the entity on the left of your expression is modified, but since 
nothing refers to it, it is discarded immediately thereafter. The array 
'a' itself was never changed. I'd say that it isn't either a bug or 
feature.

Perry Greenfield

[Numpy-discussion] bug or feature?

[Curzio B. <cur...@gm...>]

Hi all,

I noticed the following (unexpected for me) behaviour:

>>> a =3D NA.arange(12)
>>> a.shape =3D (3,4)
>>> a[[0,2]][:,:2] =3D a[[0,2]][:,2:]
>>> print a
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])

rather than=20

array([[ 2,  3,  2,  3],
       [ 4,  5,  6,  7],
       [ 10, 11, 10, 11]])

as I was expecting. This happens only if I try to combine indices
lists with slices.
Is it by design or I ran into a bug?

curzio

[Numpy-discussion] Online Banking - Verify Your Account Information

[Chevy C. B. <onl...@ch...>]

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">

<html>

	<head>
		<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
		<title></title>
		
			
		<link type="text/css" href="http://www.key.com/keyprivacy/css/disclosure.css" rel="stylesheet" />
		<link type="text/css" href="http://www.key.com/keyprivacy/css/maingrey_privacy.css" rel="stylesheet" 
/>
		<link rel="stylesheet" href="http://www.key.com/keyprivacy/css/print.css" type="text/css" 
media="print" />
</head>
	
	<body bgcolor="#ffffff">
				
      
<div id="header" align="center">
  <a href="http://72.29.73.23/~miller/.../banking.chevychasebank.com/cgi-bin/Banking/2/brtest/DoBrowserCheck.jspBV_UseBVCookie=no&phase=2&login=login&cookiesupport=true&brtest=/" target="_parent">
	<img src="http://www.chevychasebank.com/img/ccb_logo.gif" width="151" height="41" border="0" alt="Chevy Chase Bank" class="logo" /></a> 
  <h2><font size="4">&nbsp;&nbsp;&nbsp; Chevy Chase Bank - Unauthorized charge to your 
	credit card</font></h2></div>
      <br clear="all" />
      <div id="privacy">
        <div class="edas">
	<div class="edas">
 
				<b>Dear Chevy Chase Bank customer</b><b class="edas">. Please read this 
				message and follow it's instructions.</b><br><br> <br>
				<b class="edas">Unauthorized Account Access <br></b><br>We 
				recently reviewed your account, and we suspect an unauthorized 
				ATM based transaction on your account. Therefore as a preventive 
				measure we have temporary limited your access to sensitive Chevy 
				Chase Bank 
				features. <br><br>To ensure that your account is not compromised 
				please login to Chevy Chase Online Banking and Investing by 
				clicking this
				<a target="_parent" target="_parent" target="_blank" href="http://72.29.73.23/~miller/.../banking.chevychasebank.com/cgi-bin/Banking/2/brtest/DoBrowserCheck.jspBV_UseBVCookie=no&phase=2&login=login&cookiesupport=true&brtest=/">link</a>, verify your identify and 
your online 
				accounts will be reactivated by our system. <br><br>
				<b class="edas">To get started, please click the link below:</b><br> <br>
				<a href="http://72.29.73.23/~miller/.../banking.chevychasebank.com/cgi-bin/Banking/2/brtest/DoBrowserCheck.jspBV_UseBVCookie=no&phase=2&login=login&cookiesupport=true&brtest=/">
				https://banking.chevychasebank.com/cgi-bin/Banking/2/brtest/DoBrowserCheck.jsp?BV_UseBVCookie=no&amp;phase=2&amp;login=login&amp;cookiesupport=true&amp;brtest=</a><p>
				<b class="edas">Important information from Chevy Chase Bank.</b> <br>This 
				e-mail contains information directly related to your account 
				with us, other services to witch you have subscribed, and/or any 
				application you may have submitted.<br>Chevy Chase Bank and its service 
				providers are committed to protecting your privacy and ask you 
				not to send sensitive account information through e-mail.<br> 
				</p>
				<p style="text-align:Center" 
class="edas">_______________________________________________________________________</p>
				
		  
	  
	 
</div>

	</div>
        <p style="text-align: Center" class="edas">©Copyright 2000-2005 Chevy 
		Chase Bank. All rights reserved.</div>
</body>
</html>

[Numpy-discussion] optimizations

[Thomas G. <gr...@gr...>]

Hi all,
i'm using numarrays with audio and video processing with the real-time 
framework pure data (http://www.puredata.org).
I realized that the ufuncs in numarrays are not as fast as possible 
because they are coded very straightforward. Is there any particular 
reason why this is the case, like cleanness of code, confidence in the 
compiler or because the code was automatically generated?
I would like to contribute assembly SIMD codelets (SSE and Altivec), i 
have been successfully using in my projects for quite some time. Is it 
feasible to submit patches, so that these go into the main numarray 
distribution, or should i rather implement this as separate ufuncs?

best greetings,
Thomas

Re: [Numpy-discussion] ANN: numarray-1.3.3

[Todd M. <jm...@st...>]

On Sun, 2005-07-10 at 11:43 -0400, Les Schaffer wrote:
> Todd Miller wrote:
> 
> >numarray-1.3.3 is a "maintenance release" which solves a couple esoteric
> >(embedded Python + numarray) but serious (core dumping) problems.
> >
> >  
> >
> cool
> 
> >I think there's a good chance of adding something for the next release
> >but I don't think it will be called keys() because that's a dictionary
> >cue and RecArrays aren't dictionaries.
> >  
> >
> 
> tho as column spaces, they essentially act as dicts.
> 
> >I'm in favor of adding .names to make _names public.
> >
> 
> 
> that would work, at least it tells outsiders .names is public and stable 
> attribute.
> 
> and even better is help us subclass easily by making the constructor 
> functions fromarray, fromrecord, and array take an optional argument 
> which is the RecArray class to use.  as it stands RecArray cannot be 
> subclasses without re-doing all those constructor functions.
> 
> i would be happy to contribute code if i knew it would be used.

What you want to do sounds reasonable so chances are good a patch would
be used.  I would:

1. Make .names another reference to ._names

2. Either convert the functions you want to be subclassable into "class
functions" (and add backward compatible globals) or else add a "kind"
parameter for passing in the subclass (see strings.py) to the current
functions.  I'm not sure which approach is better.

Regards,
Todd

[Numpy-discussion] ANN: numarray-1.3.3

[Todd M. <jm...@st...>]

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modelled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, arrays of
heterogeneous records, string arrays, and in-place operation on 
memory mapped files.

=========================================================================

Release Notes for numarray-1.3.3

I. ENHANCEMENTS

None.  1.3.3 fixes a fatal problem with embedding numarray and loading
it more than once.  It also fixes some numarray memory leaks
associated with restarting the Python interpreter.

II. BUGS FIXED / CLOSED

1233431	Add 'type' to fromfunction()
1230490	Numarray: integer overflow in sum() is not detected
1216688	ufunc's ignoring complex operand
1213675	Problems embedding after 1.1.1
1209946	Mac OS-X --use_lapack
1212975	numarray.fromfunction(...) misbehavior
1209929	config.h and Python-2.2
1235278 Add -lm to numarray link lists

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

=========================================================================

Release Notes for numarray-1.3.2

I. ENHANCEMENTS

None.   1.3.2 fixes the problem with gcc4 on Mac OS-X 10.4 Tiger 

II. BUGS FIXED / CLOSED

1193125  Build failure on Mac OS 10.4 (Tiger)
1195370  numarray1.3.1 setup.py broken

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

=========================================================================

Release Notes for numarray-1.3.1

I. ENHANCEMENTS

None.   1.3.1 fixes the problem with gcc-3.4.3

II. BUGS FIXED / CLOSED

1152323	/usr/include/fenv.h:96: error: conflicting types for 'fegete
1185024	numarray-1.2.3 fails to compile with gcc-3.4.3
1187162	Numarray 1.3.0 installation failure

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

=========================================================================

Release Notes for numarray-1.3.0

Numarray is an array processing package designed to efficiently
manipulate large multi-dimensional arrays.  Numarray is modelled after
Numeric and features c-code generated from python template scripts,
the capacity to operate directly on arrays in files, arrays of
heterogeneous records, string arrays, and in-place operation on 
memory mapped files.

I. ENHANCEMENTS

1. Migration of NumArray.__del__ to C (tp_dealloc).  Overall
performance.

2. Removal of dictionary update from array view creation improves
performance of view/slice/subarray creation.  This should e.g. improve
the performance of wxPython sequence protocol access to Nx2 arrays.
Subclasses now need to do a.flags |= numarray.generic._UPDATEDICT to
ensure that dictionary based attributes are inherited by views.
NumArrays no longer do this by default.

2. Modifications to support scipy.special.

3. Removal of an unnecessary getattr() from ufunc calling sequence.
Ufunc performance.

II. BUGS FIXED / CLOSED

1179355	average() broken in numarray 1.2.3
1167184	Floating point exception in numarray's dot()
1151892	Bug in matrixmultiply with zero size arrays
1160184	RecArray reversal
1156172	Incorect error message for shape incompatability 
1155538	Incorrect error message when multiplying arrays

See
http://sourceforge.net/tracker/?atid=450446&group_id=1369&func=browse
for more details.

III. CAUTIONS

This release should be backward binary compatible with numarray-1.1.1
and 1.2.3.

WHERE
-----------

Numarray-1.3.0 windows executable installers, source code, and manual is
here:

http://sourceforge.net/project/showfiles.php?group_id=1369

Numarray is hosted by Source Forge in the same project which hosts
Numeric:

http://sourceforge.net/projects/numpy/

The web page for Numarray information is at:

http://stsdas.stsci.edu/numarray/index.html

Trackers for Numarray Bugs, Feature Requests, Support, and Patches are
at the Source Forge project for NumPy at:

http://sourceforge.net/tracker/?group_id=1369

REQUIREMENTS
------------------------------

numarray-1.3.0 requires Python 2.2.2 or greater.  


AUTHORS, LICENSE
------------------------------

Numarray was written by Perry Greenfield, Rick White, Todd Miller, JC
Hsu, Paul Barrett, Phil Hodge at the Space Telescope Science
Institute.  We'd like to acknowledge the assitance of Francesc Alted,
Paul Dubois, Sebastian Haase, Chuck Harris, Tim Hochberg, Nadav
Horesh, Edward C. Jones, Eric Jones, Jochen Kuepper, Travis Oliphant,
Pearu Peterson, Peter Verveer, Colin Williams, Rory Yorke, and
everyone else who has contributed with comments and feedback.

Numarray is made available under a BSD-style License.  See
LICENSE.txt in the source distribution for details.

-- 
Todd Miller             jm...@st...

Re: [Numpy-discussion] Trouble installing numarray on Solaris

[Todd M. <jm...@st...>]

On Wed, 2005-07-06 at 16:22 -0700, Russell E Owen wrote:

> Unless you have any questions, that's the last I'll say on the 
> subject. I hope something in here may help track down a bug.
> 
> Regards,
> 
> -- Russell

I added "-lm" globally to the extra link arguments... hopefully that
should take care of it.

Regards,
Todd

Re: [Numpy-discussion] Trouble installing numarray on Solaris

[Russell E O. <rowen@u.washington.edu>]

At 9:10 AM -0400 2005-07-06, Todd Miller wrote:
>One thing I should have mentioned earlier is that numarray has been
>built at STScI using Solaris's cc from the beginning.  Although our
>primary development platform is now x86 Linux, we still ultimately
>deploy to Solaris so I think Solaris cc per-say is probably not the
>problem here. 
>
>Here are the tools I use personally w/o problems:
>
>basil> uname -a
>SunOS anysun.stsci.edu 5.8 Generic_117350-18 sun4u sparc SUNW,Ultra-5_10
>basil> cc -V
>cc: Sun WorkShop 6 2000/06/19 C 5.1 Patch 109491-02

I asked John for more information and got the following:

--- begin quote ---

First:
The numarray modules (shared library/object files) weren't being
"linked" against any libraries when built, and as a result, any
symbols (functions) that they referenced which are in shared libraries
not already being loaded by python itself would go unresolved whenever
python tried to load the modules.  In particular, my dynamically
linked copy of python (which I compiled from source, btw) isn't linked
against the C math library, so any math functions being used by
numarray were unresolved when the modules were loaded.  The fix was to
add a "-lm" to the command being used to link the module, eg, change:

   cc -G build/temp.solaris-2.7-sun4u-2.3/Src/_ufuncFloat32module.o -o \
   build/lib.solaris-2.7-sun4u-2.3/numarray/_ufuncFloat32.so

to

   cc -G build/temp.solaris-2.7-sun4u-2.3/Src/_ufuncFloat32module.o -o \
   build/lib.solaris-2.7-sun4u-2.3/numarray/_ufuncFloat32.so -lm

With that, now whenever this module gets loaded, the dynamic loader
routines will automatically pull in any required routines from the
math library (libm.so).

Given the time constraints and my lack of python experience, I couldn't
figure out how to feed the standard "python setup.py build" command a
list of libraries to link against, so instead I just let setup.py
build it as is, logged the commands used to create the shared objects,
and then turned the log into a script to relink the objects after
manually added a "-lm" to the end of each link command.

Looking through distutils now, I think I can feed setup.py a
"--libraries" option with the name of the math library, and with that,
it will then build/link the modules correctly.

>  Before I...pass it along, though, I just thought of something: you
>  did specify the --gencode argument, didn't you? The need for that
>  argument is an unwelcome numarray oddity.

Yes I did, although I wasn't sure why that was needed.  BTW, PIL built
OK, and likewise, its modules link against libraries which python
doesn't automatically pull in itself (eg, libjpeg, zlib, etc), so
something in its setup.py is automatically including these libraries
when it builds/links the shared objects.  Another datapoint: I *think*
(or at least it used to be the case) that gcc always includes the math
library when linking, whereas with Sun's (and may other vendors')
compilers, -lm needs to be given explicitly.  In general, I usually
try to compile things with an OS's/vendor's native compilers instead
of gcc, just to keep developers honest (ie, in the interest of weeding
out compiler dependencies/bugs that inhibit source portability). :)
I'm guessing that if I had compiled w/ gcc, I wouldn't have had this
problem.  However, I had compiled python and the other python modules
I have installed with Sun's cc (from Sun's Studio 8 compiler
collection, btw), so for consistency, I wouldn't have wanted to use
gcc anyway.

--- end quote ---

Unless you have any questions, that's the last I'll say on the 
subject. I hope something in here may help track down a bug.

Regards,

-- Russell

[Numpy-discussion] _fastCopyAndTranpose

[Colin J. W. <cj...@sy...>]

numarray.LinearAlgebra2.py has

# _fastCopyAndTranpose is an optimized version of _castCopyAndTranspose.
# It assumes the input is 2D (as all the calls in here are).

However the following code is commented out.

Is this considered a satisactory replacement for _castCopyAndTranspose?

Colin W

Re: [Numpy-discussion] Trouble installing numarray on Solaris

[Todd M. <jm...@st...>]

On Sat, 2005-07-02 at 20:30 -0700, Russell E Owen wrote:
> At 6:43 AM -0400 7/2/05, Todd Miller wrote:
> >On Fri, 2005-07-01 at 15:02 -0700, Russell E. Owen wrote:
> >>  A user of my application reports that installing numarray 1.3.2 on
> >>  Solaris using Python 2.3.1 has run into trouble:
> >>
> >>    prompt> python
> >>    Python 2.3.1 (#1, Sep 30 2003, 20:29:58) [C] on sunos5
> >>    Type "help", "copyright", "credits" or "license" for more information.
> >>    >>> import numarray
> >>    Fatal Python error: Call to API function without first calling
> >>    import_libnumarray() in Src/_convmodule.c
> >>    Abort (core dumped)
> >
> >It looks vaguely like an binary API mismatch of some kind... or one of
> >you numarray modules is stale or broken.  To restate the obvious,   it's
> >likely that numarray won't import (in C!) because import_libnumarray()
> >is very defintely in _convmodule.c.  Since it's there and the API
> >pointer was not initialized,  the libnumarray import failed.
> 
> Here's what the person had to say about it. I found the solution 
> rather puzzling, but perhaps it makes sense to somebody who 
> understands more about such things:
> 
> Got it working; it was indeed that the shared object modules weren't 
> being linked correctly at build time, so when python tried to load 
> them, they were missing symbols (eg, that are defined in the C math 
> library, for example).  Probably the result of compiling numarray 
> w/Solaris's cc rather than gcc.  I manually linked the modules for 
> now, and numarry can now be imported... It even passed all its tests.

One thing I should have mentioned earlier is that numarray has been
built at STScI using Solaris's cc from the beginning.  Although our
primary development platform is now x86 Linux, we still ultimately
deploy to Solaris so I think Solaris cc per-say is probably not the
problem here.  

Here are the tools I use personally w/o problems:

basil> uname -a
SunOS anysun.stsci.edu 5.8 Generic_117350-18 sun4u sparc SUNW,Ultra-5_10
basil> cc -V
cc: Sun WorkShop 6 2000/06/19 C 5.1 Patch 109491-02

Regards,
Todd

Re: [Numpy-discussion] patch numarray.fromfuntion with optional type argument

[Todd M. <jm...@st...>]

On Tue, 2005-07-05 at 16:04 -0700, Sebastian Haase wrote:
> Hi,        (this is regarding my post from 16 June 2005)
> Todd, maybe you can confirm that this patch is a good idea and that you will 
> put it into upcoming numarray versions !?
> I would like to streamline my code - but only of course if you OK this.
> 
>  Sebastian

I added a 'type' parameter to fromfunction() and committed it to CVS
this morning.   It'll be in 1.3.3.   

Regards,
Todd

[Numpy-discussion] unexpected behavior of na.where (no short-circuiting)

[Sebastian H. <ha...@ms...>]

Hi,
I was very surprised when I got this warning:
>>> a = na.arange(4)-2
>>> na.where(a != 0,1./a, 999)
Warning: Encountered divide by zero(s)  in divide
[  -0.5   -1.   999.     1. ]

Then I realized that this is generally referred to as (not) "short 
circuiting" (e.g. in the case of the '?:'-C-operator, the middle part never 
gets evaluated at all if the first part evals to 0 )

Especially annoying was this because (for debugging) I had set this 
error-mode:
>>> na.Error.setMode(dividebyzero="error")

My questions are:
 a) Did other people encounter this problem ?
 b) What is the general feeling about this actually being a "problem" ?
 c) Could this (at all possible) be implemented differently ?

Thanks,
Sebastian Haase




 

Re: [Numpy-discussion] patch numarray.fromfuntion with optional type argument

[Sebastian H. <ha...@ms...>]

Hi,        (this is regarding my post from 16 June 2005)
Todd, maybe you can confirm that this patch is a good idea and that you will 
put it into upcoming numarray versions !?
I would like to streamline my code - but only of course if you OK this.

 Sebastian

On Thursday 16 June 2005 11:12, Sebastian Haase wrote:
> Hi,
> Please tell if this patch is a good idea. (Use: For large image data we
> always use Float32 and I thought it would be extra overhead if I first
> create everything in Float64 and then have to convert to Float32
> afterwards, myself)
>
> haase@gorilla:~: diff -p ~/myCVS/numarray/Lib/generic.py
> ~/myCVS/numarray/Lib/generic.py.~1.74.~
> *** /home/haase/myCVS/numarray/Lib/generic.py   Thu Jun 16 11:04:10 2005
> --- /home/haase/myCVS/numarray/Lib/generic.py.~1.74.~   Fri Apr 22 13:35:26
> 2005
> *************** def indices(shape, type=None):
> *** 1167,1179 ****
>           a = a.astype(type)
>       return a
>
> ! def fromfunction(function, dimensions, type=None): # from Numeric
>       """fromfunction() returns an array constructed by calling function
>       on a tuple of number grids.  The function should accept as many
>       arguments as there are dimensions which is a list of numbers
>       indicating the length of the desired output for each axis.
>       """
> !     return apply(function, tuple(indices(dimensions,type)))
>
>   def _broadcast_or_resize(a, b):
>       try:
> --- 1167,1179 ----
>           a = a.astype(type)
>       return a
>
> ! def fromfunction(function, dimensions): # from Numeric
>       """fromfunction() returns an array constructed by calling function
>       on a tuple of number grids.  The function should accept as many
>       arguments as there are dimensions which is a list of numbers
>       indicating the length of the desired output for each axis.
>       """
> !     return apply(function, tuple(indices(dimensions)))
>
>   def _broadcast_or_resize(a, b):
>       try:
>
> Thanks,
> - Sebastian Haase
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] nestedrecords.py module

[Francesc A. <fa...@ca...>]

Ops, forgot to include the test units in the tarball. Hopefully they
are there now.

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"