numpy-discussion

[Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?

[Tim P. <ti...@co...>]

[Huaiyu Zhu]
> There appears to be a serious bug in Python 2.2 that severely limits its
> usefulness for numerical computation:
>
> # Python 1.5.2 - 2.1
>
> >>> 1e200**2
> inf

A platform-dependent accident, there.

> >>> 1e-200**2
> 0.0
>
> # Python 2.2
>
> >>> 1e-200**2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> OverflowError: (34, 'Numerical result out of range')

That one is surprising and definitely not intended:  it suggests your
platform libm is setting errno to ERANGE for pow(1e-200, 2.0), or that your
platform C headers define INFINITY but incorrectly, or that your platform C
headers define HUGE_VAL but incorrectly, or that your platform C compiler
generates bad code, or optimizes incorrectly, for negating and/or comparing
against its definition of HUGE_VAL or INFINITY.  Python intends silent
underflow to 0 in this case, and I haven't heard of underflows raising
OverflowError before.  Please file a bug report with full details about
which operating system, Python version, compiler and C libraries you're
using (then it's going to take a wizard with access to all that stuff to
trace into it and determine the true cause).

> >>> 1e200**2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> OverflowError: (34, 'Numerical result out of range')

That one is intended; see

http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=496104

for discussion.

> This produces the following serious effects: after hours of numerical
> computation, just as the error is converging to zero, the whole thing
> suddenly unravels.

It depends on how you write your code, of course.

> Note that try/except is completely useless for this purpose.

Ditto.  If your platform C lets you get away with it, you may still be able
to get an infinity out of 1e200 * 1e200.

> I hope this is unintended behavior

Half intended, half both unintended and never before reported.

> and that there is an easy fix.

Sorry, "no" to either.

[Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?

[Tim P. <ti...@co...>]

[jo...@bo...]
> Ok, looking at SourceForge and google this seems to be fixed in cvs
> HEAD.  Would it be possible to put the same patch into the cvs
> python-2.2 branch, please? [1]
>
> Greetings,
> Jochen
>
> Footnotes:
> [1]  If it is in there, it doesn't work for me with current python cvs
>      branch release22-maint.  I still have to manually add -lieee.
>      (RedHat-7.0 with current updates.)

I don't know what "current" meant to you at the time you wrote this.
Michael Hudson did backport the patch into 2.2.1c1, which was released
yesterday.  So please try 2.2.1c1, and if you still have a problem, file a
bug report about it on SourceForge.  2.2.1 final is expected in about a
week.

Re: [Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?

[Jochen <jo...@un...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 19 Mar 2002 20:20:36 -0500 Tim Peters wrote:

Thanks for the quick answer.  The problem is resolved.

Tim> [jo...@bo...]

>> [1]  If it is in there, it doesn't work for me with current python cvs
>> branch release22-maint.  I still have to manually add -lieee.
>> (RedHat-7.0 with current updates.)

Tim> I don't know what "current" meant to you at the time you wrote this.

About 20:00 (8:00pm) EST today, March 19.

Tim> Tim> Michael Hudson did backport the patch into 2.2.1c1, which
Tim> Tim> was released yesterday.  So please try 2.2.1c1, and if you
Tim> Tim> still have a problem, file a bug report about it on
Tim> Tim> SourceForge.  2.2.1 final is expected in about a week.

Well, changing cvs from 
  release22-maint
to
  r221c1
helps.  That is, everything seems to work fine with the cvs sources
tagged r221c1.

Then, is it really necessary to mess up the cvs tags so much? Why
isn't it possible to have a single python-2.2 branch which one could
follow to get all the stuff that's incorporated into that version? [1]

There are huge differences between release22-maint and r221c1, it
seems from the number of patches applied when going from one to the
other.  But then some files are in the same (non-main) branch.  ???

Thanks for all your work, and thank you for the quick help again.

Greetings,
Jochen
- -- 
University of North Carolina                       phone: +1-919-962-4403
Department of Chemistry                            phone: +1-919-962-1579
Venable Hall CB#3290 (Kenan C148)                    fax: +1-919-843-6041
Chapel Hill, NC 27599, USA                            GnuPG key: 44BCCD8E
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6-cygwin-fcn-1 (Cygwin)
Comment: Processed by Mailcrypt and GnuPG <http://www.gnupg.org/>

iEYEARECAAYFAjyX6yYACgkQiJ/aUUS8zY76QwCdGZsJd1b+0qJ19LJ5TlvwI5fP
kbcAniNe/5eiPnEUfGbddLCpyYD1+gmr
=fLmk
-----END PGP SIGNATURE-----

RE: [Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?

[Tim P. <ti...@co...>]

[jo...@bo...]
> Thanks for the quick answer.  The problem is resolved.

Cool!  Glad to hear it.

> Well, changing cvs from
>   release22-maint
> to
>   r221c1
> helps.  That is, everything seems to work fine with the cvs sources
> tagged r221c1.

That shouldn't have made any difference -- r221c1 is merely a tag on the
release22-maint branch.  Now I can spend a lot of time trying to guess why
your checkout is screwed up (probably stale sticky flags, if it is), or you
can try blowing away your checkout and starting over.  I know which one gets
my vote <wink>.  CVS branches and tags are a nightmare:  when in any doubt,
kill the beast and start over.

> Then, is it really necessary to mess up the cvs tags so much? Why
> isn't it possible to have a single python-2.2 branch which one could
> follow to get all the stuff that's incorporated into that version?

That's what the release22-maint branch is supposed to be (and, AFIAK, is).

> There are huge differences between release22-maint and r221c1,

What makes you think so?  I just did

    cvs diff -r release22-maint -r r221c1

and it turned up expected differences in the handful of files that indeed
*have* changed since r221c1 was tagged, mostly in the docs and under the Mac
subdirectory:

Index: Doc/lib/libcopyreg.tex
Index: Doc/lib/libthreading.tex
Index: Lib/urllib.py
Index: Mac/_checkversion.py
Index: Mac/Build/PythonCore.mcp
Index: Mac/Distributions/(vise)/Python 2.2.vct
Index: Mac/Include/macbuildno.h
Index: Mac/Modules/macfsmodule.c
Index: Mac/Modules/macmodule.c
Index: Misc/NEWS
Index: PCbuild/BUILDno.txt

> ...
> Thanks for all your work, and thank you for the quick help again.

And thanks for checking that your problem is fixed in 221c1.  Had anyone
tried this stuff in 22a1 or 22a2 or 22a3 or 22a4 or 22b1 or 22b2 or 22c1
(yes, we actually cut 7 full prerelease distributions for 2.2!), it would
have worked in 2.2 out of the box.  Keep that in mind when 2.3a1 comes out
<wink>.

[Numpy-discussion] Different behaviour logical_and/and

[Aureli S. F. <Aur...@ip...>]

Hi all,

In the version of Numeric with MacPython2.2 the functions
"Numeric.logical_and" and "and" behave different, although up to the
on-line documentation they should behave the same:

>>> Numeric.logical_and(a,b)
array([0, 1, 0, 0, 0, 0, 1, 0, 0, 0])
>>> a and b
array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])

for arrays:

>>> a
array([0, 1, 0, 0, 1, 0, 1, 0, 0, 1])
>>> b
array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])

or am I misunderstood something...?

Regards,
Aureli

#################################
Aureli Soria Frisch
Fraunhofer IPK
Dept. Pattern Recognition

post: Pascalstr. 8-9, 10587 Berlin, Germany
e-mail: au...@ip...
fon: +49 30 39006-150
fax: +49 30 3917517
web: http://vision.fhg.de/~aureli/web-aureli_en.html
#################################

Re: [Numpy-discussion] Different behaviour logical_and/and

[Jochen <jo...@jo...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 20 Mar 2002 19:43:03 +0100 Aureli Soria Frisch wrote:

>>>> a
Aureli> array([0, 1, 0, 0, 1, 0, 1, 0, 0, 1])
>>>> b
Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])


>>>> Numeric.logical_and(a,b)
Aureli> array([0, 1, 0, 0, 0, 0, 1, 0, 0, 0])

This look's correct...

>>>> a and b
Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])

... and this suspiciously like b.


Greetings,
Jochen
- -- 
Einigkeit und Recht und Freiheit                http://www.Jochen-Kuepper.de
    Liberté, Égalité, Fraternité                GnuPG key: 44BCCD8E
        Sex, drugs and rock-n-roll
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: Processed by Mailcrypt and GnuPG <http://www.gnupg.org/>

iD8DBQE8mU3jiJ/aUUS8zY4RAnxdAKCfrCyep4b7TKoF+c631cJiX53GmACgtmGo
jcS9gt6j1eJJj937JqoUm6M=
=TyKW
-----END PGP SIGNATURE-----

Re: [Numpy-discussion] Different behaviour logical_and/and

[Pearu P. <pe...@ce...>]

On 20 Mar 2002, Jochen K=FCpper wrote:

> >>>> a
> Aureli> array([0, 1, 0, 0, 1, 0, 1, 0, 0, 1])
> >>>> b
> Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])
>=20
>=20
> >>>> Numeric.logical_and(a,b)
> Aureli> array([0, 1, 0, 0, 0, 0, 1, 0, 0, 0])
>=20
> This look's correct...
>=20
> >>>> a and b
> Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])
>=20
> ... and this suspiciously like b.

=2E. and also correct. It is default behaviour of Python `and'
operation. From Python Language Reference:

  The expression `x and y' first evaluates `x'; if `x' is false, its value =
=20
  is returned; otherwise, `y' is evaluated and the resulting value is
  returned.=20

So, `and' operation is "object"-operation (unless redefined to something
else) while logical_and is "elementwise"-operation.

Pearu

Re: [Numpy-discussion] Different behaviour logical_and/and

[Huaiyu Z. <hua...@ya...>]

On Thu, 21 Mar 2002, Pearu Peterson wrote:
> On 20 Mar 2002, Jochen K=FCpper wrote:
> > >>>> a
> > Aureli> array([0, 1, 0, 0, 1, 0, 1, 0, 0, 1])
> > >>>> b
> > Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])
> >=20
> > >>>> Numeric.logical_and(a,b)
> > Aureli> array([0, 1, 0, 0, 0, 0, 1, 0, 0, 0])
> >=20
> > This look's correct...
> >=20
> > >>>> a and b
> > Aureli> array([1, 1, 0, 1, 0, 0, 1, 1, 1, 0])
> >=20
> > ... and this suspiciously like b.
>=20
> . and also correct.  It is default behaviour of Python `and'
> operation. From Python Language Reference:
>=20
>   The expression `x and y' first evaluates `x'; if `x' is false, its va=
lue =20
>   is returned; otherwise, `y' is evaluated and the resulting value is
>   returned.=20
>=20
> So, `and' operation is "object"-operation (unless redefined to somethin=
g
> else) while logical_and is "elementwise"-operation.

There is a section in PEP-225 for elementwise/objectwise operators to=20
extend the meaning of ~ to an "elementizer", so that=20

[1, 0, 1, 0] and [0, 1, 1, 0]      =3D>  [0, 1, 1, 0]
[1, 0, 1, 0] ~and [0, 1, 1, 0]     =3D>  [0, 0, 1, 0]

There are several other places, entirely unrelated to numerical=20
computation, that elementization of an operator makes sense.

Huaiyu

RE: [Numpy-discussion] RE: Python 2.2 seriously crippled for numerical computation?

[Paul F D. <pa...@pf...>]

I also see the underflow problem on my Linux box 2.4.2-2. This is
certainly untenable. However, I am able to catch OverflowError in both
cases. I had a user complain about this just yesterday, so I think it is
a new behavior in Python 2.2 which I was just rolling out. A small
Fortran test problem did not exhibit the underflow bug, and caught the
overflow bug at COMPILE TIME (!).

There are two states for the IEEE underflow: one in which the hardware
sets it to zero, and the other in which the hardware signals the OS and
you can tell the OS to set it to zero. There is no standard for the
interface to this facility that I am aware of. (Usually I have had to
figure out how to make sure the underflow was handled in hardware
because the sheer cost of letting it turn into a system call was
prohibitive.) I speculate that on machines where the OS call is the
default that Python 2.2 is catching the signal when it should let it go
by.

I have not looked at this lately so something may have changed. 

You can use the kinds package that comes with Numeric to test 
for maximum and minimum exponents. kinds.default_float_kind.MAX_10_EXP
(equal to 308 on my Linux box, for example) tells you how big an
exponent a floating point number can have. MIN_10_EXP (-307 for me) is
also there.

Work around on your convergence test: instead of testing x**2 you might
test log10(x) vs. a constant or some expression involving
kinds.default_float_kind.MIN_10_EXP.

-----Original Message-----
From: num...@li...
[mailto:num...@li...] On Behalf Of Tim
Peters
Sent: Friday, March 01, 2002 9:43 PM
To: Huaiyu Zhu; num...@li...
Cc: pyt...@py...
Subject: [Numpy-discussion] RE: Python 2.2 seriously crippled for
numerical computation?


[Huaiyu Zhu]
> There appears to be a serious bug in Python 2.2 that severely limits 
> its usefulness for numerical computation:
>
> # Python 1.5.2 - 2.1
>
> >>> 1e200**2
> inf

A platform-dependent accident, there.

> >>> 1e-200**2
> 0.0
>
> # Python 2.2
>
> >>> 1e-200**2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> OverflowError: (34, 'Numerical result out of range')

That one is surprising and definitely not intended:  it suggests your
platform libm is setting errno to ERANGE for pow(1e-200, 2.0), or that
your platform C headers define INFINITY but incorrectly, or that your
platform C headers define HUGE_VAL but incorrectly, or that your
platform C compiler generates bad code, or optimizes incorrectly, for
negating and/or comparing against its definition of HUGE_VAL or
INFINITY.  Python intends silent underflow to 0 in this case, and I
haven't heard of underflows raising OverflowError before.  Please file a
bug report with full details about which operating system, Python
version, compiler and C libraries you're using (then it's going to take
a wizard with access to all that stuff to trace into it and determine
the true cause).

> >>> 1e200**2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> OverflowError: (34, 'Numerical result out of range')

That one is intended; see

http://sf.net/tracker/?group_id=5470&atid=105470&func=detail&aid=496104

for discussion.

> This produces the following serious effects: after hours of numerical 
> computation, just as the error is converging to zero, the whole thing 
> suddenly unravels.

It depends on how you write your code, of course.

> Note that try/except is completely useless for this purpose.

Ditto.  If your platform C lets you get away with it, you may still be
able to get an infinity out of 1e200 * 1e200.

> I hope this is unintended behavior

Half intended, half both unintended and never before reported.

> and that there is an easy fix.

Sorry, "no" to either.


_______________________________________________
Numpy-discussion mailing list Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion