Menu
▾
▴
numpy-discussion — Discussion list for all users of Numerical Python
You can subscribe to this list here.
| 2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
| 2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
| 2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
| 2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
| 2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
| 2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
|
From: Sebastian H. <ha...@ms...> - 2003-08-20 23:37:49
|
Hi, I'm not sure how much time you guys wants to spend on the numarray web site, but I find that a google search for 'numarray' still points to http://stsdas.stsci.edu/numarray/ saying numarray has moved! (OK it does bring me to the right page after some seconds) Anyway - doing some search on Usenet I found this http://groups.google.com/groups?threadm=vc3bh351prjbd8%40corp.supernews.com&rnum=8&prev=/groups%3Fq%3D%2522search%2Bengines%2522%2B%2522%252Bhas%2Bmoved%2522%2B%2Bhtml%2Bhead%26hl%3Den%26lr%3D%26ie%3DISO-8859-1%26safe%3Doff From: Guy Macon (guymacon+"_http://www.guymacon.com/_"03...@sp...) Subject: Re: If I can't redirect to new url, how bad is forwarding it? Newsgroups: alt.internet.search-engines Date: 2003-05-13 19:48:05 PST he essentially says adding this to the html-page might fix it soon: <head> <meta name="robots" content="noindex, follow" /> <title>http://www.example.com</title> <meta http-equiv="refresh" content="http://www.example.com" /> </head> Just a thought ... Thanks for numarray - have a great day. Sebastian Haase |
|
From: Colin J. W. <cj...@sy...> - 2003-08-18 23:21:59
|
Thanks, I made a guess and chose the wrong one. This is a lot closer to the optimum storage. Colin W. Todd Miller wrote: >On Mon, 2003-08-18 at 15:44, Colin J. Williams wrote: > > >>This statement: >> z= _n.array([True, False, True, True], type= _nt.BooleanType) >> >> > >Use type=_nt.Bool here. BooleanType is the class of Bool. > >Todd > > > |
|
From: Todd M. <jm...@st...> - 2003-08-18 19:53:20
|
On Mon, 2003-08-18 at 15:44, Colin J. Williams wrote: > This statement: > z= _n.array([True, False, True, True], type= _nt.BooleanType) Use type=_nt.Bool here. BooleanType is the class of Bool. Todd -- Todd Miller <jm...@st...> |
|
From: Colin J. W. <cj...@sy...> - 2003-08-18 19:44:38
|
This statement:
z= _n.array([True, False, True, True], type= _nt.BooleanType)
gives an exception:
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File
"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line
284, in array
type = getTypeObject(sequence, type, typecode)
File
"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line
217, in getTypeObject
return getType(type)
File
"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line
324, in getType
raise TypeError("Not a numeric type")
TypeError: Not a numeric type
If we leave out the type parameter, the array is created with 32 bit words.
Colin W.
|
|
From: Todd M. <jm...@st...> - 2003-08-18 13:58:50
|
On Mon, 2003-08-18 at 08:31, Peter Verveer wrote: > Hi Todd, > > Yes its the same bug, next time I will check the bug tracker before sounding > the alarm... > > BTW what is the preferred way of submitting a bug? Through the sourceforge > page, or the mailing list? It actually took me a few minutes to sort this question out. The tracker is definitely *easier* for me, but that's not a good discriminator. More importantly, if you use the tracker, the bug report won't get *lost*, and thus the chances are improved that the problem will actually get fixed. For more serious stuff (like the astype() slicing bug), it's good to post to numpy-discussion-list also. Thanks again for the report, Todd -- Todd Miller <jm...@st...> |
|
From: Colin J. W. <cj...@sy...> - 2003-08-18 13:36:43
|
# ta.py to check Int8, Int16
import numarray as _n
import numarray.numerictypes as _nt
b= _n.arange(4, type= _nt.Int8)
print 'b, b._type:', b, b._type
c= b*1001 # Grief here - type not
updated
print 'c, c._type:', c, c._type
e= _n.array([1, -2, 3000, 4.6], type= _nt.Int8) # undocumented feature
fraction discarded
# and lowest eight bits
retained as a signed
# integer.
print 'e, e._type:', e, e._type
f= _n.array([1, 2, 3, 4.6], type= _nt.Int8) * 9.6
print 'f, f._type:', f, f._type
g= (f.copy()*2000).astype(_nt.Int16) # undocumented - see e
above
print 'g, g._type:', g, g._type
--------------------------------------------------------------------
PythonWin 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)]
on win32.
Portions Copyright 1994-2001 Mark Hammond (mha...@sk...) -
see 'Help/About PythonWin' for further copyright information.
>>> b, b._type: [0 1 2 3] Int8
c, c._type: [ 0 -23 -46 -69] Int8
e, e._type: [ 1 -2 -72 4] Int8
f, f._type: [ 9.6 19.2 28.8 38.4] Float64
g, g._type: [ 19200 -27136 -7937 11264] Int16
Colin W.
|
|
From: Todd M. <jm...@st...> - 2003-08-18 12:27:53
|
Hi Peter, What you're demonstrating below looks like a bug in numarray which was just recently solved: https://sourceforge.net/tracker/index.php?func=detail&aid=784866&group_id=1369&atid=450446 As you've shown, numarray's astype() fails for some slices, notably those which are offset from the base of the array and therefore have a non-zero _byteoffset. This is fixed in CVS now and will be "officially released" soon. On Mon, 2003-08-18 at 08:01, Peter Verveer wrote: > Consider conversion of a slice using the astype() method: > > >>> a = array([[1,2],[3,4]]) > >>> b = a[1, ...] > >>> print b > [3 4] > > Conversion using the astype() method goes wrong: > > >>> print b.astype(Float32) > [ 1. 2.] > > This fixes it: > > >>> print b.copy().astype(Float32) > [ 3. 4.] > > A bug? > > Cheers, Peter -- Todd Miller <jm...@st...> |
|
From: Peter V. <ve...@em...> - 2003-08-18 12:02:58
|
Consider conversion of a slice using the astype() method: >>> a = array([[1,2],[3,4]]) >>> b = a[1, ...] >>> print b [3 4] Conversion using the astype() method goes wrong: >>> print b.astype(Float32) [ 1. 2.] This fixes it: >>> print b.copy().astype(Float32) [ 3. 4.] A bug? Cheers, Peter -- Dr. Peter J. Verveer Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Pet...@em... |
|
From: o.mekkaoui <o.m...@fr...> - 2003-08-15 16:43:16
|
Thanks for your help, I have tested a little example with including numarray.h and it don't work. However, if I include "libnumarray.h" instead "numarray.h", that work fine. Thanks. Omar Mekkaoui University of Cergy-Pontoise, France Todd Miller wrote: >On Thu, 2003-08-14 at 20:54, o.mekkaoui wrote: > > >>Hi, >>I would like to use numarray. >>I use Python2.2 and I have installed numarray0.6.1. >>I have to problem: >>1) when I test numarray, I got this >> >>> import numarray.testall as testall >> >>> testall.test() >>*** Tester.merge: 'numarray.numtest' in both testers; summing outcomes. >>*** Tester.merge: 'numarray.numtest.dtp' in both testers; summing outcomes. >>*** Tester.merge: 'numarray.numtest.test' in both testers; summing outcomes. >>*** Tester.merge: 'numarray.numtest.ratioarr' in both testers; summing >>outcomes.*** Tester.merge: 'numarray.numtest.comparr' in both testers; >>summing outcomes. >>*** Tester.merge: 'numarray.numtest.dummy' in both testers; summing >>outcomes. >>numeric: (0, 1037) >>records: (0, 48) >>strings: (0, 168) >>memmap: (0, 75) >> >>which differ from that given in numarray documentation. >> >> > >The "Tester.merge" lines need to be documented. They're a result of >running the tests twice in an attempt to detect cache related bugs. > >The total test counts vary from release to release. The leading zeros >are good... they mean for instance, 0 out of 1037 tests failed. Thus, >your test results show that numarray is installed and working. > > > >>2) writting a little C API program using numrray don't work too: I >>got for example >>Traceback (most recent call last): >> File "Test_num.py", line 3, in ? >> from test_num import departure >>ImportError: ./test_num.so: undefined symbol: import_libnumarray >> >>I don't understand why the MACRO import_libnumarray() is not imported. >> >> > >Make sure you included the header file "numarray.h" or "arrayobject.h". > > > >>And if someone can send me I little C API example, I will be very grateful. >> >> > >If you have the numarray source distribution, try looking in the >Examples directory for various ways to interface with numarray in C. Of >the choices there, the best ones to start with are probably either the >"High Level" API (Examples/convolve/high_levelmodule.c) or the Numeric >compatible API (Examples/convolve/numpy_compatmodule.c). > > > >>Thanks in advance for your help. >> >> > >You're welcome. Good Luck! > >Todd > > > |
|
From: Todd M. <jm...@st...> - 2003-08-15 13:18:48
|
On Thu, 2003-08-14 at 20:54, o.mekkaoui wrote: > Hi, > I would like to use numarray. > I use Python2.2 and I have installed numarray0.6.1. > I have to problem: > 1) when I test numarray, I got this > >>> import numarray.testall as testall > >>> testall.test() > *** Tester.merge: 'numarray.numtest' in both testers; summing outcomes. > *** Tester.merge: 'numarray.numtest.dtp' in both testers; summing outcomes. > *** Tester.merge: 'numarray.numtest.test' in both testers; summing outcomes. > *** Tester.merge: 'numarray.numtest.ratioarr' in both testers; summing > outcomes.*** Tester.merge: 'numarray.numtest.comparr' in both testers; > summing outcomes. > *** Tester.merge: 'numarray.numtest.dummy' in both testers; summing > outcomes. > numeric: (0, 1037) > records: (0, 48) > strings: (0, 168) > memmap: (0, 75) > > which differ from that given in numarray documentation. The "Tester.merge" lines need to be documented. They're a result of running the tests twice in an attempt to detect cache related bugs. The total test counts vary from release to release. The leading zeros are good... they mean for instance, 0 out of 1037 tests failed. Thus, your test results show that numarray is installed and working. > > > 2) writting a little C API program using numrray don't work too: I > got for example > Traceback (most recent call last): > File "Test_num.py", line 3, in ? > from test_num import departure > ImportError: ./test_num.so: undefined symbol: import_libnumarray > > I don't understand why the MACRO import_libnumarray() is not imported. Make sure you included the header file "numarray.h" or "arrayobject.h". > > And if someone can send me I little C API example, I will be very grateful. If you have the numarray source distribution, try looking in the Examples directory for various ways to interface with numarray in C. Of the choices there, the best ones to start with are probably either the "High Level" API (Examples/convolve/high_levelmodule.c) or the Numeric compatible API (Examples/convolve/numpy_compatmodule.c). > Thanks in advance for your help. You're welcome. Good Luck! Todd -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: o.mekkaoui <o.m...@fr...> - 2003-08-15 01:29:11
|
Hi,
I would like to use numarray.
I use Python2.2 and I have installed numarray0.6.1.
I have to problem:
1) when I test numarray, I got this
>>> import numarray.testall as testall
>>> testall.test()
*** Tester.merge: 'numarray.numtest' in both testers; summing outcomes.
*** Tester.merge: 'numarray.numtest.dtp' in both testers; summing outcomes.
*** Tester.merge: 'numarray.numtest.test' in both testers; summing outcomes.
*** Tester.merge: 'numarray.numtest.ratioarr' in both testers; summing
outcomes.*** Tester.merge: 'numarray.numtest.comparr' in both testers;
summing outcomes.
*** Tester.merge: 'numarray.numtest.dummy' in both testers; summing
outcomes.
numeric: (0, 1037)
records: (0, 48)
strings: (0, 168)
memmap: (0, 75)
which differ from that given in numarray documentation.
2) writting a little C API program using numrray don't work too: I
got for example
Traceback (most recent call last):
File "Test_num.py", line 3, in ?
from test_num import departure
ImportError: ./test_num.so: undefined symbol: import_libnumarray
I don't understand why the MACRO import_libnumarray() is not imported.
And if someone can send me I little C API example, I will be very grateful.
Thanks in advance for your help.
|
|
From: Fernando P. <fp...@co...> - 2003-08-14 23:37:03
|
Hi all, I think one of the strongest points in favor of python for scientific computing is the ability to write low-level code, when necessary, which can perform on-par with hand-rolled Fortran. In the past, I've been very pleased using weave's inline() tool, which relies on blitz for manipulating Numpy arrays with an very clean and convenient syntax. This is important, because manipulating multidimensional Numeric arrays in C is rather messy, and the resulting code isn't exactly an example of readability. Blitz arrays end up looking just like regular arrays, using (i,j,k) instead of [i][j][k] for indexing. Recently, I needed to do an operation which turned out to be pretty much what Numpy's innerproduct() does. I'd forgotten about innerproduct(), so I just wrote my own using inline(). Later I saw innerproduct(), and decided to compare the results. I'm a little worried by what I found, and I'd like to hear some input from the experts on this problem. I've attached all the necessary code to run my tests, in case someone is willing to do it and take a look. In summary, I found some things which concern me (a README is included in the .tgz with more info): - the blitz code, whether via inline() or a purely hand-written extension, is ~2.5 to 3 times slower than innerproduct(). Considering that this code is specialized to a few sizes and data types, this comes as a big surprise. If the only way to get maximum performance with Numpy arrays is to write by hand to the full low-level api, I know that many people will shy away from python for a certain class of projects. I truly hope I'm missing something here. - There is a significant numerical discrepancy between the two approaches (blitz vs numpy). In an innerproduct operation over 7000 entries, the discrepancy is O(1e-10) (in l2 norm). This is more than I'm comfortable with, but perhaps I'm being naive or optimistic. I view the ability to get blitzed code which performs on par with Fortran as a very important aspect of python's suitability for large-scale project where every last bit of performance matters, but where one still wants to have the ability to work with a reasonably clean syntax. I hope I'm just misusing some tools and not faced with a fundamental limitation. By the way, I'll come to Scipy'03 with many more questions/concerns along these lines, and I think it would be great to have some discussions on these issues there with the experts. Thanks in advance. Cheers, f. |
|
From: eric j. <er...@en...> - 2003-08-14 23:21:02
|
Hey group, We have a developer position open here at Enthought, and I thought I'd see if anyone is interested. The posting is below. There is also a tech writer and HCI opening listed on the website: http://www.enthought.com/careers.html thanks! eric Position: Python Developer Type: Full-Time Enthought, Inc. is a technology startup in the Austin, TX area that is looking for a motivated Python Developer to join our development team. Responsibilities include developing and deploying scientific applications within the Fortune 500. Our size and prospects provide significant opportunity for growth and advancement for Enthought employees. Candidates should have the following qualifications: - BS or MS in Computer Science or Engineering or other scientific field - 3+ years experience in the software industry - Experience developing solutions in the following languages: - Python - C/C++ - User interface design experience - Developing and deploying custom solutions for clients - Excellent verbal and written communication skills Ideal candidates will also demonstrate the following skills: - Use of the wxPython user interface library - Experience with Geophysics applications - Experience in the fields of statistics, linear algebra and calculus Please send resumes to jo...@en.... |
|
From: Francesc A. <fa...@op...> - 2003-08-14 11:02:47
|
PyTables 0.7.1 is out!
----------------------
This is a mainly a bug-fixing release, where the next problems has
been addressed:
- Fixed several memory leaks. After that, the memory
consumption when using large object trees has dropped
sensibly. However, there remains some small leaks, but
hopefully they are not very important unless you use *huge*
object trees.
- Fixed a bug that make the __getitem__ special method in
table to fail when the stop parameter in a extended slice
was not specified. That is, table[10:] now correctly returns
table[10:table.nrows+1], and not table[10:11].
- The removeRows() method in Table did not update the NROWS
attribute in Table objects, giving place to errors after
doing further updating operations (removing or adding more
rows) in the same table. This has been fixed now.
Apart of these fixes, a new lazy reading algorithm for attributes has been
activated by default. With that, the opening of objects with large
hierarchies has been improved by 60% (you can obtain another additional 10%
if using python 2.3 instead of python 2.2). The documentation has been
updated as well, specially a more detailed instructions on the compression
(zlib) libraries installation.
Also, a stress test has been conducted in order to see if PyTables can
*really* work not only with large data tables, but also with large object
trees. On it, it has been generated and checked a file with more than 1 TB
of size and more than 100 thousand tables on it!. See
http://pytables.sourceforge.net/doc/stress-test.html for details.
Let me know of any bugs, suggestions, etc. you may have.
--
Francesc Alted
|
|
From: Francesc A. <fa...@op...> - 2003-07-31 22:54:34
|
Announcing PyTables 0.7
-----------------------
PyTables is a hierarchical database package designed to efficently
manage very large amounts of data. PyTables is built on top of the
HDF5 library and the numarray package and features an object-oriented
interface that, combined with C-code generated from Pyrex sources,
makes it a fast, yet extremely easy to use tool for interactively save
and retrieve large amounts of data.
Release 0.7 is the third public beta release. The version 0.6 was
internal and will never be released.
On this release you will find:
- new AttributeSet class
- 25% I/O speed improvement
- fully multidimensional table cells support
- new column descriptors
- row deletion in tables is finally here
- much more!
More in detail:
What's new
-----------
- A new AttributeSet class has been added. This will allow the
addition and deletion of generic attributes (any scalar type plus
any Python object supported by Pickle) as easy as this:
table.attrs.date = "2003/07/28 10:32" # Attach a string to table
group._v_attrs.tempShift = 1.2 # Attach a float to group
array.attrs.detectorList = [1,2,3,4] # Attach a list to array
del array.attrs.detectorList # Detach detectorList attr from array
- PyTables now has support for fully multidimensional table cells. This
has been made possible in part by implementation of multidimensional
cells in numarray.records.RecArray object. Thanks to numarray crew,
and especially to Jin-chung Hsu, for willingly accepting to do
that, and also for including some cache improvements in RecArray.
- New column descriptors added: IntCol, Int8Col, UInt8Col, Int16Col,
UInt16Col, Int32Col, UInt32Col, Int64Col, UInt64Col, FloatCol,
Float32Col, Float64Col and StringCol. I think they are more explicit
and easy-to-use than the now deprecated (but still supported)
Col() descriptor. All the examples and user's manual has been
accordingly updated.
- The new Table.removeRows(start, stop) function allows you to remove
rows from tables. This feature was requested a long time ago. There
are still limitations, however: you cannot delete rows in extremely
large Tables (as the remaining rows after the stop parameter
are stored in memory). Nor is the performance optimized. These issues
will hopefully be addressed in future releases.
- Added iterators to File, Group and Table (they now support the special
__iter__() method). They make the object much more user-friendly,
especially in interactive mode. See documentation for usage examples.
- Added a __getitem__() method to Table that works more or less like
read(), but with extended slices support.
- As a consequence of rewriting table iterators in C (with the help of
Pyrex, of course) the table read performance has been improved
between 20% and 30%. Data selections in PyTables are now starting to
beat powerful relational databases like SQLite, even compared to
in-core selects (!). I think there is still room for another 20% or
30% speed improvement, so stay tuned.
- A checksum is now added automatically when using LZO (not with UCL
where I'm having some difficulties implementing that
capability). The Adler32 algorithm has been chosen because of its
speed. With that, the compressing/decompressing speed has dropped 1%
or 2%, which is hardly noticeable. I think this addition will allow
the cautious user to be a bit more confident about this excellent
compressor. Code has been added to be able to read files created
without this checksum (so you can be confident that you will be able
to read your existing files compressed with LZO and UCL).
- Recursion has been removed from PyTables. Before, this made the
maximum depth tree to be less than the Python recursion limit (which
depends on implementation, but is around 900, at least in
Linux). Now, the limit has been set (somewhat arbitrarily) at
2048. Thanks to John Nielsen for implementing the new iterative
method!.
- A new rootUEP parameter to openFile() has been added. You can now
define the root from which you want to start to build the object tree.
Thanks to John Nielsen for the suggestion and a first implementation.
- A small bug fixed when dealing with non-native PyTables files that
prevented the use of the "classname" filter during a listNodes()
call. Thanks to Jeff Robbins for reporting that.
- Some (non-serious) bugs were discovered and fixed.
- Updated documentation to explain all these new bells and whistles. It
is also available on the web:
http://pytables.sourceforge.net/html-doc/usersguide-html.html
- Added more unit tests (more than 350 now!)
- PyTables 0.7 *needs* numarray 0.6 or higher and HDF-1.6.0 or higher
to compile and work. It has been tested with Python 2.2 and 2.3 and
should work fine on both versions.
What is a table?
----------------
A table is defined as a collection of records whose values are stored
in fixed-length fields. All records have the same structure and all
values in each field have the same data type. The terms
"fixed-length" and "strict data types" seems to be quite a strange
requirement for an language like Python, that supports dynamic data
types, but they serve a useful function if the goal is to save very
large quantities of data (such as is generated by many scientific
applications, for example) in an efficient manner that reduces demand
on CPU time and I/O resources.
What is HDF5?
-------------
For those people who know nothing about HDF5, it is is a general
purpose library and file format for storing scientific data made at
NCSA. HDF5 can store two primary objects: datasets and groups. A
dataset is essentially a multidimensional array of data elements, and
a group is a structure for organizing objects in an HDF5 file. Using
these two basic constructs, one can create and store almost any kind of
scientific data structure, such as images, arrays of vectors, and
structured and unstructured grids. You can also mix and match them in
HDF5 files according to your needs.
Platforms
---------
I'm using Linux as the main development platform, but PyTables should
be easy to compile/install on other UNIX machines. This package has
also passed all the tests on a UltraSparc platform with Solaris 7 and
Solaris 8. It also compiles and passes all the tests on a SGI
Origin2000 with MIPS R12000 processors and running IRIX 6.5.
Regarding Windows platforms, PyTables has been tested with Windows
2000 and Windows XP, but it should also work with other flavors.
An example?
-----------
For online code examples, have a look at
http://pytables.sourceforge.net/tut/tutorial1-1.html
and
http://pytables.sourceforge.net/tut/tutorial1-2.html
Web site
--------
Go to the PyTables web site for more details:
http://pytables.sourceforge.net/
Share your experience
---------------------
Let me know of any bugs, suggestions, gripes, kudos, etc. you may
have.
Have fun!
-- Francesc Alted
fa...@op...
|
|
From: Alex M. <al...@ya...> - 2003-07-31 16:07:14
|
I'm sending this buglet-report to the discussion lists for each of pygame,
Numeric, and Python-Dev, because it ends with a recommendation for changes
(at least to docs) in each of the two extension packages and Python 2.3.1.
I'm not a regular user of pygame [I am one of Numeric]. I helped some pygame
users find out why their program was breaking (in just one point) when
migrating Python 2.2 -> 2.3 (Numeric 23.0, pygame 1.5.6 from cvs snapshot).
The problem can be narrowed down the following little buglet.py script:
import copy, pygame.display, pygame.surfarray
screen = pygame.display.set_mode((640,480),0,24)
surf = pygame.surfarray.pixels3d(screen)
xx = copy.copy(surf[10:20,30])
With python 2.2, this runs without errors. With 2.3:
Traceback (most recent call last):
File "buglet.py", line 6, in ?
xx = copy.copy(surf[10:20,30])
File "/usr/local/lib/python2.3/copy.py", line 101, in copy
raise Error("un(shallow)copyable object of type %s" % cls)
copy.Error: un(shallow)copyable object of type <type 'array'>
Workaround for the problem: add an "import Numeric" at the top of buglet.py.
Reason for the problem: copy.copy uses different mechanisms in 2.3 vs 2.2 for
shallow copies of Numeric.array instances. In 2.2, it looked for a method
__copy__ on the instance, found it, and happily used it. In 2.3, it relies
on getting a 'reductor' function from the copy_reg.dispatch_table. Module
Numeric installs such a reductor *WHEN IMPORTED* -- but pygame never
imports it even though functions such as the above-used pixels3d do return
instances of Numeric.array. So, Numeric.py doesn't run, and therefore cannot
install its pickle_array function with the copy_reg module.
Suggested fix in pygame: I would suggest making sure Numeric has been
imported when Numeric.array instances are to be returned. That's up to the
pygame maintainers, of course; at the very least, I think this issue should
be mentioned in pygame's docs (asking the user to make sure "import Numeric"
happens in any pygame-using program that manipulates such instances).
Suggested fix in Numeric: Numeric cannot DO anything much about this issue, I
think, but it should probably *mention* (in the docs regarding interfacing to
Numeric from other C-coded extensions) that other extensions should make sure
Numeric is _imported_ (from a Python POV), if said extensions return
instances of Numeric.array, to ensure such instances have full functionality
(and in particular shallow copyability).
Suggested fix in Python: a mention in the "what's new" document, section
"porting to Python 2.3", is probably warranted -- not so much about Numeric
and/or pygame extensions per se, though these are both popular 3rd party
extensions, but rather the excellent changes in the shallow copy mechanics,
which may indeed require code changes on the part of extension authors.
Thanks,
Alex
|
|
From: Perry G. <pe...@st...> - 2003-07-29 20:24:50
|
> # produce all possible combinations of rind and cind as > # indices into an equivalent flattened for a > indices = add.outer(rind*a.shape[1],cind).flat > put(a, indices, b.flat) > Whoops! I suppose that should be: put(a.flat, indices, b.flat) |
|
From: Perry G. <pe...@st...> - 2003-07-29 20:23:46
|
> for ia, ib in zip(rind, range(len(rind))): > for ja, jb in zip(cind, range(len(cind))): > a[ia,ja] = b[ib,jb] > print a > > And the question is: is there a good way to avoid the double loop > using Numeric? > > Thanks, > John > At least one way, but it isn't particularly convenient or pretty (though the ugly details could be hidden in a function). (and I warn you, I haven't tested this, but something like this ought to work) # produce all possible combinations of rind and cind as # indices into an equivalent flattened for a indices = add.outer(rind*a.shape[1],cind).flat put(a, indices, b.flat) One can do this for numarray as well in a bit more straightforward way (depending on your view) [again, untested] t = ones(b.shape) a[multiply.outer(rind,t[0]),cind*t] = b In other words, this constructs two index arrays, one for each dimension, each with the same shape as the source array, b where the locations in that array correspond to the desired index in the source array. For numeric, you must do all the selection on a 1-d array, hence the index arithmetic. Perry |
|
From: John H. <jdh...@ac...> - 2003-07-29 19:57:12
|
>>>>> "Perry" == Perry Greenfield <pe...@st...> writes:
Perry> [Note that I reordered the index arrays to match b; I
Perry> wasn't sure what was intended]
Yep, I think that was a blooper on my part; to be precise, here is the
matlab code
a = zeros( 8,8 );
b = ones( 3,4 );
rind = [2,4,6];
cind = [1,4,5,6];
a(rind, cind) = b;
and the equivalent python
from Numeric import *
a = zeros( (8,8) )
b = ones( (3,4) )
rind = [1,3,5]
cind = [0,3,4,5]
for ia, ib in zip(rind, range(len(rind))):
for ja, jb in zip(cind, range(len(cind))):
a[ia,ja] = b[ib,jb]
print a
And the question is: is there a good way to avoid the double loop
using Numeric?
Thanks,
John
|
|
From: Perry G. <pe...@st...> - 2003-07-29 19:24:06
|
> from scipy import * > from RandomArray import * > from numarray import * > I believe the problem is in this part. You are getting some functions (like rand) which return a numeric array which some numarray functions don't recognize (we will be fixing this soon, they should accept any Python sequence that fits the basic requirements). For the moment you will need to generate a numarray array for that function to work. In the latest release you would do the imports as follows (for version 0.6 and later) from numarray.numeric import * from numarray.random_array import * a = random((5,5)) and so forth Perry |
|
From: Nils W. <wag...@vd...> - 2003-07-29 19:06:44
|
-------------------
Nils Wagner writes:
>
> In any case I am interested in a reliable
workaround for this feature.
>
> A small example would be appreciated.
> Thanks in advance.
>
Numarray (and Numeric to a certain extent) currently
is
centered around what rows or columns you want rather
than
those you don't. Writing a reasonably efficient
function
to do this in a more convenient way is not hard. Here
is
an example for numarray (not tested, nor does it have
any error checking to ensure the input array has
sufficient
dimensions or that the index is within the array
bounds):
def remove(arr, index, dim=0):
"""Remove subarray at index and dimension
specified"""
# make selected dimension the first one
swparr = swapaxes(arr, 0, dim)
indices = range(swparr.shape[0])
del indices[index]
newarray = swparr[indices]
# reorder axes as they originally appeared.
return swapaxes(newarray, 0, dim)
Then you can say
x = remove(x, 5)
to remove the 6th row or
x = remove(x, 4, dim=1)
to remove the 5th column.
(depending on what you call rows or columns; in the
examples
I'm taking an image-oriented view rather than a
matrix-oriented
view)
Perry
Perry,
Thank you for your reply. Unfortunately, I am not
familiar with numarray.
This is the output of my test program.
from scipy import *
from RandomArray import *
from numarray import *
def remove(arr, index, dim=0):
"""Remove subarray at index and dimension specified"""
# make selected dimension the first one
swparr = swapaxes(arr, 0, dim)
indices = range(swparr.shape[0])
del indices[index]
newarray = swparr[indices]
# reorder axes as they originally appeared.
return swapaxes(newarray, 0, dim)
#Then you can say
a = rand(5,5)
print a
a = remove(a, 2)
print a
#to remove the 3th row or
#to remove the 3th column
a = remove(a, 2, dim=1)
print a
[[ 0.87329948 0.81829292 0.72929943 0.13012239 0.49291104]
[ 0.58665377 0.78030288 0.85906255 0.11409029 0.77719343]
[ 0.01325159 0.03389675 0.25417367 0.51117259 0.61979091]
[ 0.32406124 0.24500449 0.06147733 0.04323368 0.64497042]
[ 0.10133368 0.25028452 0.33645368 0.34395722 0.34052679]]
Traceback (most recent call last):
File "nils.py", line 19, in ?
a = remove(a, 2)
File "nils.py", line 8, in remove
swparr = swapaxes(arr, 0, dim)
File "/usr/lib/python2.2/site-packages/numarray/generic.py", line
855, in swapaxes
v = _nc.inputarray(array).view()
File "/usr/lib/python2.2/site-packages/numarray/numarraycore.py",
line 310, in inputarray
return array(seq, type=type, typecode=typecode, copy=0)
File "/usr/lib/python2.2/site-packages/numarray/numarraycore.py",
line 303, in array
raise ValueError("Unknown input type")
ValueError: Unknown input type
Any idea ?
Nils
_______________________________________________
SciPy-user mailing list
Sci...@sc...
http://www.scipy.net/mailman/listinfo/scipy-user
|
|
From: Todd M. <jm...@st...> - 2003-07-29 18:38:49
|
I discovered today (some bug reports have hinted at this but didn't spell it out completely) that I forgot Mac OS 10 testing for numarray-0.6. As it turns out, the numarray examples and add-ons don't compile correctly due to a small untested change. I corrected the problem and uploaded a new tarball to Source Forge: http://osdn.dl.sourceforge.net/sourceforge/numpy/numarray-0.6.1.tar.gz Non-Mac users of numarray-0.6 should just ignore numarray-0.6.1. This is a source update only. Todd -- Todd Miller jm...@st... STSCI / ESS / SSB |
|
From: Perry G. <pe...@st...> - 2003-07-29 18:24:01
|
Nils Wagner writes:
>
> In any case I am interested in a reliable workaround for this feature.
>
> A small example would be appreciated.
> Thanks in advance.
>
Numarray (and Numeric to a certain extent) currently is
centered around what rows or columns you want rather than
those you don't. Writing a reasonably efficient function
to do this in a more convenient way is not hard. Here is
an example for numarray (not tested, nor does it have
any error checking to ensure the input array has sufficient
dimensions or that the index is within the array bounds):
def remove(arr, index, dim=0):
"""Remove subarray at index and dimension specified"""
# make selected dimension the first one
swparr = swapaxes(arr, 0, dim)
indices = range(swparr.shape[0])
del indices[index]
newarray = swparr[indices]
# reorder axes as they originally appeared.
return swapaxes(newarray, 0, dim)
Then you can say
x = remove(x, 5)
to remove the 6th row or
x = remove(x, 4, dim=1)
to remove the 5th column.
(depending on what you call rows or columns; in the examples
I'm taking an image-oriented view rather than a matrix-oriented
view)
Perry
|
|
From: Perry G. <pe...@st...> - 2003-07-29 17:59:18
|
> If I have a large matrix, say
>
> a = zeros( (20,20) )
>
> and a small matrix
>
> b = ones( 3,4 )
>
> and a list of row indices and a list of col indices
>
> rind = [2,4,9,15]
> cind = [1,4,12]
>
> what is the best way to assign the submatrix to a[rind,cind]. I
> understand this is possible in numarray, but is there a good way to do
> it in Numeric?
>
> In matlab, you could do a(rind, cind)=b; Is there some reshape, put
> magic I can do to make this efficient in Numeric?
>
> Thanks,
> John Hunter
Note that numarray doesn't do the same thing as matlab in this
regard. In particular, the expression
a[rind, cind] = b
would fail since it is expecting an array on the right hand side
that is the same shape as the index arrays. The assignment here
is taken to mean:
a[rind[0], cind[0]] = b[0]
a[rind[1], cind[1]] = b[1]
etc.
not:
a[rind[0], cind[0]] = b[0,0]
etc.
To do what you want doesn't currently have any atomic operation
in numarray. Is this widely used?
At the moment I think you would be forced to iterate over
one dimension. E.g.,
for i in range(len(rind)):
a[cind, rind[i]] = b[:, i]
[Note that I reordered the index arrays to match b; I wasn't sure
what was intended]
There may be some clever way to avoid the explicit looping. Nothing
prevents the addition of such a feature to numarray (other than
work!), but it would likely have to use a function. I think
the current defintion of how multiple index puts and takes work with
brackets is more useful than the interpretation you are using and isn't
likely to change.
Perry
|
|
From: Perry G. <pe...@st...> - 2003-07-29 17:48:30
|
> -----Original Message----- > From: num...@li... > [mailto:num...@li...]On Behalf Of John > Hunter > Sent: Tuesday, July 29, 2003 12:30 PM > To: Numerical Python Discussion > Subject: [Numpy-discussion] assigning to submatrix > > > > If I have a large matrix, say > > a = zeros( (20,20) ) > > and a small matrix > > b = ones( 3,4 ) > > and a list of row indices and a list of col indices > > rind = [2,4,9,15] > cind = [1,4,12] > > what is the best way to assign the submatrix to a[rind,cind]. I > understand this is possible in numarray, but is there a good way to do > it in Numeric? > > In matlab, you could do a(rind, cind)=b; Is there some reshape, put > magic I can do to make this efficient in Numeric? > > Thanks, > John Hunter > > > ------------------------------------------------------- > This SF.Net email sponsored by: Free pre-built ASP.NET sites including > Data Reports, E-commerce, Portals, and Forums are available now. > Download today and enter to win an XBOX or Visual Studio .NET. > http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072 > 303_01/01 > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > |
258 messages has been excluded from this view by a project administrator.
