You can subscribe to this list here.
2000 |
Jan
(8) |
Feb
(49) |
Mar
(48) |
Apr
(28) |
May
(37) |
Jun
(28) |
Jul
(16) |
Aug
(16) |
Sep
(44) |
Oct
(61) |
Nov
(31) |
Dec
(24) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2001 |
Jan
(56) |
Feb
(54) |
Mar
(41) |
Apr
(71) |
May
(48) |
Jun
(32) |
Jul
(53) |
Aug
(91) |
Sep
(56) |
Oct
(33) |
Nov
(81) |
Dec
(54) |
2002 |
Jan
(72) |
Feb
(37) |
Mar
(126) |
Apr
(62) |
May
(34) |
Jun
(124) |
Jul
(36) |
Aug
(34) |
Sep
(60) |
Oct
(37) |
Nov
(23) |
Dec
(104) |
2003 |
Jan
(110) |
Feb
(73) |
Mar
(42) |
Apr
(8) |
May
(76) |
Jun
(14) |
Jul
(52) |
Aug
(26) |
Sep
(108) |
Oct
(82) |
Nov
(89) |
Dec
(94) |
2004 |
Jan
(117) |
Feb
(86) |
Mar
(75) |
Apr
(55) |
May
(75) |
Jun
(160) |
Jul
(152) |
Aug
(86) |
Sep
(75) |
Oct
(134) |
Nov
(62) |
Dec
(60) |
2005 |
Jan
(187) |
Feb
(318) |
Mar
(296) |
Apr
(205) |
May
(84) |
Jun
(63) |
Jul
(122) |
Aug
(59) |
Sep
(66) |
Oct
(148) |
Nov
(120) |
Dec
(70) |
2006 |
Jan
(460) |
Feb
(683) |
Mar
(589) |
Apr
(559) |
May
(445) |
Jun
(712) |
Jul
(815) |
Aug
(663) |
Sep
(559) |
Oct
(930) |
Nov
(373) |
Dec
|
From: Todd M. <jm...@st...> - 2003-05-14 20:26:36
|
On Wed, 2003-05-14 at 16:21, Andrew P. Lentvorski, Jr. wrote: > What is the official way to zero out an array in numarray/Numeric? > I'm don't think there is one in numarray; which is to say, I'd do what you did below. > While I can create a new array of all zeros and then assign it to the old > variable, this is extremely wasteful of memory. > > Currently, I am just using: > > >>> import numarray > >>> a = numarray.arange(6) > >>> a > array([0, 1, 2, 3, 4, 5]) > >>> a[:] = 0.0 > >>> a > array([0, 0, 0, 0, 0, 0]) > > I looked through the manual for a function or array mathod which would > accomplish the same thing, but I didn't find an obvious one. Did I miss > something obvious? I don't think so. Todd |
From: Andrew P. L. Jr. <bs...@al...> - 2003-05-14 20:18:16
|
What is the official way to zero out an array in numarray/Numeric? While I can create a new array of all zeros and then assign it to the old variable, this is extremely wasteful of memory. Currently, I am just using: >>> import numarray >>> a = numarray.arange(6) >>> a array([0, 1, 2, 3, 4, 5]) >>> a[:] = 0.0 >>> a array([0, 0, 0, 0, 0, 0]) I looked through the manual for a function or array mathod which would accomplish the same thing, but I didn't find an obvious one. Did I miss something obvious? Thanks, -a |
From: Andrew P. L. Jr. <bs...@al...> - 2003-05-14 20:09:58
|
On Wed, 14 May 2003, Todd Miller wrote: > Point taken. I guess that changes the addon mappings to: > > LinearAlgrebra2 --> numarray.linear_algebra > FFT2 --> numarray.fft > RandomArray2 --> numarray.random_array > Convolve --> numarray.convolve > Convolve.Image --> numarray.image > MA2 --> numarray.ma I prefer this naming scheme as it helps visually delineate between modules and classes. ie: numarray.linear_algebra.LUMatrix is likely to be a class numarray.linear_algebra.lumatrix is likely to be a module It seems that most people use StudlyCaps for class names in Python. While there aren't a lot of times that this matters, I have bumped into this more often than I expect. But I may just be doing odd things with modules. -a |
From: Todd M. <jm...@st...> - 2003-05-14 11:54:39
|
Francesc Alted wrote: >A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > >>I like the idea, because it would mean that I can release my PyQwt >>plot package so that it can be build for Numeric and numarray >>on the same system. It reduces the risk of calling 'PyQwt for numarray' >>functions with 'Numeric arrays' as arguments and vice-versa (for me it >>seems to work, but it scares me to death). >> >> > >I think this is not necessary as you can always distinguish if the user is >passing a numarray or Numeric object, like in: > >In [3]: a=Numeric.array([1,2]) > >In [4]: b=numarray.array([1,2]) > >In [5]: type(a) >Out[5]: <type 'array'> > >In [6]: type(b) >Out[6]: <class 'numarray.NumArray'> > > >It's just a check before accessing the object. > >Cheers, > > > I think installing in numarray.addons (or not) is a package-by-package decision. Some packages might prefer to be independent. That's fine. Others might prefer (or require) dedicated versions, particularly those packages which use compiled extensions. For those instances where dedicated versions are necessary, numarray.addons is a recommendation to keep clear seperation between core numarray (maintained and distributed by one group) and software which has been layered on it (maintained and distributed by other groups). Todd |
From: Kasper S. <Kas...@ir...> - 2003-05-14 11:40:27
|
> I have no preference myself (obviously!). Lower case and underscores > may be more in line with our group's style. The Python Style Guide [1] doesn't give a definitive answer, but there seems to be a preference for the lowercase in this situation. bye, Kasper [1] http://www.python.org/doc/essays/styleguide.html P.S. A simple reply on this list just replies to the owner. I would prefer that it just gives me Num...@li... instead. -- Hi! I'm a .signature virus! copy me into your .signature file to help me spread! Thanks to DRM, you know that something has been built in environment of unspecified degree of security, from source you cannot check, written by programmers you don't know, released after passing QA of unknown quality and which is released under a license which disclaims any responsibility... |
From: Todd M. <jm...@st...> - 2003-05-14 11:30:06
|
Konrad Hinsen wrote: >On Tuesday 13 May 2003 23:50, Todd Miller wrote: > > > >>Here are the planned renamings for the numarray modules as we transition >>to a package: >> >>numarray --> numarray.numeric >>recarray --> numarray.records >>chararray --> numarray.strings >>ndarray --> numarray.generic >>* --> numarray.* >> >>Note that class and function names will remain unchanged, so >>recarray.RecArray will become numarray.records.RecArray. >> >>Here are the planned renamings for the current and planned add-on packages >>consolidated into a single add-ons distribution: >> >>LinearAlgrebra2 --> numarray.LinearAlgebra >>FFT2 --> numarray.FFT >>RandomArray2 --> numarray.RandomArray >>Convolve --> numarray.Convolve >>Convolve.Image --> numarray.Image >>MA2 --> numarray.MA >> >> > >Please use a uniform capitalization scheme, no matter which one. Either >numarray.image or Numarray.Image (I prefer the latter, but both are better >than the mixed one). > >Konrad. > > Point taken. I guess that changes the addon mappings to: LinearAlgrebra2 --> numarray.linear_algebra FFT2 --> numarray.fft RandomArray2 --> numarray.random_array Convolve --> numarray.convolve Convolve.Image --> numarray.image MA2 --> numarray.ma or changes everything to: numarray --> Numarray.Numeric recarray --> Numarray.Records chararray --> Numarray.Strings ndarray --> Numarray.Generic * --> Numarray.* LinearAlgrebra2 --> Numarray.LinearAlgebra FFT2 --> Numarray.FFT RandomArray2 --> Numarray.RandomArray Convolve --> Numarray.Convolve Convolve.Image --> Numarray.Image MA2 --> Numarray.MA I have no preference myself (obviously!). Lower case and underscores may be more in line with our group's style. Todd |
From: Todd M. <jm...@st...> - 2003-05-14 11:07:40
|
David M. Cooke wrote: >On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > >>As has been mentioned before, we're planning to repackage numarray as >>a package rather than as a collection of modules. We're soliciting >>comments now because we only want to do this once. >> >> >... > > >>Future 3rd party packages should locate themselves under the >>numarray.addons sub-package: >> >>* --> numarray.addons.* >> >> > >What's the rationale for having a separate place for third-party >packages? Shouldn't they just install themselves as their own packages >or modules? (i.e., in site-packages/) > > > I think there are two goals which are served by numarray.addons: 1. It becomes clear what is in the numarray core, and what isn't. This is a maintenance issue. 2. It becomes clear that a particular extension was built explicitly for numarray. This might be valuable in a transition phase from Numeric to numarray. Todd |
From: Kasper S. <Kas...@ir...> - 2003-05-14 10:02:29
|
> > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) Maybe it's nice to have both. Leave it to the third-party to decide whether it's a good idea to install in numarray.addons or in site-packages/. For stuff that's not too dependent on numpy (PyGame for instance) it would be better to install in site-packages/. For other things (like PyQwt) it might be better to locate them in numarray.addons. bye, Kasper |
From: Francesc A. <fa...@op...> - 2003-05-14 09:45:16
|
A Dimecres 14 Maig 2003 09:24, Gerard Vermeulen va escriure: > > I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' > functions with 'Numeric arrays' as arguments and vice-versa (for me it > seems to work, but it scares me to death). I think this is not necessary as you can always distinguish if the user i= s passing a numarray or Numeric object, like in: In [3]: a=3DNumeric.array([1,2]) In [4]: b=3Dnumarray.array([1,2]) In [5]: type(a) Out[5]: <type 'array'> In [6]: type(b) Out[6]: <class 'numarray.NumArray'> It's just a check before accessing the object. Cheers, --=20 Francesc Alted |
From: Konrad H. <hi...@cn...> - 2003-05-14 09:28:05
|
On Tuesday 13 May 2003 23:50, Todd Miller wrote: > Here are the planned renamings for the numarray modules as we transitio= n > to a package: > > numarray --> numarray.numeric > recarray --> numarray.records > chararray --> numarray.strings > ndarray --> numarray.generic > * --> numarray.* > > Note that class and function names will remain unchanged, so > recarray.RecArray will become numarray.records.RecArray. > > Here are the planned renamings for the current and planned add-on packa= ges > consolidated into a single add-ons distribution: > > LinearAlgrebra2 --> numarray.LinearAlgebra > FFT2 --> numarray.FFT > RandomArray2 --> numarray.RandomArray > Convolve --> numarray.Convolve > Convolve.Image --> numarray.Image > MA2 --> numarray.MA Please use a uniform capitalization scheme, no matter which one. Either=20 numarray.image or Numarray.Image (I prefer the latter, but both are bette= r=20 than the mixed one). Konrad. --=20 -------------------------------------------------------------------------= ------ Konrad Hinsen | E-Mail: hi...@cn... Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------= ------ |
From: Konrad H. <hi...@cn...> - 2003-05-14 09:26:16
|
> I like the idea, because it would mean that I can release my PyQwt > plot package so that it can be build for Numeric and numarray > on the same system. It reduces the risk of calling 'PyQwt for numarray' I'd like to check that this doesn't cause any technical problems with pac= kage=20 managers, distutils, etc. Otherwise it will prevent people from switching= to=20 numarray. Konrad. --=20 -------------------------------------------------------------------------= ------ Konrad Hinsen | E-Mail: hi...@cn... Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------= ------ |
From: Gerard V. <gve...@gr...> - 2003-05-14 07:24:33
|
On Tue, 13 May 2003 18:13:31 -0400 "David M. Cooke" <co...@ph...> wrote: > On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > > As has been mentioned before, we're planning to repackage numarray as > > a package rather than as a collection of modules. We're soliciting > > comments now because we only want to do this once. > ... > > Future 3rd party packages should locate themselves under the > > numarray.addons sub-package: > > > > * --> numarray.addons.* > > What's the rationale for having a separate place for third-party > packages? Shouldn't they just install themselves as their own packages > or modules? (i.e., in site-packages/) > I like the idea, because it would mean that I can release my PyQwt plot package so that it can be build for Numeric and numarray on the same system. It reduces the risk of calling 'PyQwt for numarray' functions with 'Numeric arrays' as arguments and vice-versa (for me it seems to work, but it scares me to death). Gerard |
From: Sebastian H. <ha...@ms...> - 2003-05-14 02:13:13
|
Hi - I hope it's OK to just resent my posting from Feb 22. I didn't get any response at the time. Also - looking into the code again - I am wondering what the best way to handle the reference counting would be ? Here come my original posting: > Short follow up: > 1) Is it planned to support this more directly? > 2) How much does it cost to create a buffer object if it > uses my already allocated memory ? > 3) Can I change the pointer so that it points to a > different memory space WITHOUT having to recreate any > python objects? Or would that "confuse" the buffer or > numarray? (We are hoping to aquire 30 images per second > - the images should get written into a circular buffer so > that the data can be written to disk in larger chunks - > but the python array should always refer to the current > image ) > > Thanks for all the nice toys (tools) ;-) > Sebastian Haase > > > > On Fri, 17 Jan 2003 18:16:01 -0500 > Todd Miller <jm...@st...> wrote: > >Sebastian Haase wrote: > > > >>Hi, > >>What is the C API to make an array that got allocated, > >>let's say, by a = new short[512*512], > >>accessible to python as numarray. > >> > >What you want to do is not currently supported well in C. > > The way to do what you want is: > > > >1. Create a buffer object from your C++ array. The > >buffer object can be built such that it refers to the > >original copy of the data. > > > >2. Call back into Python (numarray.NumArray) with your > >buffer object as the buffer parameter. > > > >You can scavenge the code in NA_newAll (Src/newarray.ch) > >for most of the callback. > > > >>I tried NA_New - but that seems to make a copy. > >>I would need it to use the original memory space > >>so that I can "observe" the array from Python WHILE > >>the underlying C array changes (it's actually a camera > >>image) > >> > >That sounds cool! > > > >> > >>Thanks, > >>Sebastian Haase |
From: David M. C. <co...@ph...> - 2003-05-13 22:15:37
|
On Tue, May 13, 2003 at 05:50:07PM -0400, Todd Miller wrote: > As has been mentioned before, we're planning to repackage numarray as > a package rather than as a collection of modules. We're soliciting > comments now because we only want to do this once. ... > Future 3rd party packages should locate themselves under the > numarray.addons sub-package: > > * --> numarray.addons.* What's the rationale for having a separate place for third-party packages? Shouldn't they just install themselves as their own packages or modules? (i.e., in site-packages/) -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |co...@ph... |
From: Todd M. <jm...@st...> - 2003-05-13 21:49:19
|
As has been mentioned before, we're planning to repackage numarray as a package rather than as a collection of modules. We're soliciting comments now because we only want to do this once. Here are the planned renamings for the numarray modules as we transition to a package: numarray --> numarray.numeric recarray --> numarray.records chararray --> numarray.strings ndarray --> numarray.generic * --> numarray.* Note that class and function names will remain unchanged, so recarray.RecArray will become numarray.records.RecArray. Here are the planned renamings for the current and planned add-on packages consolidated into a single add-ons distribution: LinearAlgrebra2 --> numarray.LinearAlgebra FFT2 --> numarray.FFT RandomArray2 --> numarray.RandomArray Convolve --> numarray.Convolve Convolve.Image --> numarray.Image MA2 --> numarray.MA Future 3rd party packages should locate themselves under the numarray.addons sub-package: * --> numarray.addons.* The package name (shown here as numarray) is still undecided. By including stub modules which emulate the "import behavior" of numarray-0.5, we'll make it possible to ignore / work around the effects of the re-packaging for the time being. The stub modules will be activated either through the creation of a .pth file or via additions to PYTHONPATH to make them visible. Comments? |
From: Andrew N. <aln...@st...> - 2003-05-13 06:25:09
|
On Tue, 13 May 2003, eric jones wrote: > Try fromstring. I think it should do what you want: > > >>> from Numeric import * print fromstring.__doc__ > > fromstring(string, typecode='l', count=-1) returns a new 1d array > initialized from the raw binary data in string. If count is > positive, the new array will have count elements, otherwise it's > size is determined by the size of string. Thanks, Eric. That did the trick. The manual said that Numeric.array() takes a sequence type as one of its arguments, and considering that a string is a sequence type, I thought that, given an appropriate typecode, it would do the conversion automatically in the same way that array.array does. Is this difference an intended feature of numpy? Andrew. |
From: eric j. <er...@en...> - 2003-05-13 06:10:36
|
Try fromstring. I think it should do what you want: >>> from Numeric import * >>> print fromstring.__doc__ fromstring(string, typecode='l', count=-1) returns a new 1d array initialized from the raw binary data in string. If count is positive, the new array will have count elements, otherwise it's size is determined by the size of string. eric ---------------------------------------------- eric jones 515 Congress Ave www.enthought.com Suite 1614 512 536-1057 Austin, Tx 78701 > -----Original Message----- > From: num...@li... [mailto:numpy- > dis...@li...] On Behalf Of Andrew Nesbit > Sent: Monday, May 12, 2003 11:57 PM > To: num...@li... > Subject: [Numpy-discussion] Converting string to array > > I have a string representing a sequence of 16-bit audio samples, read > in from an AIFF file like this: > > f = aifc.open("filename.aiff", "r") > data = f.readframes(f.getnframes()) > > I can convert this to a tuple of signed 16-bit integers using the > Python standard library "array" module: > > a = array.array("h", data) > > This gives samples in the range [-32768, 32767] and everything is > fine. > > However, instead of using module array I want to use Numeric's array > function in an equivalent way. I've been trying to do it like this: > > a = Numeric.array(data, Numeric.Int16) > > Now, a.itemsize() outputs 2, which seems fine, but when I iterate over > the resulting array (using standard a[i] notation), the results > indicate that the array is being treated as a sequence of 8-bit, > rather than 16-bit, integers. > > What am I doing wrong here? > > Thanks in advance. > > Andrew. > > > > > ------------------------------------------------------- > Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara > The only event dedicated to issues related to Linux enterprise solutions > www.enterpriselinuxforum.com > > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion |
From: Andrew N. <aln...@st...> - 2003-05-13 06:00:08
|
I have a string representing a sequence of 16-bit audio samples, read in from an AIFF file like this: f = aifc.open("filename.aiff", "r") data = f.readframes(f.getnframes()) I can convert this to a tuple of signed 16-bit integers using the Python standard library "array" module: a = array.array("h", data) This gives samples in the range [-32768, 32767] and everything is fine. However, instead of using module array I want to use Numeric's array function in an equivalent way. I've been trying to do it like this: a = Numeric.array(data, Numeric.Int16) Now, a.itemsize() outputs 2, which seems fine, but when I iterate over the resulting array (using standard a[i] notation), the results indicate that the array is being treated as a sequence of 8-bit, rather than 16-bit, integers. What am I doing wrong here? Thanks in advance. Andrew. |
From: <gve...@gr...> - 2003-05-12 22:52:19
|
> > > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > > science and engineering -- http://gerard.vermeulen.free.fr ) > > > > I discovered by accident that PyQwt compiled with > > #include <numarray/arrayobject.h> > > plots Numeric-arrays and that PyQwt compiled with > > #include <Numeric/arrayobject.h> > > plots numarray-arrays. > > > > This part sounds a little backward. Presumably this was a typo. > No Todd, this is no typo. I launched a test program from the wrong directory and was shocked to see that it worked (have checked with python -vv which modules were loaded). The other way around worked, too. Of course, the normal cases work, too. (verified that multiplying a Numeric array with numarray does not work) > > > The only interface between PyQwt and Numeric (or numarray) passes through > > PyArray_ContiguousFromObject to cast sequences into arrays from which data > > is copied into Qwt's data structures. > > > > PyQwt never returns Numeric- or numarray-arrays. > > > > Is this feature platform dependent? > > Yes, if I understand your question correctly. > > > Or is it safe to release PyQwt, > > claiming that a Numeric-compiled PyQwt is compatible with numarray > > and vice-versa? > > No dice. numarray is currently trying for source level compatibility > only, so an extension compiled for Numeric creates slightly different > object code than one compiled for numarray; slightly different means > inoperable. Still, it's good to know that PyQwt is very close to > workable for numarray. > Well, my experience shows that on x86-Linux the layout of the data in a numarray-array and Numeric-array is sufficiently close that both PyArray_ContiguousFromObject() functions also interprete the arrays of the 'cousin' correctly (at least in 1D). I realize that this observation is very hackish, but it may help to speed up the acceptance of numarray. Gerard ------------------------------------------------------------- This message was sent using HTTPS service from CNRS Grenoble. ---> https://grenoble.cnrs.fr <--- |
From: Todd M. <jm...@st...> - 2003-05-12 20:51:15
|
On Mon, 2003-05-12 at 16:06, Gerard Vermeulen wrote: > Hi, > > It was remarkable easy to make my plot package PyQwt for Numeric, also > compatible with numarray :-) > Great! That's they way things are *supposed* to work, and usually do, but don't always. > ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for > science and engineering -- http://gerard.vermeulen.free.fr ) > > I discovered by accident that PyQwt compiled with > #include <numarray/arrayobject.h> > plots Numeric-arrays and that PyQwt compiled with > #include <Numeric/arrayobject.h> > plots numarray-arrays. > This part sounds a little backward. Presumably this was a typo. > The only interface between PyQwt and Numeric (or numarray) passes through > PyArray_ContiguousFromObject to cast sequences into arrays from which data > is copied into Qwt's data structures. > > PyQwt never returns Numeric- or numarray-arrays. > > Is this feature platform dependent? Yes, if I understand your question correctly. > Or is it safe to release PyQwt, > claiming that a Numeric-compiled PyQwt is compatible with numarray > and vice-versa? No dice. numarray is currently trying for source level compatibility only, so an extension compiled for Numeric creates slightly different object code than one compiled for numarray; slightly different means inoperable. Still, it's good to know that PyQwt is very close to workable for numarray. > Gerard > _______________________________________________ > Numpy-discussion mailing list > Num...@li... > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jm...@st... STSCI / ESS / SSB |
From: Gerard V. <gve...@gr...> - 2003-05-12 20:07:05
|
Hi, It was remarkable easy to make my plot package PyQwt for Numeric, also compatible with numarray :-) ( PyQwt is a wrapper for Qwt, a Qt-based library with widgets useful for science and engineering -- http://gerard.vermeulen.free.fr ) I discovered by accident that PyQwt compiled with #include <numarray/arrayobject.h> plots Numeric-arrays and that PyQwt compiled with #include <Numeric/arrayobject.h> plots numarray-arrays. The only interface between PyQwt and Numeric (or numarray) passes through PyArray_ContiguousFromObject to cast sequences into arrays from which data is copied into Qwt's data structures. PyQwt never returns Numeric- or numarray-arrays. Is this feature platform dependent? Or is it safe to release PyQwt, claiming that a Numeric-compiled PyQwt is compatible with numarray and vice-versa? Gerard |
From: Francesc A. <fa...@op...> - 2003-05-12 18:12:45
|
PyTables 0.5.1 -------------- This is a maintenance release of PyTables. Due to a problem with an speci= fic optimization in PyTables 0.5, it does not work with numarray 0.4 (althoug= h it works just fine with numarray 0.5). Thanks to Marc Gehling for reporti= ng that. Todd Miller has already warned me that this optimization was not safe, so= I am *disabling* it in 0.5.1. The consequence is that the 20% of improvemen= t during reading tables has almost evaporated to a rather small 4%, but tha= t's life!. If you already have installed PyTables 0.5, I strongly suggest you to upgrade to 0.5.1, even if you are already using numarray 0.5. I will try = to further investigate the problem, and, if a good solution is found, I will enable again the optimization in a future release. Another new thing you can find in 0.5.1 is that the use of "UInt64" data types has been removed (it has been replaced by the "Int64" type) of the tutorial chapters in User's Manual. I've done that because the numarray Windows version does not support such a type (due to MSVC compiler limitations). Now, the tutorial section should run fine in all the suppor= ted platforms (even Windows). My apologies for being a sinner and trying to optimize too soon ;-) Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted |
From: Michiel J. L. de H. <md...@im...> - 2003-05-12 09:48:32
|
Hi, I have a C extension module that makes use of the random number functions in ranlib. Since numpy's RandomArray also relies on ranlib, there is a ranlib.so in site-packages/Numeric, and I found the ranlib.h header file in the include/python2.3/Numeric directory, containing the prototypes for the ranlib routines that I am trying to use. So I was hoping to #include this header file and link to numpy's ranlib.so instead of including another copy of ranlib as part of my C extension module. However, I haven't been able to get this to work -- on Linux and Mac OS X, gcc does not recognize ranlib.so as a library it can link to, whereas on Cygwin gcc cannot find the routines I am looking for in ranlib.dll. Does anybody know if it is possible to link to ranlib.so? If not, what is the function of the ranlib.h include file? Unfortunately the calls to the ranlib routines are way down in my C extension module, I wouldn't be able to call those routines at the Python level. Many thanks, --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon |
From: Francesc A. <fa...@op...> - 2003-05-10 11:39:08
|
Announcing PyTables 0.5 ----------------------- This is the second public beta release. On this release you will find a 20% of I/O speed improvement over the previous one (0.4), some bugs has been fixed and support for a couple of compression (LZO and UCL) libraries has been added, and... a long awaited Windows version is finally available!. More in detail: What's new ----------- - As a consequence of some twiking the write/read performance has been improved by a 20% overall. One particular case were performance has largely increased (0.5 is up to 6 times faster than 0.4) is when column elements are unidimensional arrays. This impressive speed-up is mainly because of the recent improvements in numarray 0.5 performance (good work, folks!). With that, the reading speed is reaching its theoretical maximum (at least when using the current data access schema). - When reading a Table object, and the user wants to fetch column elements which are unidimensional arrays, a copy of the array from the I/O buffer is delivered automatically to him, so that there is no need to make a call to .copy() method of the numarray arrays anymore. It think this is more comfortable for the user. - The compression was enabled by default in version 0.4, despite of what was stated in the documentation. Now, this has been corrected and compression is *disabled* by default. - Support for two new compression libraries: LZO and UCL (http://www.oberhumer.com/opensource/). These libraries are made by Markus F.X.J. Oberhumer, and they stand for allowing *very* fast decompression. Now, if your data is compressible, you can obtain better reading speed than if not using compression at all!. The improvement is still more noticeable if your are dealing with extremely large (and compressible) data sets. Read the online documentation for more info about that: http://pytables.sourceforge.net/html-doc/usersguide-html3.html#subsection= 3.4.1 - A couple of memory leaks has been isolated and fixed (it was hard, but I finally did it!). - A bug with column ordering of tables that happens in some special situations has been fixed (thanks to Stan Heckman for reporting this and suggesting the patch). - File class has now an 'isopen' attribute in order to check if a file is open or not. - Updated documentation, specially for giving advice about the use of the new compression libraries. See "Compression issues" subsection, (also on the web: http://pytables.sourceforge.net/html-doc/usersguide-html.html) - Added more unit tests (up to 218 now!) - PyTables has been tested against newest numarray 0.5 and it works just fine. It even works well with Python 2.3b1. - And last, but not least, a Windows version is available!. Thanks to Alan McIntyre for its porting!. There is even a binary ready for click and install. What it is ---------- In short, PyTables provides a powerful and very Pythonic interface to process and organize your table and array data on disk. Its goal is to enable the end user to manipulate easily scientific data tables and Numerical and numarray Python objects in a persistent hierarchical structure. The foundation of the underlying hierarchical data organization is the excellent HDF5 library (http://hdf.ncsa.uiuc.edu/HDF5). A table is defined as a collection of records whose values are stored in fixed-length fields. All records have the same structure and all values in each field have the same data type. The terms "fixed-length" and strict "data types" seems to be quite a strange requirement for an interpreted language like Python, but they serve a useful function if the goal is to save very large quantities of data (such as is generated by many scientific applications, for example) in an efficient manner that reduces demand on CPU time and I/O resources. Quite a bit effort has been invested to make browsing the hierarchical data structure a pleasant experience. PyTables implements just two (orthogonal) easy-to-use methods for browsing. What is HDF5? ------------- For those people who know nothing about HDF5, it is is a general purpose library and file format for storing scientific data made at NCSA. HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic constructs, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs. Platforms --------- I'm using Linux as the main development platform, but PyTables should be easy to compile/install on other UNIX machines. This package has also passed all the tests on a UltraSparc platform with Solaris 7 and Solaris 8. It also compiles and passes all the tests on a SGI Origin2000 with MIPS R12000 processors and running IRIX 6.5. With Windows, PyTables has been tested with Windows 2000 Professional SP1 and Windows XP, but it should also work with other flavors. An example? ----------- For online code examples, have a look at http://pytables.sourceforge.net/tut/tutorial1-1.html and=20 http://pytables.sourceforge.net/tut/tutorial1-2.html Web site -------- Go to the PyTables web site for more details: http://pytables.sourceforge.net/ Share your experience --------------------- Let me know of any bugs, suggestions, gripes, kudos, etc. you may have. Have fun! -- Francesc Alted |
From: Perry G. <pe...@st...> - 2003-05-07 20:35:42
|
There was general consensus that numarray should be organized as a package. We are about to start doing so. It does raise some issues (and opportunities) however. 1) What should the package be called? numarray is a possiblity, but we have an opportunity to take a different approach. For example if we could name the package "array" (unfortunately, we can't) then the following becomes possible (these are just examples, we haven't settled on any naming scheme yet). from array.numeric import * # instead of "from numarray import *" import array.fft import array.records # currently recarray Since array is not avaible (there is already an array module in the Python Standard Library) what are the alternatives? Here are some: num numarray arr arrays ndarray I've polled people locally and arrays and ndarray were the most popular, but there doesn't seem to be a clear winner. I tend toward arr somewhat because of the brevity but do realize it isn't terribly descriptive. The drawback of "arrays" is that it may be visually confused with array. 2) What to call the package components? Locally the favorites were numarray --> numeric recarray --> records chararray--> strings (but since we plan to implement PyObject arrays there is a possibilty of real Python string arrays) ndarray --> generic (or base) 3) Where to place 3rd party array extension modules (by that I mean all those not distributed with numarray)? Should they go in the package directory? I'd prefer that they go into a separate directory from the things that are part of the standard numarray distribution (a site-packages equivalent in the package?) 4) We would likely also include in the package numarray.py, recarray.py, chararray.py, ndarray.py so that if the package was added to the path, the current imports would continue to work. This is intended as a short-term measure (i.e., they are deprecated and will disappear at version 1.0, or earlier). We may also keep these files at the current nummarray interface so that some of the software we currently distribute does not need to change immediately to match the changes we have been talking about. We realize that these are big changes, but if we move to a package structure, we ought to think about what makes the most sense (and if it is any comfort, we suffer the consequences of change more than anyone else). Better now than later. Screams? Perry |