You can subscribe to this list here.
2000 
_{Jan}
(8) 
_{Feb}
(49) 
_{Mar}
(48) 
_{Apr}
(28) 
_{May}
(37) 
_{Jun}
(28) 
_{Jul}
(16) 
_{Aug}
(16) 
_{Sep}
(44) 
_{Oct}
(61) 
_{Nov}
(31) 
_{Dec}
(24) 

2001 
_{Jan}
(56) 
_{Feb}
(54) 
_{Mar}
(41) 
_{Apr}
(71) 
_{May}
(48) 
_{Jun}
(32) 
_{Jul}
(53) 
_{Aug}
(91) 
_{Sep}
(56) 
_{Oct}
(33) 
_{Nov}
(81) 
_{Dec}
(54) 
2002 
_{Jan}
(72) 
_{Feb}
(37) 
_{Mar}
(126) 
_{Apr}
(62) 
_{May}
(34) 
_{Jun}
(124) 
_{Jul}
(36) 
_{Aug}
(34) 
_{Sep}
(60) 
_{Oct}
(37) 
_{Nov}
(23) 
_{Dec}
(104) 
2003 
_{Jan}
(110) 
_{Feb}
(73) 
_{Mar}
(42) 
_{Apr}
(8) 
_{May}
(76) 
_{Jun}
(14) 
_{Jul}
(52) 
_{Aug}
(26) 
_{Sep}
(108) 
_{Oct}
(82) 
_{Nov}
(89) 
_{Dec}
(94) 
2004 
_{Jan}
(117) 
_{Feb}
(86) 
_{Mar}
(75) 
_{Apr}
(55) 
_{May}
(75) 
_{Jun}
(160) 
_{Jul}
(152) 
_{Aug}
(86) 
_{Sep}
(75) 
_{Oct}
(134) 
_{Nov}
(62) 
_{Dec}
(60) 
2005 
_{Jan}
(187) 
_{Feb}
(318) 
_{Mar}
(296) 
_{Apr}
(205) 
_{May}
(84) 
_{Jun}
(63) 
_{Jul}
(122) 
_{Aug}
(59) 
_{Sep}
(66) 
_{Oct}
(148) 
_{Nov}
(120) 
_{Dec}
(70) 
2006 
_{Jan}
(460) 
_{Feb}
(683) 
_{Mar}
(589) 
_{Apr}
(559) 
_{May}
(445) 
_{Jun}
(712) 
_{Jul}
(815) 
_{Aug}
(663) 
_{Sep}
(559) 
_{Oct}
(930) 
_{Nov}
(373) 
_{Dec}

S  M  T  W  T  F  S 




1

2
(5) 
3
(2) 
4
(2) 
5

6

7
(7) 
8
(1) 
9

10
(2) 
11
(1) 
12
(1) 
13
(1) 
14

15
(12) 
16
(6) 
17
(3) 
18
(4) 
19
(1) 
20
(2) 
21
(10) 
22
(3) 
23
(2) 
24
(15) 
25
(3) 
26
(3) 
27
(11) 
28
(3) 
29
(1) 
30
(5) 
31
(4) 

From: Rob Hooft <rob@ho...>  20030131 21:35:34

Chris Barker wrote: > > X = transpose([x]+[y]) > > > well, I learned a little bit more about Numeric today. > I've been skipping through a lot of messages today because I was getting behind on mailing list traffic, but I missed one thing in the discussion so far (sorry if it was marked already): transpose doesn't actually do any work. Actually, transpose only sets the "strides" counts differently, and this is blazingly fast. What is NOT fast is using the transposed array later! The problem is that many routines actually require a contiguous array, and will make a temporary local contiguous copy. This may happen multiple times if the lifetime of the transposed array is long. Even routines that do not require a contiguous array and can actually use the strides may run significantly slower because the CPU cache is trashed a lot by the high strides. Moral: you can't test this code by looping a 1000 times through it, you actually should take into account the time it takes to make a contiguous array immediately after the transpose call. Regards, Rob Hooft  Rob W.W. Hooft  rob@...  http://www.hooft.net/people/rob/ 
From: Chris Barker <Chris.B<arker@no...>  20030131 20:43:11

John Hunter wrote: > John> I have two equal length 1D arrays of 2564096 complex or > John> floating point numbers which I need to put into a > John> shape=(len(x),2) array. > I tested all the suggested methods and the transpose with [x] and [y] > was the clear winner, with an 8 fold speed up over my original code. > The concatenate method was between 23 times faster. I was a little surprised by this, as I figured that the transpose method made an extra copy of the data (array() makes one copy, transpose() another. So I looked at the source for concatenate: def concatenate(a, axis=0): """concatenate(a, axis=0) joins the tuple of sequences in a into a single NumPy array. """ if axis == 0: return multiarray.concatenate(a) else: new_list = [] for m in a: new_list.append(swapaxes(m, axis, 0)) return swapaxes(multiarray.concatenate(new_list), axis, 0) So, if you are concantenating along anything other than the zeroth axis, you end up doing something similar to the transpose method. Seeign this, I trioed something else: def test_concat2(x,y): x.shape = (1,1) y.shape = (1,1) X = transpose( concatenate( (x, y) ) ) x.shape = (1,) y.shape = (1,) This then uses the native concatenate, but requires an extra copy in teh transpose. Here's a somewhat cleaner version, though you get more copies: def test_concat3(x,y): "Thanks to Chris Barker and Bryan Cole" X = transpose( concatenate( ( reshape(x,(1,1)), reshape(y,(1,1)) ) ) ) Here are the test results: testing on vectors of length: 4096 test_concat 0.286280035973 test_transpose 0.100033998489 test_naive 0.805399060249 test_concat3 0.109319090843 test_concat2 0.136469960213 All the transpose methods are essentially a tie. Would it be that hard for concatenate to do it's thing for any axis in C? It does seem like this is a fairly basic operation, and shouldn't require more than one copy. By the way, I realised that the transpose method had an extra call. transpose() can take an approprriate python sequence, so this works just fine: def test_transpose2(x,y): X = transpose([x]+[y]) However, it doesn't really save you the copy, as I'm retty sure transpose makes a copy internally anyway. Test results: testing on vectors of length: 4096 test_transpose 0.104995965958 test_transpose2 0.103582024574 I think the winner is: X = transpose([x]+[y]) well, I learned a little bit more about Numeric today. Chris  Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@... 
From: Todd Miller <jmiller@st...>  20030131 15:30:03

Sebastian Haase wrote: >Hi everybody, >I tried a 'python2.2 setup.py install' >of numarray on a Mac running osX (10.1; I have also Fink installed) >I starts crunching until: >/usr/bin/ld: Undefined symbols: >_fclearexcept >_fetestexcept > >Anyone out there, who uses numarray on osX ? > >I'm thankful for any pointer... > >Sebastian Haase > > Hi Sebastian, I am very much a MacAmateur, but I have run numarray under osX by first installing a local UNIX version of Python using the source tarball. The steps were roughly as follows: 1. Obtain and unpack the Python source tarball in you home directory. cd there. 2. Configure Python using: ./configure prefix=$HOME 3. Edit the Makefile for the following: 61c61 > LDFLAGS=  < LDFLAGS= framework System framework CoreServices framework Foundation This was the only (reasonable) way I could figure out how to tunnel link time options down through the distutils in the proper command line order. I'm not really sure this is a minimal set of frameworks, but it did at least work. 4. Build and install python: make ; make install 5. Obtain and unpack the numarray source tarball. cd there. 6. Build and install numarray: python setupall.py install 7. Put $HOME/bin on your PATH and rehash. Todd > > > > >This SF.NET email is sponsored by: >SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See! >http://www.vasoftware.com >_______________________________________________ >Numpydiscussion mailing list >Numpydiscussion@... >https://lists.sourceforge.net/lists/listinfo/numpydiscussion > > 
From: Sebastian Haase <haase@ms...>  20030131 13:11:14

Hi everybody, I tried a 'python2.2 setup.py install' of numarray on a Mac running osX (10.1; I have also Fink installed) I starts crunching until: /usr/bin/ld: Undefined symbols: _fclearexcept _fetestexcept Anyone out there, who uses numarray on osX ? I'm thankful for any pointer... Sebastian Haase 
From: John Hunter <jdhunter@ac...>  20030130 22:17:30

I needed some spectral analysis functions, and finding none available, wrote my own. I use matlab a lot, so I wrote them to be matlab compatible. If you all think these look OK, I'm happy to submit them for inclusion into MLab.  """ Spectral analysis functions for Numerical python written for compatability with matlab commands with the same names. psd  Power spectral density uing Welch's average periodogram csd  Cross spectral density uing Welch's average periodogram cohere  Coherence (normalized cross spectral density) corrcoef  The matrix of correlation coefficients The functions are designed to work for real and complex valued Numeric arrays. One of the major differences between this code and matlab's is that I use functions for 'detrend' and 'window', and matlab uses vectors. This can be easily changed, but I think the functional approach is a bit more elegant. Please send comments, questions and bugs to: Author: John D. Hunter <jdhunter@...> """ from __future__ import division from MLab import mean, hanning, cov from Numeric import zeros, ones, diagonal, transpose, matrixmultiply, \ resize, sqrt, divide, array, Float, Complex, concatenate, \ convolve, dot, conjugate, absolute, arange, reshape from FFT import fft def norm(x): return sqrt(dot(x,x)) def window_hanning(x): return hanning(len(x))*x def window_none(x): return x def detrend_mean(x): return x  mean(x) def detrend_none(x): return x def detrend_linear(x): """Remove the best fit line from x""" # I'm going to regress x on xx=range(len(x)) and return # x  (b*xx+a) xx = arange(len(x), typecode=x.typecode()) X = transpose(array([xx]+[x])) C = cov(X) b = C[0,1]/C[0,0] a = mean(x)  b*mean(xx) return x(b*xx+a) def psd(x, NFFT=256, Fs=2, detrend=detrend_none, window=window_hanning, noverlap=0): """ The power spectral density by Welches average periodogram method. The vector x is divided into NFFT length segments. Each segment is detrended by function detrend and windowed by function window. noperlap gives the length of the overlap between segments. The absolute(fft(segment))**2 of each segment are averaged to compute Pxx, with a scaling to correct for power loss due to windowing. Fs is the sampling frequency.  NFFT must be a power of 2  detrend and window are functions, unlike in matlab where they are vectors.  if length x < NFFT, it will be zero padded to NFFT Refs: Bendat & Piersol  Random Data: Analysis and Measurement Procedures, John Wiley & Sons (1986) """ if NFFT % 2: raise ValueError, 'NFFT must be a power of 2' # zero pad x up to NFFT if it is shorter than NFFT if len(x)<NFFT: n = len(x) x = resize(x, (NFFT,)) x[n:] = 0 # for real x, ignore the negative frequencies if x.typecode()==Complex: numFreqs = NFFT else: numFreqs = NFFT//2+1 windowVals = window(ones((NFFT,),x.typecode())) step = NFFTnoverlap ind = range(0,len(x)NFFT+1,step) n = len(ind) Pxx = zeros((numFreqs,n), Float) # do the ffts of the slices for i in range(n): thisX = x[ind[i]:ind[i]+NFFT] thisX = windowVals*detrend(thisX) fx = absolute(fft(thisX))**2 Pxx[:,i] = fx[:numFreqs] # Scale the spectrum by the norm of the window to compensate for # windowing loss; see Bendat & Piersol Sec 11.5.2 if n>1: Pxx = mean(Pxx,1) Pxx = divide(Pxx, norm(windowVals)**2) freqs = Fs/NFFT*arange(0,numFreqs) return Pxx, freqs def csd(x, y, NFFT=256, Fs=2, detrend=detrend_none, window=window_hanning, noverlap=0): """ The cross spectral density Pxy by Welches average periodogram method. The vectors x and y are divided into NFFT length segments. Each segment is detrended by function detrend and windowed by function window. noverlap gives the length of the overlap between segments. The product of the direct FFTs of x and y are averaged over each segment to compute Pxy, with a scaling to correct for power loss due to windowing. Fs is the sampling frequency. NFFT must be a power of 2 Refs: Bendat & Piersol  Random Data: Analysis and Measurement Procedures, John Wiley & Sons (1986) """ if NFFT % 2: raise ValueError, 'NFFT must be a power of 2' # zero pad x and y up to NFFT if they are shorter than NFFT if len(x)<NFFT: n = len(x) x = resize(x, (NFFT,)) x[n:] = 0 if len(y)<NFFT: n = len(y) y = resize(y, (NFFT,)) y[n:] = 0 # for real x, ignore the negative frequencies if x.typecode()==Complex: numFreqs = NFFT else: numFreqs = NFFT//2+1 windowVals = window(ones((NFFT,),x.typecode())) step = NFFTnoverlap ind = range(0,len(x)NFFT+1,step) n = len(ind) Pxy = zeros((numFreqs,n), Complex) # do the ffts of the slices for i in range(n): thisX = x[ind[i]:ind[i]+NFFT] thisX = windowVals*detrend(thisX) thisY = y[ind[i]:ind[i]+NFFT] thisY = windowVals*detrend(thisY) fx = fft(thisX) fy = fft(thisY) Pxy[:,i] = fy[:numFreqs]*conjugate(fx[:numFreqs]) # Scale the spectrum by the norm of the window to compensate for # windowing loss; see Bendat & Piersol Sec 11.5.2 if n>1: Pxy = mean(Pxy,1) Pxy = divide(Pxy, norm(windowVals)**2) freqs = Fs/NFFT*arange(0,numFreqs) return Pxy, freqs def cohere(x, y, NFFT=256, Fs=2, detrend=detrend_none, window=window_hanning, noverlap=0): """ cohere the coherence between x and y. Coherence is the normalized cross spectral density Cxy = Pxy^2/(Pxx*Pyy) The return value is (Cxy, f), where f are the frequencies of the coherence vector. See the docs for psd and csd for information about the function arguments NFFT, detrend, windowm noverlap, as well as the methods used to compute Pxy, Pxx and Pyy. """ Pxx,f = psd(x, NFFT=NFFT, Fs=Fs, detrend=detrend, window=window, noverlap=noverlap) Pyy,f = psd(y, NFFT=NFFT, Fs=Fs, detrend=detrend, window=window, noverlap=noverlap) Pxy,f = csd(x, y, NFFT=NFFT, Fs=Fs, detrend=detrend, window=window, noverlap=noverlap) Cxy = divide(absolute(Pxy)**2, Pxx*Pyy) return Cxy, f def corrcoef(*args): """ corrcoef(X) where X is a matrix returns a matrix of correlation coefficients for each row of X. corrcoef(x,y) where x and y are vectors returns the matrix or correlation coefficients for x and y. Numeric arrays can be real or complex The correlation matrix is defined from the covariance matrix C as r(i,j) = C[i,j] / (C[i,i]*C[j,j]) """ if len(args)==2: X = transpose(array([args[0]]+[args[1]])) elif len(args==1): X = args[0] else: raise RuntimeError, 'Only expecting 1 or 2 arguments' C = cov(X) d = resize(diagonal(C), (2,1)) r = divide(C,sqrt(matrixmultiply(d,transpose(d))))[0,1] try: return r.real except AttributeError: return r  I wrote a little test code comparing the output of matlab's equivalent functions. Basically, I compute the psd or cohere in matlab and python and do the rms difference on the resultant vectors RMS cohere python/matlab difference 0.000854587104587 RMS psd python/matlab difference 0.00210783306638 I am not sure where these differences are arising, but they are quite small. I'm going to keep trying to track them down. For corrcoef, the answers are the same past 8 significant digits. Hope this helps! John Hunter 
From: John Hunter <jdhunter@ac...>  20030130 19:25:28

>>>>> "John" == John Hunter <jdhunter@...> writes: John> I have two equal length 1D arrays of 2564096 complex or John> floating point numbers which I need to put into a John> shape=(len(x),2) array. John> I need to do this a lot, so I would like to use the most John> efficient means. Currently I am doing: I tested all the suggested methods and the transpose with [x] and [y] was the clear winner, with an 8 fold speed up over my original code. The concatenate method was between 23 times faster. Thanks to all who responded, John Hunter cruncher2:~/python/test> python test.py test_naive test_naive 0.480427026749 cruncher2:~/python/test> python test.py test_concat test_concat 0.189149975777 cruncher2:~/python/test> python test.py test_transpose test_transpose 0.0698409080505 from Numeric import transpose, concatenate, reshape, array, zeros from RandomArray import normal import time, sys def test_naive(x,y): "Naive approach" X = zeros( (len(x),2), typecode=x.typecode()) X[:,0] = x X[:,1] = y def test_concat(x,y): "Thanks to Chris Barker and Bryan Cole" X = concatenate( ( reshape(x,(1,1)), reshape(y,(1,1)) ), 1) def test_transpose(x,y): "Thanks to Joachim Saul" X = transpose(array([x]+[y])) m = {'test_naive' : test_naive, 'test_concat' : test_concat, 'test_transpose' : test_transpose} nse1 = normal(0.0, 1.0, (4096,)) nse2 = normal(0.0, 1.0, nse1.shape) N = 1000 trials = range(N) func = m[sys.argv[1]] t1 = time.time() for i in trials: func(nse1,nse2) t2 = time.time() print sys.argv[1], t2t1 
From: Pearu Peterson <pearu@ce...>  20030130 09:50:39

On Wed, 29 Jan 2003, John Hunter wrote: > > I have two equal length 1D arrays of 2564096 complex or floating > point numbers which I need to put into a shape=(len(x),2) array. > > I need to do this a lot, so I would like to use the most efficient > means. Currently I am doing: > > def somefunc(x,y): > X = zeros( (len(x),2), typecode=x.typecode()) > X[:,0] = x > X[:,1] = y > do_something_with(X) > > Is this the fastest way? May be you could arange your algorithm so that you first create X and then reference its columns by x,y without copying: # Allocate memory X = zeros( (n,2), typecode=.. ) # Get references to columns x = X[:,0] y = X[:,1] while 1: do_something_inplace_with(x,y) do_something_with(X) Pearu 
From: Karthikesh Raju <karthik@ja...>  20030130 09:46:18

Hi, i was tring out something like this import Numeric import LinearAlgebra import cmath import RandomArray import copy def sMatrix(pd, code, window): if window == 0: nprime = 1 else: nprime = window K, C = Numeric.shape(code) K1, L = Numeric.shape(pd) # check if K == K1 and raise an exception here sCode = Numeric.zeros([nprime*C,K*L*(window+1)],'d') for k in range(K): for l in range(L): code1 = copy.deepcopy(Numeric.array(code[k,0:Cpd[k,l]])) code1.shape = (Cpd[k,l],1) sCode1= Numeric.concatenate((Numeric.zeros([pd[k,l],1]),Numeric.zeros([C*window,1]),code1)) sCode[:, (window+1)*l+window*L*k] = copy.deepcopy(sCode1) return sCode if __name__ == "__main__": pd = Numeric.array([[2]]) code = Numeric.array([[1,1,1,1,1]]) np = sMatrix(pd,code,0) print np print ""*30 np = sMatrix(pd,code,1) print Numeric.shape(np) print np print ""*30 np = sMatrix(pd,code,2) print Numeric.shape(np) print np print ""*30  And i get struck with the following error message:: Traceback (most recent call last): File "sMatrix.py", line 31, in ? np = sMatrix(pd,code,0) File "sMatrix.py", line 24, in sMatrix sCode[:, (window+1)*l+window*L*k] = copy.deepcopy(sCode1) ValueError: Object too deep for desired array  i think it is due to the many deep copy operations taht i am performing. i want to be in a position where slices of matrices should not be references, but should be copies itself and i should be able to move these copies around. (May be it is inefficient, but that is what i did in Matlab and want some compatibility, till i learn more of python and till i migrate to python completely). Is there a way out? Why is this an problem? Am i missing something. Best regards, karthik  Karthikesh Raju, email: karthik@... Researcher, http://www.cis.hut.fi/karthik Helsinki University of Technology, Tel: +3589451 5389 Laboratory of Comp. & Info. Sc., Fax: +3589451 3277 Department of Computer Sc., P.O Box 5400, FIN 02015 HUT, Espoo, FINLAND  
From: Joachim Saul <list@js...>  20030130 09:19:48

* John Hunter [20030129 22:13]: > def somefunc(x,y): > X = zeros( (len(x),2), typecode=x.typecode()) > X[:,0] = x > X[:,1] = y > do_something_with(X) > > Is this the fastest way? X = transpose(array([x]+[y])) It may not be the fastest possible way, but should be about a factor of two faster; better than nothing. Cheers, Joachim 
From: John Hunter <jdhunter@ac...>  20030129 21:12:32

I have two equal length 1D arrays of 2564096 complex or floating point numbers which I need to put into a shape=(len(x),2) array. I need to do this a lot, so I would like to use the most efficient means. Currently I am doing: def somefunc(x,y): X = zeros( (len(x),2), typecode=x.typecode()) X[:,0] = x X[:,1] = y do_something_with(X) Is this the fastest way? Thanks, John Hunter 
From: Perry Greenfield <perry@st...>  20030128 21:58:40

> Yes, this is critical. Isn't there a plan to make the scalar  rank0 > array dicotomy a little cleaner in NumArray ? > Hmmm, I'd like to say yes, but I'm not sure what exactly you are referring to. Please elaborate on how you think it should be changed. About the only thing that comes to mind is that repr() for rank0 will be different for numarray than Numeric, and that it will never be the result of any reduction or similar selection. > > > I also agree that the point is not subclassing per se, it's > > > polymorphism. It should be easy to write a class that acts > like an array > > > in all the ways that you need it to. > > > > True, and that is a weak point of NumPy. > > Is this getting any better with NumArray? > Again, I hope so, but I find this too general to know if it satisfies anyone's specific goals. I'd like to see specific examples. I think it is often tricker than people initially think. Perry 
From: Francesc Alted <falted@op...>  20030128 19:41:32

Hi, A couple of points related with numarray type objects: 1. When working with numeric types instances like UInt8 or Float64, is there a way to access to their enumeration NumarrayType C counterpart?. T= hat can be handy when want to map from these objects and integers. For example, right now, I'm forced to use these mappings in Pyrex: # Conversion tables from/to classes to the numarray enum types toenum =3D {num.Int8:tInt8, num.UInt8:tUInt8, num.Int16:tInt16, num.UInt16:tUInt16, num.Int32:tInt32, num.UInt32:tUInt32, num.Float32:tFloat32, num.Float64:tFloat64, CharType:97 # ascii(97) > 'a' # Special case (to be correct= ed) } toclass =3D {tInt8:num.Int8, tUInt8:num.UInt8, tInt16:num.Int16, tUInt16:num.UInt16, tInt32:num.Int32, tUInt32:num.UInt32, tFloat32:num.Float32, tFloat64:num.Float64, 97:CharType # ascii(97) > 'a' # Special case (to be correc= ted) } (yes, Pyrex lets you do that kind of "miracles", like mappings between Python objects and C integers) but if I had this access directly from the object (for example Int8.enumType), my code (and Cextensions in general) could look simpler. 2. I understand now why Todd was worried about CharArray objects to be assigned to an enumerated type. In fact, if you look at the above maps, I have to map myself this special object as the number 97 (which is the asc= ii value for character "a"). 97 is ok for now because it can't collide (at least for a while) with other enumeration types. My suggestion is that it would be a good thing to have a reserved enum ty= pe for CharArray. And I think that mapping CharArrays with Bool or Int8, wou= ld not be a good solution because chararray objects differ in some ways from them, that it would be a mess to distinguish both objects in Ccode by ju= st looking at its enumeration type.=20 I don't know, but maybe recarrays also merit a place in enumeration (?).=20 By the way, after the discussion with Todd I finally decided to remove al= l the Numeric charcodes (and related codes) from PyTables. However, I can still manage Numeric objects by converting them to numarray and accessing the class type with the .type() method. An you know that? the code looks much more logical and neat, and best of all, less errorprone (well, at least I hope so!). I definitely encourage you to do a similar transition = in numarray (although I guess that would be more difficult because you still need to Numeric compatibility). Thanks, =20 Francesc Alted 
From: Chris Barker <Chris.B<arker@no...>  20030128 19:01:30

Konrad Hinsen wrote: > > M = array(l) > > Mt = M.transpose() > > > > just isn't that much worse than: > > > > Mt = transpose(l) > > No, but the automatic conversion enables me to write functions that > accept any sequence type without even having to think about it. I've used that to, but I also frequently use something like this: def function(A): A = array(A) ... Which is pretty simple to. > Moreover, it is almost essential in many situations to accept scalars > in place of arrays, because scalars fulfill the role of rank0 arrays. Yes, this is critical. Isn't there a plan to make the scalar  rank0 array dicotomy a little cleaner in NumArray ? > > I also agree that the point is not subclassing per se, it's > > polymorphism. It should be easy to write a class that acts like an array > > in all the ways that you need it to. > > True, and that is a weak point of NumPy. Is this getting any better with NumArray? Chris  Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@... 
From: Todd Miller <jmiller@st...>  20030127 20:53:57

Francesc Alted wrote: >A Dilluns 27 Gener 2003 20:52, Todd Miller va escriure: > > >>>Oh!, I didn't know that. In such a case, I think it's worth to consider >>>the possibility to define records as classes descendants from >>>metaclasses. But, of course, you have the ultimate decision. >>> >>> >>I don't know what you mean here. Please spell it out a little more. >> >> > >I was trying to mean that using something like : > >class Small(IsRecord): > field1 = defineType(CharType, 2, default="", position=1) > field2 = defineType(Int32, 1, position=2) > field3 = Float64 > >as as container for recarray metadata is definitely possible instead of the >tuple (formats="2aid",names=("field1","field2", "field3")), if using >Python2.2. IsRecord is a metaclass (introduced in Python 2.2) that allows >you to effectively separate the declared attributes from the implicit ones >in normal classes. > >Of course, you can taylor IsRecord so as to fulfill your needs. > >I hope that I have expressed myself more clearly now, > > > I looked at your docs here: http://pytables.sourceforge.net/htmldoc/usersguidehtml4.html#section4.2 and what you said above clicked. Thanks. Todd 
From: Francesc Alted <falted@op...>  20030127 20:15:08

A Dilluns 27 Gener 2003 20:52, Todd Miller va escriure: > > > >Oh!, I didn't know that. In such a case, I think it's worth to conside= r > > the possibility to define records as classes descendants from > > metaclasses. But, of course, you have the ultimate decision. > > I don't know what you mean here. Please spell it out a little more. I was trying to mean that using something like : class Small(IsRecord): field1 =3D defineType(CharType, 2, default=3D"", position=3D1) field2 =3D defineType(Int32, 1, position=3D2) field3 =3D Float64 as as container for recarray metadata is definitely possible instead of t= he tuple (formats=3D"2aid",names=3D("field1","field2", "field3")), if using Python2.2. IsRecord is a metaclass (introduced in Python 2.2) that allows you to effectively separate the declared attributes from the implicit one= s in normal classes. Of course, you can taylor IsRecord so as to fulfill your needs. I hope that I have expressed myself more clearly now, =20 Francesc Alted 
From: Todd Miller <jmiller@st...>  20030127 19:53:01

Francesc Alted wrote: >A Dilluns 27 Gener 2003 17:28, Todd Miller va escriure: > > >>>So, as numarray has to work with previous python versions, there is no >>>point to care about that. >>> >>> >>In truth, numarray0.4 and up already require Python2.2 and up. >> >> > >Oh!, I didn't know that. In such a case, I think it's worth to consider the >possibility to define records as classes descendants from metaclasses. But, >of course, you have the ultimate decision. > > I don't know what you mean here. Please spell it out a little more. > > >>>>I'm thinking the general format for this may be converting Ntuples of >>>>types and ints into N arrays of types and ints. And vice versa. >>>>It's obvious how this works with numarray types. I think the chararray >>>>types need work and need to be mapped into the same integer enumeration >>>>as the numeric types in a nonoverlapping way. >>>> >>>> >>>I can't catch your point here. Why there should be a problem with >>>chararrays?. >>> >>> >>What I was trying to see is that chararray types are not as well >>designed as the numarray types, nor are they reflected in the CAPI. >> >> > >I see. Well, is it really desirable such a unification? CharArray entities >come from a module and NumArray from another one, and that should be ok. Why >bother in creating a unified API or integer enumeration?. > It may not be necessary. Int8 with repitition factors may work about the same. 
From: Todd Miller <jmiller@st...>  20030127 19:38:07

Paul F Dubois wrote: >IMHO you can assume any Python you want. Look to the long term here, not the >short. > You lost me. numarray0.4 needs at least Python2.2 or baseclasses don't exist. I had a slow Python equivalent for the baseclass as I refactored prior to numarray0.4, but it's gone now. > >I'm a bit uncertain on MA as to whether my old design is right. Maybe I >should be inheriting from NDarray? So that MA is more of a sibling of >numarray rather than a wrapper of it? > > I asked Perry about this one. His points (salted a little by me) were: 1. If you inherit from NumArray, you also inherit from NDArray. If you only inherit from NDArray, all you get are the structural operations. 2. If you inherit from NumArray, you can use Liskov substitution to pass MA's directly into extensions expecting NumArrays. This substitution may or may not be good. Also, isinstance(anMA, numarray) will return True. 3. If you inherit from NumArray, you get numerical method definitions which may or may not be applicable to MA. With a little thrashing, we might also get MAs to work for ufuncs. In fact, ufuncs are the key to whether or not the NumArray numerical methods add any value. Todd > > 
From: Francesc Alted <falted@op...>  20030127 19:22:02

A Dilluns 27 Gener 2003 17:28, Todd Miller va escriure: > >So, as numarray has to work with previous python versions, there is no > > point to care about that. > > In truth, numarray0.4 and up already require Python2.2 and up. Oh!, I didn't know that. In such a case, I think it's worth to consider t= he possibility to define records as classes descendants from metaclasses. Bu= t, of course, you have the ultimate decision. > >>I'm thinking the general format for this may be converting Ntuples o= f > >>types and ints into N arrays of types and ints. And vice versa. > >>It's obvious how this works with numarray types. I think the chararr= ay > >>types need work and need to be mapped into the same integer enumerati= on > >>as the numeric types in a nonoverlapping way. > > > >I can't catch your point here. Why there should be a problem with > >chararrays?. > > What I was trying to see is that chararray types are not as well > designed as the numarray types, nor are they reflected in the CAPI. I see. Well, is it really desirable such a unification? CharArray entitie= s come from a module and NumArray from another one, and that should be ok. = Why bother in creating a unified API or integer enumeration?. I think this should be not a big drawback for Cextension crafters (although, to say t= he truth, that would be very elegant if you manage to do that, but maybe it = is not worth the effort, I don't know). =20 Francesc Alted 
From: Todd Miller <jmiller@st...>  20030127 18:33:44

Chris Barker wrote: >Francesc Alted wrote: > > > >>So, as numarray has to work with previous python versions, >> >> > >Why? Anyone using NumArray is either starting from scratch or porting >from Numeric, so having to port to a newer version of Python is a very >small deal. > > Just to make it very clear: numarray0.4 and up require Python2.2 or higher. Up until numarray0.4 (released in November), that was not the case, and numarray ran (and was tested!) on Python2.0 and higher. The desire to increase Clevel Numeric compatability and to improve simple indexing speed led us to a C baseclass, which is only supported in Python2.2 and up. Todd 
From: Chris Barker <Chris.B<arker@no...>  20030127 18:19:30

Francesc Alted wrote: > So, as numarray has to work with previous python versions, Why? Anyone using NumArray is either starting from scratch or porting from Numeric, so having to port to a newer version of Python is a very small deal. Chris  Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@... 
From: Francesc Alted <falted@op...>  20030127 16:38:21

A Dilluns 27 Gener 2003 15:42, Todd Miller va escriure: > Yes. So one question is: if we were to add typerepetition tuples to > recarray as an alternative to the current character code strings, woul= d > that be any form of improvement to recarray from your perspective? Well, at least, charcodes can be avoided. I think it's a big win... or ma= ybe not as big? > > As I see it, recarray currently has a clean seperation between format > and naming which permits the latter to be optional. Before changing > that, I'd need a clear argument why. (I didn't design and generally > don't even maintain recarray). One argument is the fact that a map is very clear to the user, although t= hat such a map can be built *after* the names and format are passed to the recarray constructor and be accessible as an atribute. However, the latte= r solution is worse IMO, because the user has to supply two separate pieces= of information when, actually, these should be regarded as a unity. Anyway, this maybe a subjective perception. > >This would not be difficult to support because, by accessing to the > >Small().__dict__, you get also a dictionary. In addition, the latter w= ill > >ensure (by construction) that you are not using a nonvalid python > >identifier, which is mandatory in my current implementation. I find th= ese > >containers (dictionaries and classes) both elegant and convenient. > > I'm not trying to be Mr. Negative here, but one thing to keep in mind Oh dear, you are right!. In fact, I forgot that to make this to work, you need to use the metaclasses introduced in Python 2.2 (see Alex Martelli's post: http://mail.python.org/pipermail/pythonlist/2002July/112007.html)= =2E I was following this recipe, but I forgot that I was using Python 2.2. So, as numarray has to work with previous python versions, there is no po= int to care about that. > > I'm sure that's not the easiest way to capture struct layout, but I > take your point. Since position matters to me, I'd prefer that > capturing them was implicit. Since it doesn't to you, it seems OK for > it to be explicit. Either default mode can support the other, but > capturing order with tuples is free, while capturing order with a > __dict__ will take some kind of extra work. That's right. We have some different needs and priorities, and we should take the approach better suited to each other. But exchanging points of v= iew is always a great thing. > > I'm thinking the general format for this may be converting Ntuples of > types and ints into N arrays of types and ints. And vice versa. > It's obvious how this works with numarray types. I think the chararray > types need work and need to be mapped into the same integer enumeration > as the numeric types in a nonoverlapping way. > I can't catch your point here. Why there should be a problem with chararrays?. =20 Francesc Alted 
From: Todd Miller <jmiller@st...>  20030127 16:29:32

Francesc Alted wrote: >A Dilluns 27 Gener 2003 15:42, Todd Miller va escriure: > > >>Yes. So one question is: if we were to add typerepetition tuples to >>recarray as an alternative to the current character code strings, would >>that be any form of improvement to recarray from your perspective? >> >> > >Well, at least, charcodes can be avoided. I think it's a big win... or maybe >not as big? > > I think that avoiding the charcodes would be an improvement. Typerepetition tuples provide a clear well defined way to define data formats. It's not so clear that it eliminates the requirement for ongoing Numeric compatability, but it might. > > >>As I see it, recarray currently has a clean seperation between format >>and naming which permits the latter to be optional. Before changing >>that, I'd need a clear argument why. (I didn't design and generally >>don't even maintain recarray). >> >> > >One argument is the fact that a map is very clear to the user, although that >such a map can be built *after* the names and format are passed to the >recarray constructor and be accessible as an atribute. However, the latter >solution is worse IMO, because the user has to supply two separate pieces of >information when, actually, these should be regarded as a unity. Anyway, >this maybe a subjective perception. > > Well, I think there's truth to the danger of seperating names from data declarations, but it is easy to map keys(), values() to the seperate pieces in a different layer if necessary. >This would not be difficult to support because, by accessing to the >Small().__dict__, you get also a dictionary. In addition, the latter will >ensure (by construction) that you are not using a nonvalid python >identifier, which is mandatory in my current implementation. I find these >containers (dictionaries and classes) both elegant and convenient. > > >>I'm not trying to be Mr. Negative here, but one thing to keep in mind >> >> > >Oh dear, you are right!. > For a few seconds there, I thought I was on a roll! >In fact, I forgot that to make this to work, you >need to use the metaclasses introduced in Python 2.2 (see Alex Martelli's >post: http://mail.python.org/pipermail/pythonlist/2002July/112007.html). >I was following this recipe, but I forgot that I was using Python 2.2. > >So, as numarray has to work with previous python versions, there is no point >to care about that. > > In truth, numarray0.4 and up already require Python2.2 and up. >I'm sure that's not the easiest way to capture struct layout, but I >take your point. Since position matters to me, I'd prefer that >capturing them was implicit. Since it doesn't to you, it seems OK for >it to be explicit. Either default mode can support the other, but >capturing order with tuples is free, while capturing order with a >__dict__ will take some kind of extra work. > > > >That's right. We have some different needs and priorities, and we should >take the approach better suited to each other. But exchanging points of view >is always a great thing. > > > >>I'm thinking the general format for this may be converting Ntuples of >>types and ints into N arrays of types and ints. And vice versa. >>It's obvious how this works with numarray types. I think the chararray >>types need work and need to be mapped into the same integer enumeration >>as the numeric types in a nonoverlapping way. >> >> >> > >I can't catch your point here. Why there should be a problem with >chararrays?. > What I was trying to see is that chararray types are not as well designed as the numarray types, nor are they reflected in the CAPI. > > 
From: Todd Miller <jmiller@st...>  20030127 14:43:08

Francesc Alted wrote: >Yeah. We just differ in the way to arrange this metadata to be passed to the >recarray constructor. But I think this is secondary compared to the >flexibility that a verbose approach offers compared with the actual string >format. > Yes. So one question is: if we were to add typerepetition tuples to recarray as an alternative to the current character code strings, would that be any form of improvement to recarray from your perspective? As I see it, recarray currently has a clean seperation between format and naming which permits the latter to be optional. Before changing that, I'd need a clear argument why. (I didn't design and generally don't even maintain recarray). >In fact, more than one container might be supported to define the >metadata; one can start with tuples as you suggest, but in the future other >ways can be added (if considered convenient). > > >For example, I think I'll stick with the dictionary option for PyTables, but >also a class declaration for the metadata would be supported, like in : > >class Small(IsRecord): > var1 = defineType(CharType, 2, "") > var2 = defineType(Int32, 1) > var3 = Float64 > >This would not be difficult to support because, by accessing to the >Small().__dict__, you get also a dictionary. In addition, the latter will >ensure (by construction) that you are not using a nonvalid python >identifier, which is mandatory in my current implementation. I find these >containers (dictionaries and classes) both elegant and convenient. > > I'm not trying to be Mr. Negative here, but one thing to keep in mind is this: >>> class C: ... pass ... >>> c = C() >>> dir(c.__dict__) ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__repr__', '__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] Which is to say, the instance dictionary is a little cluttered, and it might not be that easy to determine which objects in it are there to define the data format. >>Just like the type repetition tuple except also including field names >>and default values. I don't think you lost me. For what we do, the >>exact physical layout of the "struct" is important, so order matters. I >>see order as part of the >>metadata, but I don't usually deal with metaentities so maybe I've >>got that part wrong. :) >> > >Well, if you need positional fields, you may add a (optional) parameter, >called for example, "position" so that you can fix it. > > I'm sure that's not the easiest way to capture struct layout, but I take your point. Since position matters to me, I'd prefer that capturing them was implicit. Since it doesn't to you, it seems OK for it to be explicit. Either default mode can support the other, but capturing order with tuples is free, while capturing order with a __dict__ will take some kind of extra work. >>I was thinking that if the above was an issue, we could write an API >>function(s) to "compile" the typerepetition tuple into arrays of ints >>which describe the type of each field and corresponding repetition factor. >> >> > >Yeah, I agree that this would be the best solution. That way, the charcodes >will be factored out from the code, and by just providing such and API (both >in Python and C), would be enough to reconstruct them, if needed. That will >allow a more consistent numarray internal code. > > I'm thinking the general format for this may be converting Ntuples of types and ints into N arrays of types and ints. And vice versa. It's obvious how this works with numarray types. I think the chararray types need work and need to be mapped into the same integer enumeration as the numeric types in a nonoverlapping way. >See you Monday, > > > >Right, how did you know that? :) > > Insightful on weekends anyway, Todd 
From: Francesc Alted <falted@op...>  20030127 12:01:15

A Dissabte 25 Gener 2003 20:30, Todd Miller va escriure: > > Still think I'd prefer something seperable: > > recarrStruct =3D ( (CharType, 16), > UInt8, > Int16, > Int32, > Int32, > Float32, > (Float64, 32), > Int64 ) > > recarrFields =3D ["name", > "TDCcount", > "ADCcount", > "grid_i", > "grid_j", > "pressure", > "temperature", > "idnumber"] > > I guess it might not be quite as good for large structs. Me too... > > It's a good sign that both you and I thought of an identical tuple > format; it's the obvious > minimal one. Yeah. We just differ in the way to arrange this metadata to be passed to = the recarray constructor. But I think this is secondary compared to the flexibility that a verbose approach offers compared with the actual strin= g format. In fact, more than one container might be supported to define the metadata; one can start with tuples as you suggest, but in the future oth= er ways can be added (if considered convenient). For example, I think I'll stick with the dictionary option for PyTables, = but also a class declaration for the metadata would be supported, like in : class Small(IsRecord): var1 =3D defineType(CharType, 2, "") var2 =3D defineType(Int32, 1) var3 =3D Float64 This would not be difficult to support because, by accessing to the Small().__dict__, you get also a dictionary. In addition, the latter will ensure (by construction) that you are not using a nonvalid python identifier, which is mandatory in my current implementation. I find these containers (dictionaries and classes) both elegant and convenient. > > Just like the type repetition tuple except also including field names > and default values. I don't think you lost me. For what we do, the > exact physical layout of the "struct" is important, so order matters. = I > see order as part of the > metadata, but I don't usually deal with metaentities so maybe I've > got that part wrong. :) > Well, if you need positional fields, you may add a (optional) parameter, called for example, "position" so that you can fix it.=20 > > I was thinking that if the above was an issue, we could write an API > function(s) to "compile" the typerepetition tuple into arrays of ints > which describe the type of each field and corresponding repetition fact= or. Yeah, I agree that this would be the best solution. That way, the charcod= es will be factored out from the code, and by just providing such and API (b= oth in Python and C), would be enough to reconstruct them, if needed. That wi= ll allow a more consistent numarray internal code.=20 > > See you Monday, Right, how did you know that? :) =20 Francesc Alted 
From: Joachim Saul <list@js...>  20030126 12:02:29

* baecker@... [26.01.2003 11:40]: > I just wondered if there is a "nicer" way of generating > a complex diagonal matrix than > a) > v=arange(10,typecode=Complex) > mat=diag(v) > b) > v=arange(10) > mat=diag(v)+0j > > Namely, wouldn't something like > v=arange(10) > mat=diag(v,typecode=Complex) > be nicer? No, because diag() is supposed to create a diagonal, but *not* to cast to another type. If you wanted to add that "functionality" to functions like diag(), you would also have to add it to functions like reshape() etc., i.e. practically everywhere. The way it is handled now is reasonably simple and flexible, and there is really no advantage of your suggestion compared to approach a). Cheers, Joachim 