numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] Py2exe / numpy troubles

[<kor...@id...>]

My Configuration :
  Windows XP pro,
  ActivePython 2.4.2.10,
  Scipy 0.4.9, Numpy 0.9.8,
  MatplotLib 0.87.1,
  Py2exe 0.6.5,
  WxPython 2.6
---- 1) Be very careful on how you generate the file
"...\dist\library.zip".I don't know why, but the zip file generated by
hand doesn't work. Check its size! Specific zip format? Specific options
to generate it? I didn't check source files to know how library.zip is
generated.My method is the following one:    - Extract the
...\test\dist\library.zip file in ...\test\dist\library    - Add the file
...\Python24\Lib\site-packages\numpy\core\_internal.pyc in
...\test\dist\library\numpy\core.    - Use Winzip to Add the
...\test\dist\library\numpy directory to the ...\dist\library.zip file
I know, it is not really beautiful but it seems to work. It is a temporary
solution for debug. I am new in Python so my style is not really
"academic"

---- 2) If you use my setup.py file, one more time, be careful cause of
the wx specific dll: wxmsw26uh_vc.dllI don't know why, but Py2Exe doesn't
find it. I remove that dll from the compilation phase and I copy it by
hand in ...\test\dist directory.An idea may be the modification of
setup.py file to indicate the path of that dll or something like that.DOES
ANYONE
HAVE THE SOLUTION?

---- 3)I am still blocked on my second issue
> >2) Second problem: I don't know how to resolve that issue:
> >
> >Traceback (most recent call last):> >  File "profiler_ftt.py", line 15,
in ?
> >    from matplotlib.backends.backend_wx import Toolbar, FigureCanvasWx,\
> >  File "matplotlib\backends\backend_wx.pyc", line 152, in ?
> >  File "matplotlib\backend_bases.pyc", line 10, in ?
> >  File "matplotlib\colors.pyc", line 33, in ?
> >  File "matplotlib\numerix\__init__.pyc", line 67, in ?
> >  File "numpy\__init__.pyc", line 35, in ?
> >  File "numpy\_import_tools.pyc", line 173, in __call__
> >  File "numpy\_import_tools.pyc", line 68, in _init_info_modules
> >  File "<string>", line 1, in ?
> >  File "numpy\random\__init__.pyc", line 3, in ?
> >  File "numpy\random\mtrand.pyc", line 12, in ?
> >  File "numpy\random\mtrand.pyc", line 10, in __load
> >  File "numpy.pxi", line 32, in mtrand
> >  AttributeError: 'module' object has no attribute 'dtype'
> >
> >I don't find the file numpy.pxi in my file tree nor in
   \test\dist\library.zip.
> >I browsed the web in the hope to find a solution but nothing.
> >It seems that this issue is well known but no solution provided in
   mailing lists.
> >
> >What is that file  "numpix.pxi"? Where to find it or how is it
   generated?
> >How to resolve that execution issue? Regards,Tristan

could you post your setup file please? i can look at it i may not be much
help but some is better than none

[Numpy-discussion] Py2exe / numpy troubles

[tristan C. <tco...@ya...>]

> >1) First Problem: numpy\core\_internal.pyc not included in Library.zip
> 
>C:\Lameness\dist>templatewindow.exe
> Traceback (most recent call last):
>   File "templatewindow.py", line 7, in ?
>   File "wxmpl.pyc", line 25, in ?
>   File "matplotlib\numerix\__init__.pyc", line 60, in ?
>   File "Numeric.pyc", line 91, in ?
>   File "numpy\__init__.pyc", line 35, in ?
>   File "numpy\core\__init__.pyc", line 6, in ?
>   File "numpy\core\umath.pyc", line 12, in ?
>   File "numpy\core\umath.pyc", line 10, in __load
> AttributeError: 'module' object has no attribute '_ARRAY_API'
> 
> 
> 
> >I resolved that issue by adding the file
> >...\Python24\Lib\site-packages\numpy\core\_internal.pyc in
> >...\test\dist\library.zip\numpy\core.
> >Each time I compile that executable, I add the file by hand.
> >Does anybody know how to automatically add that file?
> 
> although mine was in \python23 respectively :)
> 
> thanks for this fix
> now i have this problem
> 
> C:\Lameness\dist>templatewindow.exe
> Traceback (most recent call last):
>   File "c:\python23\lib\site-packages\py2exe\boot_common.py", line 92, in ?
>     import linecache
> ImportError: No module named linecache
> Traceback (most recent call last):
>   File "templatewindow.py", line 1, in ?
> ImportError: No module named wx
> 
> C:\Lameness\dist>
> 
> 
> current setup.py =
> 
> ########################################################
> from distutils.filelist import findall
> import os
> import matplotlib
> matplotlibdatadir = matplotlib.get_data_path()
> matplotlibdata = findall(matplotlibdatadir)
> matplotlibdata_files = []
> for f in matplotlibdata:
>     dirname = os.path.join('matplotlibdata', f[len(matplotlibdatadir)+1:])
>     matplotlibdata_files.append((os.path.split(dirname)[0], [f]))
> 
> 
> packages = ['matplotlib', 'pytz']
> includes = []
> excludes = []
> dll_excludes = ['libgdk_pixbuf-2.0-0.dll',
>                 'libgobject-2.0-0.dll',
>                 'libgdk-win32-2.0-0.dll',
>                 'wxmsw26uh_vc.dll']
> 
> 
> opts = { 'py2exe': { 'packages' : packages,
>                      'includes' : includes,
>                      'excludes' : excludes,
>                      'dll_excludes' : dll_excludes
>                     }
>          }
> 
> setup ( console=['templatewindow.py'],
>         options = opts,
>         data_files = matplotlibdata_files
>         )
> ##########################################################
> 
> anyone seen this problem before?
> 
> first line of template window = import wx
>
My Configuration : Windows XP pro, ActivePython 2.4.2.10, Scipy 0.4.9, Numpy 0.9.8, MatplotLib 0.87.1, Py2exe 0.6.5, WxPython 2.6 ---- 1) Be very careful on how you generate the file "...\dist\library.zip".I don't know why, but the zip file generated by hand doesn't work. Check its size! Specific zip format? Specific options to generate it? I didn't check source files to know how library.zip is generated.My method is the following one:    - Extract the ...\test\dist\library.zip file in ...\test\dist\library    - Add the file ...\Python24\Lib\site-packages\numpy\core\_internal.pyc in ...\test\dist\library\numpy\core.    - Use Winzip to Add the ...\test\dist\library\numpy directory to the ...\dist\library.zip fileI know, it is not really beautiful but it seems to work. It is a temporary solution for debug. I am new in Python so my style is not really "academic" ---- 2) If you use my setup.py file, one more time, be careful cause of the wx specific dll: wxmsw26uh_vc.dllI don't
 know why, but Py2Exe doesn't find it. I remove that dll from the compilation phase and I copy it by hand in ...\test\dist directory.An idea may be the modification of setup.py file to indicate the path of that dll or something like that.DOES ANYONE HAVE THE SOLUTION?  ---- 3)I am still blocked on my second issue > >2) Second problem: I don't know how to resolve that issue:> > > >Traceback (most recent call last):> >  File "profiler_ftt.py", line 15, in ?> >    from matplotlib.backends.backend_wx import Toolbar, FigureCanvasWx,\> >  File "matplotlib\backends\backend_wx.pyc", line 152, in ?> >  File "matplotlib\backend_bases.pyc", line 10, in ?> >  File "matplotlib\colors.pyc", line 33, in ?> >  File "matplotlib\numerix\__init__.pyc", line 67, in ?> >  File "numpy\__init__.pyc", line 35, in ?> >  File "numpy\_import_tools.pyc", line 173, in __call__> >  File "numpy\_import_tools.pyc", line 68, in _init_info_modules> >  File "<string>", line 1, in ?> >  File
 "numpy\random\__init__.pyc", line 3, in ?> >  File "numpy\random\mtrand.pyc", line 12, in ?> >  File "numpy\random\mtrand.pyc", line 10, in __load> >  File "numpy.pxi", line 32, in mtrand> >AttributeError: 'module' object has no attribute 'dtype'> > > >I don't find the file numpy.pxi in my file tree nor in \test\dist\library.zip.> >I browsed the web in the hope to find a solution but nothing.> >It seems that this issue is well known but no solution provided in mailing lists.> > > >What is that file  "numpix.pxi"? Where to find it or how is it generated?> >How to resolve that execution issue? Regards,Tristan

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Tim H. <tim...@ie...>]

David M. Cooke wrote:
> On Tue, 29 Aug 2006 14:03:39 -0700
> Tim Hochberg <tim...@ie...> wrote:
>
>   
>> Of these,  clip, conjugate and round support an 'out' argument like that 
>> supported by ufunces;  byteswap has a boolean argument telling it 
>> whether to perform operations in place; and sort always operates in 
>> place. Noting that the ufunc-like methods (max, argmax, etc) appear to 
>> support the 'out' argument as well although it's not documented for most 
>> of them, it looks to me as if the two odd methods are byteswap and sort. 
>> The method situation could be made more consistent by swapping the 
>> boolean inplace flag in byteswapped with another 'out' argument and also 
>> having sort not operate in place by default, but also supply an out 
>> argument there. Thus:
>>
>> b = a.sort()   # Returns a copy
>> a.sort(out=a) # Sorts a in place
>> a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort() 
>> in this case since we don't want to rewrite the sort routines)
>>     
>
> Ugh. That's completely different semantics from sort() on lists, so I think
> it would be a source of bugs (at least, it would mean keeping two different
> ideas of .sort() in my head).
>   
Thinking about it a bit more, I'd leave sort alone (returning None and 
all).. I was (over)reacting to changing to sort to return self, which 
makes the set of methods both less consistent within itself, less 
consistent with python and more error prone IMO, which seems the worst 
possibility.

For the moment at least I do stand by the suggestion of changing 
byteswap to match the rest of the methods, as that would remove one 
outlier in the set methods.

-tim

[Numpy-discussion] fromiter shape argument -- was Re: For loop tips

[Torgil S. <tor...@gm...>]

>    return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))

Is it possible for fromiter to take an optional shape (or count)
argument in addition to the dtype argument? If both is given it could
preallocate memory and we only have to iterate over L once.

//Torgil

On 8/29/06, Keith Goodman <kwg...@gm...> wrote:
> On 8/29/06, Torgil Svensson <tor...@gm...> wrote:
> > something like this?
> >
> > def list2index(L):
> >    uL=sorted(set(L))
> >    idx=dict((y,x) for x,y in enumerate(uL))
> >    return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))
>
> Wow. That's amazing. Thank you.
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Alan G I. <ai...@am...>]

On Tue, 29 Aug 2006, Tim Hochberg apparently wrote:=20
> b =3D a.sort()   # Returns a copy=20

Given the extant Python vocabulary,
this seems like a bad idea to me.
(Better to call it 'sorted' in this case.)

fwiw,
Alan Isaac

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Charles R H. <cha...@gm...>]

Hi,

On 8/29/06, kor...@id... <kor...@id...> wrote:
>
> >I find it much cleaner to write
>
> >x = foo.bar().baz(param).frob()
>
> >than
>
> >foo.bar()
> >foo.baz(param)
> >x = foo.frob()
>
> >but perhaps others disagree.
>
> Both of these look "clean" but i do not think that moving 3 lines to one
> line makes code "cleaner"  They both do the same thing and if someone that
> does not know what .bar() .baz(param) and .frob() are IMO the second
> version that takes place on three lines would be easier to understand.
>
>
>
> >I'm generally +0 on this idea (it seems like the clarity in writing
> >comes largely for interactive users), and don't see much difficulty in
> >separating the constructs.   On the other hand, I don't see much problem
> >in returning a reference to self either.
>
> >I guess you are worried about the situation where you write
>
> >b = a.sort()
>
> >and think you have a new array, but in fact have a new reference to the
> >already-altered 'a'?
>
> >Hmm..  So, how is this different from the fact that
>
> >b = a[1:10:3] already returns a reference to 'a'
>
> >(I suppose in the fact that it actually returns a new object just one
> >that happens to share the same data with a).
>
> >However, I suppose that other methods don't return a reference to an
> >already-altered object, do they.
>
> >Tim's argument has moved me from +0 to -0
>
> >-Travis
>
>
> I couldn't agree more with you and Tim on this.  I would rather have code
> that works all the time and will not possibly confuse people later, like
> the example of
>
> >b = a.sort()
> >and think you have a new array, but in fact have a new reference to the
> >already-altered 'a'?
>
> alot of people have problems grasping this "memory management" type of
> programming...or at least in my C class half of the kids dropped out
> because the couldnt keep track of
>
> b = a.sort() meaning that b was actually just referencing a and if you
> changed b then a was changed also.


Maybe they should start with assembly (or mix ;) instead of C? In any case,
references are  pointer wrappers and pointers seem to be the biggest bugaboo
in C. Maybe everyone should start with Fortran where most everything was a
reference. I say "was" because the last Fortran I used was F77 and I have no
idea what the current situation is. I suppose the in/out specs make a
difference.

But then again who on this list has problems remembering things like that
> anyways right?...
> ~Kenny


Chuck

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Fernando P. <fpe...@gm...>]

On 8/29/06, David M. Cooke <co...@ph...> wrote:
> On Tue, 29 Aug 2006 14:03:39 -0700
> Tim Hochberg <tim...@ie...> wrote:

> > b = a.sort()   # Returns a copy
> > a.sort(out=a) # Sorts a in place
> > a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort()
> > in this case since we don't want to rewrite the sort routines)
>
> Ugh. That's completely different semantics from sort() on lists, so I think
> it would be a source of bugs (at least, it would mean keeping two different
> ideas of .sort() in my head).

Agreed.  Except where very well justified (such as slicing returning
views for memory reasons), let's keep numpy arrays similar to native
lists in their behavior...

Special cases aren't special enough to break the rules.

and all that :)

Cheers,

f

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[David M. C. <co...@ph...>]

On Tue, 29 Aug 2006 13:25:14 -0600
"Charles R Harris" <cha...@gm...> wrote:

> Hi Fernando,
> 
> On 8/29/06, Fernando Perez <fpe...@gm...> wrote:
> >
> > On 8/29/06, Charles R Harris <cha...@gm...> wrote:
> >
> > > Speaking of features, I wonder if more of the methods should return
> > > references. For instance, it might be nice to write something like:
> > >
> > >  a.sort().searchsorted([...])
> > >
> > > instead of making two statements out of it.
> >
> > +1 for more 'return self' at the end of methods which currently don't
> > return anything (well, we get the default None), as long as it's
> > sensible.  I really like this 'message chaining' style of programming,
> > and it annoys me that much of the python stdlib gratuitously prevents
> > it by NOT returning self in places where it would be a perfectly
> > sensible thing to do.

-1, for the same reasons l.sort() doesn't (for a list l). For lists, the
reason .sort() returns None is because it makes it clear it's a mutation.
Returning self would make it look like it was doing a copy.

> My pet peeve example: a.reverse()
> 
> I would also like to see simple methods for "+=" operator and such. Then one
> could write
> 
> x = a.copy().add(10)

There are:
x = a.copy().__add__(10)

or, for +=:
x.__iadd__(10)

> One could make a whole reverse polish translator out of such operations and
> a few parenthesis. I have in mind some sort of code optimizer.

It wouldn't be anymore efficient than the other way. For a code optimizer,
you'll either have to parse the python code or use special objects (much like
numexpr does), and then you might as well use the operators.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Charles R H. <cha...@gm...>]

Hi Tim,

On 8/29/06, Tim Hochberg <tim...@ie...> wrote:
>
> Charles R Harris wrote:
> >
> >
> > On 8/29/06, *Tim Hochberg* <tim...@ie...
> > <mailto:tim...@ie...>> wrote:
> >
> >     Charles R Harris wrote:
> >     > Hi,
> >     >
> >     > On 8/29/06, *Tim Hochberg* <tim...@ie...
> >     <mailto:tim...@ie...>
> >     > <mailto:tim...@ie... <mailto:tim...@ie...>>>
> >     wrote:
> >     >
> >     >
> >     >     -0.5 from me if what we're talking about here is having
> mutating
> >     >     methods
> >     >     return self rather than None. Chaining stuff is pretty, but
> >     having
> >     >     methods that mutate self and return self looks like a source
> of
> >     >     elusive
> >     >     bugs to me.
> >     >
> >     >     -tim
> >     >
> >     >
> >     > But how is that any worse than the current mutating operators? I
> >     think
> >     > the operating principal is that methods generally work in place,
> >     > functions make copies. The exceptions to this rule need to be
> noted.
> >     Is that really the case? I was more under the impression that there
> >     wasn't much rhyme nor reason to this. Let's do a quick
> dir(somearray)
> >     and see what we get (I'll strip out the __XXX__ names):
> >
> >     'all', 'any', 'argmax', 'argmin', 'argsort', 'astype', 'base',
> >     'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate',
> 'copy',
> >     'ctypes', 'cumprod', 'cumsum', 'data', 'diagonal', 'dtype', 'dump',
> >     'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag',
> >     'item',
> >     'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder',
> >     'nonzero', 'prod', 'ptp', 'put', 'putmask', 'ravel', 'real',
> >     'repeat',
> >     'reshape', 'resize', 'round', 'searchsorted', 'setfield',
> 'setflags',
> >     'shape', 'size', 'sort', 'squeeze', 'std', 'strides', 'sum',
> >     'swapaxes',
> >     'take', 'tofile', 'tolist', 'tostring', 'trace', 'transpose',
> >     'var', 'view'
> >
> >
> > There are certainly many methods where inplace operations make no
> > sense. But for such things as conjugate and clip I think it should be
> > preferred. Think of them as analogs of the "+=" operators that allow
> > memory efficient inplace operations. At the moment there are too few
> > such operators, IMHO, and that makes it hard to write memory efficient
> > code when you want to do so. If you need a copy, the functional form
> > should be the preferred way to go and can easily be implement by
> > constructions like a.copy().sort().
> So let's make this clear; what you are proposing is more that just
> returning self for more operations. You are proposing changing the
> meaning of the existing methods to operate in place rather than return
> new objects. It seems awfully late in the day to be considering this
> being that we're on the edge of 1.0 and this would could break any
> existing numpy code that is out there.
>
> Just for grins let's look at the operations that could potentially
> benefit from being done in place. I think they are:
>    byteswap
>    clip
>    conjugate
>    round
>    sort
>
> Of these,  clip, conjugate and round support an 'out' argument like that
> supported by ufunces;  byteswap has a boolean argument telling it
> whether to perform operations in place; and sort always operates in
> place. Noting that the ufunc-like methods (max, argmax, etc) appear to
> support the 'out' argument as well although it's not documented for most
> of them, it looks to me as if the two odd methods are byteswap and sort.
> The method situation could be made more consistent by swapping the
> boolean inplace flag in byteswapped with another 'out' argument and also
> having sort not operate in place by default, but also supply an out
> argument there. Thus:
>
> b = a.sort()   # Returns a copy
> a.sort(out=a) # Sorts a in place
> a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort()
> in this case since we don't want to rewrite the sort routines)
>
> On the whole I think that this would be an improvement, but it may be
> too late in the day to actually implement it since 1.0 is coming up.
> There would still be a few methods (fill, put, etc) that modify the
> array in place and return None, but I haven't heard any complaints about
> those.


That sounds like a good idea. One could keep the present behaviour in most
cases by supplying a default value, although the out keyword might need a
None value to indicate "copy" and a 'Self' value that means in place, or
something like that, and then have all reasonable methods return values.
That way the change would be transparent. The changes to the sort method
would all be upper level, the low level sorting routines would remain
unchanged.

Methods are new, so code that needs to be changed is code specifically
written for Numpy and now is the time to make these sort of decisions.

-tim


Chuck

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[David M. C. <co...@ph...>]

On Tue, 29 Aug 2006 14:03:39 -0700
Tim Hochberg <tim...@ie...> wrote:

> Of these,  clip, conjugate and round support an 'out' argument like that 
> supported by ufunces;  byteswap has a boolean argument telling it 
> whether to perform operations in place; and sort always operates in 
> place. Noting that the ufunc-like methods (max, argmax, etc) appear to 
> support the 'out' argument as well although it's not documented for most 
> of them, it looks to me as if the two odd methods are byteswap and sort. 
> The method situation could be made more consistent by swapping the 
> boolean inplace flag in byteswapped with another 'out' argument and also 
> having sort not operate in place by default, but also supply an out 
> argument there. Thus:
> 
> b = a.sort()   # Returns a copy
> a.sort(out=a) # Sorts a in place
> a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort() 
> in this case since we don't want to rewrite the sort routines)

Ugh. That's completely different semantics from sort() on lists, so I think
it would be a source of bugs (at least, it would mean keeping two different
ideas of .sort() in my head).

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[<kor...@id...>]

>I find it much cleaner to write

>x = foo.bar().baz(param).frob()

>than

>foo.bar()
>foo.baz(param)
>x = foo.frob()

>but perhaps others disagree.

Both of these look "clean" but i do not think that moving 3 lines to one
line makes code "cleaner"  They both do the same thing and if someone that
does not know what .bar() .baz(param) and .frob() are IMO the second
version that takes place on three lines would be easier to understand.



>I'm generally +0 on this idea (it seems like the clarity in writing
>comes largely for interactive users), and don't see much difficulty in
>separating the constructs.   On the other hand, I don't see much problem
>in returning a reference to self either.

>I guess you are worried about the situation where you write

>b = a.sort()

>and think you have a new array, but in fact have a new reference to the
>already-altered 'a'?

>Hmm..  So, how is this different from the fact that

>b = a[1:10:3] already returns a reference to 'a'

>(I suppose in the fact that it actually returns a new object just one
>that happens to share the same data with a).

>However, I suppose that other methods don't return a reference to an
>already-altered object, do they.

>Tim's argument has moved me from +0 to -0

>-Travis


I couldn't agree more with you and Tim on this.  I would rather have code
that works all the time and will not possibly confuse people later, like
the example of

>b = a.sort()
>and think you have a new array, but in fact have a new reference to the
>already-altered 'a'?

alot of people have problems grasping this "memory management" type of
programming...or at least in my C class half of the kids dropped out
because the couldnt keep track of

b = a.sort() meaning that b was actually just referencing a and if you
changed b then a was changed also.
But then again who on this list has problems remembering things like that
anyways right?...
~Kenny

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Tim H. <tim...@ie...>]

Charles R Harris wrote:
>
>
> On 8/29/06, *Tim Hochberg* <tim...@ie... 
> <mailto:tim...@ie...>> wrote:
>
>     Charles R Harris wrote:
>     > Hi,
>     >
>     > On 8/29/06, *Tim Hochberg* <tim...@ie...
>     <mailto:tim...@ie...>
>     > <mailto:tim...@ie... <mailto:tim...@ie...>>>
>     wrote:
>     >
>     >
>     >     -0.5 from me if what we're talking about here is having mutating
>     >     methods
>     >     return self rather than None. Chaining stuff is pretty, but
>     having
>     >     methods that mutate self and return self looks like a source of
>     >     elusive
>     >     bugs to me.
>     >
>     >     -tim
>     >
>     >
>     > But how is that any worse than the current mutating operators? I
>     think
>     > the operating principal is that methods generally work in place,
>     > functions make copies. The exceptions to this rule need to be noted.
>     Is that really the case? I was more under the impression that there
>     wasn't much rhyme nor reason to this. Let's do a quick dir(somearray)
>     and see what we get (I'll strip out the __XXX__ names):
>
>     'all', 'any', 'argmax', 'argmin', 'argsort', 'astype', 'base',
>     'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate', 'copy',
>     'ctypes', 'cumprod', 'cumsum', 'data', 'diagonal', 'dtype', 'dump',
>     'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag',
>     'item',
>     'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder',
>     'nonzero', 'prod', 'ptp', 'put', 'putmask', 'ravel', 'real',
>     'repeat',
>     'reshape', 'resize', 'round', 'searchsorted', 'setfield', 'setflags',
>     'shape', 'size', 'sort', 'squeeze', 'std', 'strides', 'sum',
>     'swapaxes',
>     'take', 'tofile', 'tolist', 'tostring', 'trace', 'transpose',
>     'var', 'view' 
>
>
> There are certainly many methods where inplace operations make no 
> sense. But for such things as conjugate and clip I think it should be 
> preferred. Think of them as analogs of the "+=" operators that allow 
> memory efficient inplace operations. At the moment there are too few 
> such operators, IMHO, and that makes it hard to write memory efficient 
> code when you want to do so. If you need a copy, the functional form 
> should be the preferred way to go and can easily be implement by 
> constructions like a.copy().sort().
So let's make this clear; what you are proposing is more that just 
returning self for more operations. You are proposing changing the 
meaning of the existing methods to operate in place rather than return 
new objects. It seems awfully late in the day to be considering this 
being that we're on the edge of 1.0 and this would could break any 
existing numpy code that is out there.

Just for grins let's look at the operations that could potentially 
benefit from being done in place. I think they are:
   byteswap
   clip
   conjugate
   round
   sort
   
Of these,  clip, conjugate and round support an 'out' argument like that 
supported by ufunces;  byteswap has a boolean argument telling it 
whether to perform operations in place; and sort always operates in 
place. Noting that the ufunc-like methods (max, argmax, etc) appear to 
support the 'out' argument as well although it's not documented for most 
of them, it looks to me as if the two odd methods are byteswap and sort. 
The method situation could be made more consistent by swapping the 
boolean inplace flag in byteswapped with another 'out' argument and also 
having sort not operate in place by default, but also supply an out 
argument there. Thus:

b = a.sort()   # Returns a copy
a.sort(out=a) # Sorts a in place
a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort() 
in this case since we don't want to rewrite the sort routines)

On the whole I think that this would be an improvement, but it may be 
too late in the day to actually implement it since 1.0 is coming up. 
There would still be a few methods (fill, put, etc) that modify the 
array in place and return None, but I haven't heard any complaints about 
those.


-tim

>
>     Hmmm. Without taking too much time to go through these one at a time,
>     I'm pretty certain that they do not in general mutate things in place.
>     Probably at least half return, or can return new arrays, sometimes
>     with
>     references to the original data, but new shapes, sometimes with
>     completely new data. In fact, other than sort, I'm not sure which of
>     these does mutate in place.
>
>     -tim
>
>
> Chuck
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> ------------------------------------------------------------------------
>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>   

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Christopher B. <Chr...@no...>]

Fernando Perez wrote:
> more 'return self' at the end of methods which currently don't
> return anything (well, we get the default None), as long as it's
> sensible. 

+1

Though I'm a bit hesitant: if it's really consistent that methods that 
alter the object in place NEVER return themselves, the there is 
something to be said for that.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Travis O. <oli...@ee...>]

Tim Hochberg wrote:

>-0.5 from me if what we're talking about here is having mutating methods 
>return self rather than None. Chaining stuff is pretty, but having 
>methods that mutate self and return self looks like a source of elusive 
>bugs to me.
>  
>

I'm generally +0 on this idea (it seems like the clarity in writing 
comes largely for interactive users), and don't see much difficulty in 
separating the constructs.   On the other hand, I don't see much problem 
in returning a reference to self either.  

I guess you are worried about the situation where you write

b = a.sort()

and think you have a new array, but in fact have a new reference to the 
already-altered 'a'?

Hmm..  So, how is this different from the fact that

b = a[1:10:3] already returns a reference to 'a'

(I suppose in the fact that it actually returns a new object just one 
that happens to share the same data with a).

However, I suppose that other methods don't return a reference to an 
already-altered object, do they. 

Tim's argument has moved me from +0 to -0

-Travis

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Travis O. <oli...@ee...>]

Charles R Harris wrote:

>
>     The 1.0rc1 release of NumPy will be mid September I suspect.
>
>     Also, I recognize that the default-axis switch is a burden for
>     those who
>     have already transitioned code to use NumPy (for those just
>     starting out
>     it's not a big deal because of the compatibility layer). 
>
>
> I am curious as to why you made this switch. Not complaining, mind.


New-comers to NumPy asked why there were different conventions on the 
methods and the functions for the axis argument.  The only reason was 
backward compatibility.  Because we had already created a compatibility 
layer for code transitioning, that seemed like a weak reason to keep the 
current behavior.

The problem is it left early NumPy adopters (including me :-) ) in a bit 
of a bind, when it comes to code (like SciPy) that had already been 
converted.  

Arguments like Fernando's: "it's better to have a bit of pain now, then 
regrets later" also were convincing.

-Travis

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Charles R H. <cha...@gm...>]

On 8/29/06, Tim Hochberg <tim...@ie...> wrote:
>
> Charles R Harris wrote:
> > Hi,
> >
> > On 8/29/06, *Tim Hochberg* <tim...@ie...
> > <mailto:tim...@ie...>> wrote:
> >
> >
> >     -0.5 from me if what we're talking about here is having mutating
> >     methods
> >     return self rather than None. Chaining stuff is pretty, but having
> >     methods that mutate self and return self looks like a source of
> >     elusive
> >     bugs to me.
> >
> >     -tim
> >
> >
> > But how is that any worse than the current mutating operators? I think
> > the operating principal is that methods generally work in place,
> > functions make copies. The exceptions to this rule need to be noted.
> Is that really the case? I was more under the impression that there
> wasn't much rhyme nor reason to this. Let's do a quick dir(somearray)
> and see what we get (I'll strip out the __XXX__ names):
>
> 'all', 'any', 'argmax', 'argmin', 'argsort', 'astype', 'base',
> 'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate', 'copy',
> 'ctypes', 'cumprod', 'cumsum', 'data', 'diagonal', 'dtype', 'dump',
> 'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag', 'item',
> 'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder',
> 'nonzero', 'prod', 'ptp', 'put', 'putmask', 'ravel', 'real', 'repeat',
> 'reshape', 'resize', 'round', 'searchsorted', 'setfield', 'setflags',
> 'shape', 'size', 'sort', 'squeeze', 'std', 'strides', 'sum', 'swapaxes',
> 'take', 'tofile', 'tolist', 'tostring', 'trace', 'transpose', 'var',
> 'view'


There are certainly many methods where inplace operations make no sense. But
for such things as conjugate and clip I think it should be preferred. Think
of them as analogs of the "+=" operators that allow memory efficient inplace
operations. At the moment there are too few such operators, IMHO, and that
makes it hard to write memory efficient code when you want to do so. If you
need a copy, the functional form should be the preferred way to go and can
easily be implement by constructions like a.copy().sort().

Hmmm. Without taking too much time to go through these one at a time,
> I'm pretty certain that they do not in general mutate things in place.
> Probably at least half return, or can return new arrays, sometimes with
> references to the original data, but new shapes, sometimes with
> completely new data. In fact, other than sort, I'm not sure which of
> these does mutate in place.
>
> -tim


Chuck

Re: [Numpy-discussion] For loop tips

[Keith G. <kwg...@gm...>]

On 8/29/06, Torgil Svensson <tor...@gm...> wrote:
> something like this?
>
> def list2index(L):
>    uL=sorted(set(L))
>    idx=dict((y,x) for x,y in enumerate(uL))
>    return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))

Wow. That's amazing. Thank you.

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Tim H. <tim...@ie...>]

Charles R Harris wrote:
> Hi,
>
> On 8/29/06, *Tim Hochberg* <tim...@ie... 
> <mailto:tim...@ie...>> wrote:
>
>
>     -0.5 from me if what we're talking about here is having mutating
>     methods
>     return self rather than None. Chaining stuff is pretty, but having
>     methods that mutate self and return self looks like a source of
>     elusive
>     bugs to me.
>
>     -tim
>
>
> But how is that any worse than the current mutating operators? I think 
> the operating principal is that methods generally work in place, 
> functions make copies. The exceptions to this rule need to be noted.
Is that really the case? I was more under the impression that there 
wasn't much rhyme nor reason to this. Let's do a quick dir(somearray) 
and see what we get (I'll strip out the __XXX__ names):

'all', 'any', 'argmax', 'argmin', 'argsort', 'astype', 'base', 
'byteswap', 'choose', 'clip', 'compress', 'conj', 'conjugate', 'copy', 
'ctypes', 'cumprod', 'cumsum', 'data', 'diagonal', 'dtype', 'dump', 
'dumps', 'fill', 'flags', 'flat', 'flatten', 'getfield', 'imag', 'item', 
'itemsize', 'max', 'mean', 'min', 'nbytes', 'ndim', 'newbyteorder', 
'nonzero', 'prod', 'ptp', 'put', 'putmask', 'ravel', 'real', 'repeat', 
'reshape', 'resize', 'round', 'searchsorted', 'setfield', 'setflags', 
'shape', 'size', 'sort', 'squeeze', 'std', 'strides', 'sum', 'swapaxes', 
'take', 'tofile', 'tolist', 'tostring', 'trace', 'transpose', 'var', 'view'

Hmmm. Without taking too much time to go through these one at a time, 
I'm pretty certain that they do not in general mutate things in place. 
Probably at least half return, or can return new arrays, sometimes with 
references to the original data, but new shapes, sometimes with 
completely new data. In fact, other than sort, I'm not sure which of 
these does mutate in place.

-tim

Re: [Numpy-discussion] For loop tips

[Torgil S. <tor...@gm...>]

something like this?

def list2index(L):
   uL=sorted(set(L))
   idx=dict((y,x) for x,y in enumerate(uL))
   return uL,asmatrix(fromiter((idx[x] for x in L),dtype=int))

//Torgil

On 8/29/06, Keith Goodman <kwg...@gm...> wrote:
> On 8/29/06, Tim Hochberg <tim...@ie...> wrote:
> > Keith Goodman wrote:
> > > I have a very long list that contains many repeated elements. The
> > > elements of the list can be either all numbers, or all strings, or all
> > > dates [datetime.date].
> > >
> > > I want to convert the list into a matrix where each unique element of
> > > the list is assigned a consecutive integer starting from zero.
> > >
> > If what you want is that the first unique element get's zero, the second
> > one, I don't think the code below will work in general since the dict
> > does not preserve order. You might want to look at the results for the
> > character case to see what I mean. If you're looking for something else,
> > you'll need to elaborate a bit. Since list2index doesn't return
> > anything, it's not entirely clear what the answer consists of. Just idx?
> > Idx plus uL?
>
> The output I wanted (in my mind, but unfortunately not in my previous
> email) is idx and uL where uL[0] corresponds to the zeros in idx,
> uL[1] corresponds to the ones in idx. etc.
>
> I'd also like the uL's to be ordered (now I see that characters and
> dates aren't ordered, ooops, thanks for telling me about that). Or
> optionally ordered by a second list input which if present would be
> used instead of the unique values of L.
>
> Thank you all for the huge improvements to my code. I'll learn a lot
> studying all of them.
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Alan G I. <ai...@am...>]

On Tue, 29 Aug 2006, Tim Hochberg apparently wrote:=20
> -0.5 from me if what we're talking about here is having=20
> mutating methods return self rather than None. Chaining=20
> stuff is pretty, but having methods that mutate self and=20
> return self looks like a source of elusive bugs to me.=20

I believe this reasoning was the basis of sort (method,=20
returns None) and sorted (function, returns new object)
in Python. =20

I believe that was a long and divisive
discussion ...

Cheers,
Alan Isaac

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Charles R H. <cha...@gm...>]

Hi,

On 8/29/06, Tim Hochberg <tim...@ie...> wrote:
>
>
> -0.5 from me if what we're talking about here is having mutating methods
> return self rather than None. Chaining stuff is pretty, but having
> methods that mutate self and return self looks like a source of elusive
> bugs to me.
>
> -tim


But how is that any worse than the current mutating operators? I think the
operating principal is that methods generally work in place, functions make
copies. The exceptions to this rule need to be noted.

Chuck

Re: [Numpy-discussion] For loop tips

[Keith G. <kwg...@gm...>]

On 8/29/06, Tim Hochberg <tim...@ie...> wrote:
> Keith Goodman wrote:
> > I have a very long list that contains many repeated elements. The
> > elements of the list can be either all numbers, or all strings, or all
> > dates [datetime.date].
> >
> > I want to convert the list into a matrix where each unique element of
> > the list is assigned a consecutive integer starting from zero.
> >
> If what you want is that the first unique element get's zero, the second
> one, I don't think the code below will work in general since the dict
> does not preserve order. You might want to look at the results for the
> character case to see what I mean. If you're looking for something else,
> you'll need to elaborate a bit. Since list2index doesn't return
> anything, it's not entirely clear what the answer consists of. Just idx?
> Idx plus uL?

The output I wanted (in my mind, but unfortunately not in my previous
email) is idx and uL where uL[0] corresponds to the zeros in idx,
uL[1] corresponds to the ones in idx. etc.

I'd also like the uL's to be ordered (now I see that characters and
dates aren't ordered, ooops, thanks for telling me about that). Or
optionally ordered by a second list input which if present would be
used instead of the unique values of L.

Thank you all for the huge improvements to my code. I'll learn a lot
studying all of them.

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Tim H. <tim...@ie...>]

-0.5 from me if what we're talking about here is having mutating methods 
return self rather than None. Chaining stuff is pretty, but having 
methods that mutate self and return self looks like a source of elusive 
bugs to me.

-tim


Rudolph van der Merwe wrote:
> This definitely gets my vote as well (for what it's worth).
>
> R.
>
> On 8/29/06, Fernando Perez <fpe...@gm...> wrote:
>   
>> +1 for more 'return self' at the end of methods which currently don't
>> return anything (well, we get the default None), as long as it's
>> sensible.  I really like this 'message chaining' style of programming,
>> and it annoys me that much of the python stdlib gratuitously prevents
>> it by NOT returning self in places where it would be a perfectly
>> sensible thing to do.
>>
>> I find it much cleaner to write
>>
>> x = foo.bar().baz(param).frob()
>>
>> than
>>
>> foo.bar()
>> foo.baz(param)
>> x = foo.frob()
>>
>> but perhaps others disagree.
>>
>> Cheers,
>>
>> f
>>     
>
>   

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Charles R H. <cha...@gm...>]

Hi Fernando,

On 8/29/06, Fernando Perez <fpe...@gm...> wrote:
>
> On 8/29/06, Charles R Harris <cha...@gm...> wrote:
>
> > Speaking of features, I wonder if more of the methods should return
> > references. For instance, it might be nice to write something like:
> >
> >  a.sort().searchsorted([...])
> >
> > instead of making two statements out of it.
>
> +1 for more 'return self' at the end of methods which currently don't
> return anything (well, we get the default None), as long as it's
> sensible.  I really like this 'message chaining' style of programming,
> and it annoys me that much of the python stdlib gratuitously prevents
> it by NOT returning self in places where it would be a perfectly
> sensible thing to do.


My pet peeve example: a.reverse()

I would also like to see simple methods for "+=" operator and such. Then one
could write

x = a.copy().add(10)

One could make a whole reverse polish translator out of such operations and
a few parenthesis. I have in mind some sort of code optimizer.

Chuck

Re: [Numpy-discussion] Release of 1.0b5 this weekend

[Rudolph v. d. M. <rud...@gm...>]

This definitely gets my vote as well (for what it's worth).

R.

On 8/29/06, Fernando Perez <fpe...@gm...> wrote:
> +1 for more 'return self' at the end of methods which currently don't
> return anything (well, we get the default None), as long as it's
> sensible.  I really like this 'message chaining' style of programming,
> and it annoys me that much of the python stdlib gratuitously prevents
> it by NOT returning self in places where it would be a perfectly
> sensible thing to do.
>
> I find it much cleaner to write
>
> x = foo.bar().baz(param).frob()
>
> than
>
> foo.bar()
> foo.baz(param)
> x = foo.frob()
>
> but perhaps others disagree.
>
> Cheers,
>
> f

-- 
Rudolph van der Merwe
Karoo Array Telescope / Square Kilometer Array - http://www.ska.ac.za