numpy-discussion

[Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Travis E. O. <oli...@ie...>]

The 4th beta release of NumPy 1.0 has just been made available.

NumPy 1.0 represents the culmination of over 18 months of work to unify 
the Numeric and Numarray array packages into a single best-of-breed 
array package for Python.

NumPy supports all the features of Numeric and Numarray with a healthy 
dose of it's own improved features.

It's time to start porting your applications to use NumPy as Numeric is 
no longer maintained and Numarray will only be maintained for a few more 
months.

Porting is not difficult especially using the compatibility layers 
numpy.oldnumeric and numpy.numarray and the alter_code1.py modules in 
those packages.  The full C-API of Numeric is supported as is the C-API 
of Numarray.

More information is available at  http://numpy.scipy.org


NumPy Developers

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Les S. <sch...@op...>]

Travis E. Oliphant wrote:
> Porting is not difficult especially using the compatibility layers 
> numpy.oldnumeric and numpy.numarray and the alter_code1.py modules in 
> those packages.  The full C-API of Numeric is supported as is the C-API 
> of Numarray.
>   

this is not true of numpy.core.records  (nee numarray.records):

1. numarray's records.py does not show up in numpy.numarray.

2. my code that uses recarrays is now broken if i use
numpy.core.records; for one thing, you have no .info attribute. another
example: strings pushed into the arrays *apparently* were stripped
automagically in the old recarray (so we coded appropriately), but now
are not.  

3. near zero docstrings for this module, hard to see how the new
records  works.

4. last year i made a case for the old records to return a list of the
column names. it looks like the column names are now attributes of the
record object, any chance of getting a list of them
recarrayObj.get_colNames() or some such? yes, in working code, we know
what the names are, but in test code we are creating recarrays from
parsing of Excel spreadsheets, and for testing purposes, its nice to
know what records THINKS are the names of all the columns.

Les Schaffer

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Robert K. <rob...@gm...>]

Les Schaffer wrote:

> 4. last year i made a case for the old records to return a list of the
> column names. it looks like the column names are now attributes of the
> record object, any chance of getting a list of them
> recarrayObj.get_colNames() or some such? yes, in working code, we know
> what the names are, but in test code we are creating recarrays from
> parsing of Excel spreadsheets, and for testing purposes, its nice to
> know what records THINKS are the names of all the columns.

In [2]: from numpy import *

In [3]: rec.fromarrays(ones(10, dtype=float)
Display all 628 possibilities? (y or n)

In [3]: a = rec.fromarrays([ones(10, dtype=float), ones(10, dtype=int)], 
names='float,int', formats=[float, int])

In [4]: a
Out[4]:
recarray([(1.0, 1), (1.0, 1), (1.0, 1), (1.0, 1), (1.0, 1), (1.0, 1),
        (1.0, 1), (1.0, 1), (1.0, 1), (1.0, 1)],
       dtype=[('float', '>f8'), ('int', '>i4')])

In [6]: a.dtype.names
Out[6]: ('float', 'int')

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Robert K. <rob...@gm...>]

Les Schaffer wrote:

> 3. near zero docstrings for this module, hard to see how the new
> records  works.

http://www.scipy.org/RecordArrays

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Les S. <sch...@op...>]

Robert Kern wrote:
> http://www.scipy.org/RecordArrays
>   

which didn't help one iota. look, someone is charging for documentation,
but the claim is the free docstrings have docs. for the records module,
this ain't so. documentation means someone knows what is the complete
public interface. yes, examples help.

earlier, you said:

> In [6]: a.dtype.names
> Out[6]: ('float', 'int')

congratulations, this can be the first docstring in records.

now what about the incompatibility between old and new.


les schaffer

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Alan G I. <ai...@am...>]

On Sat, 26 Aug 2006, Les Schaffer apparently wrote:=20
> congratulations, this can be the first docstring in=20
> records.  now what about the incompatibility between old=20
> and new=20

I am always mystified when someone requesting free help
adopts a pissy tone if they do not immediately
get what they wish.

It reminds me a bit of my youngest child, age 7,
whom I am still teaching the advantages of politeness.

Cheers,
Alan Isaac

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Les S. <sch...@op...>]

Alan G Isaac wrote:
> I am always mystified when someone requesting free help
> adopts a pissy tone if they do not immediately
> get what they wish.
>
> It reminds me a bit of my youngest child, age 7,
> whom I am still teaching the advantages of politeness.
>   

you are refering to robert kern i take it???? because i am 52. and
relax, i have given plenty of free help in my life, and constantly asked
for it, pissy tones and all. so save the moral speech for your friends.

les

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Alan G I. <ai...@am...>]

On Sat, 26 Aug 2006, Les Schaffer apparently wrote:=20
> save the moral speech=20

I did not say anything about morals.
I spoke only of *advantages* of politeness,
which someone age 52 might still need to ponder.

Of course I bothered to write because I read this list and=20
appreciate in addition to its helpfulness that it generally=20
maintains a more polite tone.  This too has value.

Cheers,
Alan Isaac=20

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Les S. <sch...@op...>]

Alan G Isaac wrote:

> Of course I bothered to write because I read this list and 
> appreciate in addition to its helpfulness that it generally 
> maintains a more polite tone.  This too has value.
>
>
>   
so, you want to work on improving the documentation of this poorly
documented module? then lets get down to details. i'll pitch in some
time to add docstrings, if i know they will be used.

les

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Robert K. <rob...@gm...>]

Les Schaffer wrote:
> i'll pitch in some
> time to add docstrings, if i know they will be used.

Of course they will.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Alan G I. <ai...@am...>]

> Les Schaffer wrote: 
>> i'll pitch in some 
>> time to add docstrings, if i know they will be used. 


On Sat, 26 Aug 2006, Robert Kern apparently wrote: 
> Of course they will. 


Did Albert's initiative get any traction?
http://www.mail-archive.com/num...@li.../msg01616.html
If so, Les might profit from coordinating with him.

Is the preferred approach, as Albert suggested,
to submit documentation patches attached to tickets?

Cheers,
Alan Isaac

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Robert K. <rob...@gm...>]

Alan G Isaac wrote:

> Did Albert's initiative get any traction?
> http://www.mail-archive.com/num...@li.../msg01616.html
> If so, Les might profit from coordinating with him.

Not so much. Not many people showed up to the sprints, and most of those that 
did were working on their slides for their talks at the actual conference. Next 
year, sprints will come *after* the talks.

> Is the preferred approach, as Albert suggested,
> to submit documentation patches attached to tickets?

Yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Travis O. <oli...@ie...>]

Les Schaffer wrote:
> Travis E. Oliphant wrote:
>   
>> Porting is not difficult especially using the compatibility layers 
>> numpy.oldnumeric and numpy.numarray and the alter_code1.py modules in 
>> those packages.  The full C-API of Numeric is supported as is the C-API 
>> of Numarray.
>>   
>>     
>
> this is not true of numpy.core.records  (nee numarray.records):
>
> 1. numarray's records.py does not show up in numpy.numarray.
>   
Your right.  It's an oversight that needs to be corrected.    NumPy has 
a very capable records facility and the great people at STSCI have been 
very helpful in pointing out issues to help make it work reasonably like 
the numarray version.  In addition, the records.py module started as a 
direct grab of the numarray code-base, so I think I may have mistakenly 
believed it was equivalent.  But, it really should also be in the 
numarray compatibility module.

The same is true of the chararrays defined in numpy with respect to the 
numarray.strings module.
> 2. my code that uses recarrays is now broken if i use
> numpy.core.records; for one thing, you have no .info attribute. 
All the attributes are not supported.  The purpose of 
numpy.numarray.alter_code1 is to fix those attributes for you to numpy 
equivalents.  In the case of info, for example, there is the function 
numpy.numarray.info(self) instead of self.info().

> another
> example: strings pushed into the arrays *apparently* were stripped
> automagically in the old recarray (so we coded appropriately), but now
> are not.  
>   
We could try and address this in the compatibility module (there is the 
raw ability available to deal with this exactly as numarray did).   
Someone with more experience with numarray would really be able to help 
here as I'm not as aware of these kinds of issues, until they are 
pointed out. 
> 3. near zero docstrings for this module, hard to see how the new
> records  works.
>   
The records.py code has a lot of code taken and adapted from numarray 
nearly directly.   The docstrings present there were also copied over, 
but nothing more was added.  There is plenty of work to do on the 
docstrings in general.  This is an area, that even newcomers can 
contribute to greatly.  Contributions are greatly welcome.
> 4. last year i made a case for the old records to return a list of the
> column names. 
I prefer the word "field" names now so as to avoid over-use of the word 
"column", but one thing to understand about the record array is that it 
is a pretty "simple" sub-class.  And the basic ndarray, by itself 
contains the essential functionality of record arrays.   The whole 
purpose of the record sub-class is to come up with an interface that is 
"easier-to use," (right now that just means allowing attribute access to 
the field names).   Many may find that using the ndarray directly may be 
just what they are wanting and don't need the attribute-access allowed 
by the record-array sub-class.

> it looks like the column names are now attributes of the
> record object, any chance of getting a list of them
> recarrayObj.get_colNames() or some such? 
Right now, the column names are properties of the data-type object 
associated with the array, so that  recarrayObj.dtype.names will give 
you a list

The data-type object also has other properties which are useful. 

Thanks for your review.   We really need the help of as many numarray 
people as possible to make sure that the transition for them is easier.  
I've tried very hard to make sure that the numarray users have the tools 
they need to make the transition easier, but I know that more could be 
done.   Unfortunately, my availability to help with this is rapidly 
waning, however, as I have to move focus back to my teaching and research.

-Travis




-Travis

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[<sch...@op...>]

we have an Excel parser class with a method convert2RecArrayD that:

1. takes as input an Excel file name, plus an optional cell washing function (see below)

2. creates a recarray for each worksheet (we use UsedRange for the range of cells)
in the spreadsheet (via array()) and adds to a Python dict with keyword the name of the worksheet. the column -- errr field -- names are grabbed from the first row in each worksheet.

3. each cell in the spreadsheet is run thru the optional (else default) washer function. the default does unicode conversion plus some string.strip'ping 

we are using the spreadsheets as Resource files for a database application. so we are 
only reading the spreadsheets, not writing to them.

if this is useful, we'd be happy to put it somewhere useful.

Les

----- Original Message -----
From: Alan G Isaac <ai...@am...>
Date: Sunday, August 27, 2006 3:55 pm
Subject: Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available


> If your Excel parsing has general application and 
> illustrates applications beyond say 
> http://www.bigbold.com/snippets/posts/show/2036
> maybe you could post a URL to some code.

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Les S. <sch...@op...>]

Travis:

thanks for your response. over the next couple days i will be working
with the records module, trying to fix things so we can move from
numarray to numpy. i will try to collect some docstrings that can be
added to the code base.


Travis Oliphant wrote:
> Your right.  It's an oversight that needs to be corrected.    NumPy has 
> a very capable records facility and the great people at STSCI have been 
> very helpful in pointing out issues to help make it work reasonably like 
> the numarray version.  In addition, the records.py module started as a 
> direct grab of the numarray code-base, so I think I may have mistakenly 
> believed it was equivalent.  But, it really should also be in the 
> numarray compatibility module.
>   

this would solve our problem in the short run, so at least we can switch
to numpy and keep our code running.

> The same is true of the chararrays defined in numpy with respect to the 
> numarray.strings module.
>   

i take it this might solve the problem (below) of the automagic strip
with the numarray package.

>> 2. my code that uses recarrays is now broken if i use
>> numpy.core.records; for one thing, you have no .info attribute. 
>>     
> All the attributes are not supported.  The purpose of 
> numpy.numarray.alter_code1 is to fix those attributes for you to numpy 
> equivalents.  In the case of info, for example, there is the function 
> numpy.numarray.info(self) instead of self.info().
>   

thanks. i wasn't clear how to call the info function. now when i try
this, i get:

Traceback (most recent call last):
  File "<stdin>", line 772, in ?
  File "<stdin>", line 751, in _test_TableManager
  File "<stdin>", line 462, in build_db_table_structures
  File "<stdin>", line 108, in _create_tables_structure_from_rsrc
  File "C:\Program
Files\Python24\Lib\site-packages\numpy\numarray\functions.py", line 350,
in info
    print "aligned: ", obj.flags.isaligned
AttributeError: 'numpy.flagsobj' object has no attribute 'isaligned'

>   
>> another example: strings pushed into the arrays *apparently* were stripped
>> automagically in the old recarray (so we coded appropriately), but now
>> are not.  
>>   
>>     
> We could try and address this in the compatibility module (there is the 
> raw ability available to deal with this exactly as numarray did).   
> Someone with more experience with numarray would really be able to help 
> here as I'm not as aware of these kinds of issues, until they are 
> pointed out. 
>   

this would be great, because then i could find out where else code is
broke   ;-)

i will make my code changes in such a way that i can keep testing for
incompatibilities. so for now, i will add code to strip the
leading/trailing spaces off, but suitably if'ed so when this gets fixed
in numpy, i can pull out the strips and see if anything else works
differently than numarray.records.

>> 3. near zero docstrings for this module, hard to see how the new
>> records  works.
>>   
>>     
> The records.py code has a lot of code taken and adapted from numarray 
> nearly directly.   The docstrings present there were also copied over, 
> but nothing more was added.  There is plenty of work to do on the 
> docstrings in general.  This is an area, that even newcomers can 
> contribute to greatly.  Contributions are greatly welcome.
>   

ok, i will try and doc suggestions to whomever they should be sent to.

>> 4. last year i made a case for the old records to return a list of the
>> column names. 
>>     
> I prefer the word "field" names now so as to avoid over-use of the word 
> "column"

i have columnitis because we are parsing excel spreadsheets and pushing
them into recarrays. the first row of each spreadsheet has a set of
column names -- errrr, field names -- which is why we originally
attracted to records, since it gave us a way to grab columns -- errr,
fields -- easily and out of the box.

> but one thing to understand about the record array is that it 
> is a pretty "simple" sub-class.  And the basic ndarray, by itself 
> contains the essential functionality of record arrays.   The whole 
> purpose of the record sub-class is to come up with an interface that is 
> "easier-to use," (right now that just means allowing attribute access to 
> the field names).   Many may find that using the ndarray directly may be 
> just what they are wanting and don't need the attribute-access allowed 
> by the record-array sub-class.
>   

i'll look into how the raw ndarray works. like i said, we like that we
can get a listing of each column like so:

  recObj['column_errrr_fieldname']
>   
>> it looks like the column names are now attributes of the
>> record object, any chance of getting a list of them
>> recarrayObj.get_colNames() or some such? 
>>     
> Right now, the column names are properties of the data-type object 
> associated with the array, so that  recarrayObj.dtype.names will give 
> you a list
>
> The data-type object also has other properties which are useful. 
>   

it looks too like one can now create an ordinary array and PUSH IN
column -- errr, field -- information with dtype, is that right? pretty
slick if true.

i have some comments on the helper functions for creating record and
recarray objects, but i will leave that for later.

Les
>   

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Alan G I. <ai...@am...>]

On Sun, 27 Aug 2006, Les Schaffer apparently wrote: 
> we are parsing excel spreadsheets and pushing them into 
> recarrays 

If your Excel parsing has general application and 
illustrates applications beyond say 
http://www.bigbold.com/snippets/posts/show/2036
maybe you could post a URL to some code.

Cheers,
Alan Isaac

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[Bruce W. <bru...@gm...>]

Hi, Travis

I can pack my scripts into an executable with py2exe, but errors occur
once it runs:

No scipy-style subpackage 'random' found in D:\test\dist\numpy.
Ignoring: No module named info
import core -> failed: No module named _internal
import lib -> failed: 'module' object has no attribute '_ARRAY_API'
import linalg -> failed: 'module' object has no attribute '_ARRAY_API'
import dft -> failed: 'module' object has no attribute '_ARRAY_API'
Traceback (most recent call last):
  File "main.py", line 9, in ?
  File "numpy\__init__.pyc", line 49, in ?
    =01=07=02
  File "numpy\add_newdocs.pyc", line 2, in ?
    =10gkDc
  File "numpy\lib\__init__.pyc", line 5, in ?

  File "numpy\lib\type_check.pyc", line 8, in ?

  File "numpy\core\__init__.pyc", line 6, in ?

  File "numpy\core\umath.pyc", line 12, in ?

  File "numpy\core\umath.pyc", line 10, in __load

AttributeError: 'module' object has no attribute '_ARRAY_API'


This is the main.py file:
#=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
# filename:main.py
import wx
import numpy

class myFrame(wx.Frame):
    def __init__(self, *args, **kwds):
        wx.Frame.__init__(self, *args, **kwds)
        ##------ your widgets
        ##------ put stuff into sizer
        self.sizer_ =3D wx.BoxSizer(wx.VERTICAL)
        ## self.sizer_.Add(your_ctrl, proportion =3D 1, flag =3D wx.EXPAND)

        ## apply sizer
        self.SetSizer(self.sizer_)
        self.SetAutoLayout(True)

def main():    ## {{{
    app =3D wx.PySimpleApp(0)
    frame =3D myFrame(None, -1, title =3D '')
    frame.Show(True)
    app.SetTopWindow(frame)
    app.MainLoop()
##   }}}

if __name__ =3D=3D "__main__":main()
#=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
# filename:setup.py
import glob
import sys

from distutils.core import setup
import py2exe

includes =3D ["encodings",
            "encodings.*",
           ]

excludes =3D ["javax.comm"]

options =3D {
    "py2exe":
    {
        #"compressed": 1,
        #"optimize": 0,
        #"bundle_files":2,
        "skip_archive":1,
        "includes": includes,
        'excludes': excludes
    }
          }

setup(
    version =3D "0.1",
    description =3D "",
    name =3D "test",
    options =3D options,
    windows =3D [
        {
            "script":"main.py",
        }
        ],
    #zipfile =3D None,
    )


and I run this command to compile the scripts:
python setup.py py2exe

and all packages I use are:
python2.4.3
numpy-0.98
py2exe-0.6.5
wxpython-2.6.3.2

I unistalled Numeric before I compiled scripts.

If you google "numpy py2exe", you can find others guys stumbled by the
same issue with ease:

http://aspn.activestate.com/ASPN/Mail/Message/py2exe-users/3249182
http://www.nabble.com/matplotlib,-numpy-and-py2exe-t1901429.html

I just hope this can be fixed in the next table release of numpy.

On 8/29/06, Travis Oliphant <oli...@ie...> wrote:
> bruce.who.hk wrote:
> > Hi, Travis
> >
> > I just wonder if NumPy 1.0b4 can get along with py2exe? Just a few week=
s ago I made a application in Python. At first I used Numpy, it works OK, b=
ut I cannot pack it into a workable executable with py2exe and the XXX.log =
saied that numpy cannot find some module. I found some hints in py2exe wiki=
, but it still doesn't work. At Last I tried Numeric instead and it got OK.=
 I just hope that you donnot stop the maintenance of Numeric before you are=
 sure that Numpy can work with py2exe.
> >
> We've already stopped maintenance of Numeric nearly 1 year ago.    If
> NumPy doesn't work with py2exe then we need help figuring out why.  The
> beta-release period is the perfect time to fix that.  I've never used
> py2exe myself, but I seem to recall that some have been able to make it
> work.
>
> The problem may just be listing the right set of modules to carry along
> because you may not be able to get that with just the Python-side
> imports.  Post any errors you receive to
> num...@li...
>
> Thanks,
>
>
> -Travis
>
>


Bruce Who

Re: [Numpy-discussion] [ANN] NumPy 1.0b4 now available

[bruce.who.hk <bru...@gm...>]

Hi, Travis

I tried numpy1.0b4 and add this to setup.py

includes = ["numpy.core._internal"]

then it works! And all scripts can be packed into a single executables with 
        "bundle_files":2,
        "skip_archive":0,
        zipfile = None,

--skip_archive option is not needed now.


-------------------------------------------------------------
>I suspect you need to force-include the numpy/core/_internal.py file by 
>specifying it in your setup.py file as explained on the py2exe site. 
>That module is only imported by the multiarraymodule.c file which I 
>suspect py2exe can't automatically discern.
>
>In 1.0 we removed the package-loader issues which are probably giving 
>the scipy-style subpackage errors.  So, very likely you might be O.K. 
>with the beta releases of 1.0 as long as you tell py2exe about 
>numpy/core/_internal.py so that it gets included in the distribution.
>
>Please post any successes.
>
>Best,
>
>-Travis
>
>-- 
>http://mail.python.org/mailman/listinfo/python-list

------------------				 
bruce.who.hk
2006-08-30