numpy-discussion — Discussion list for all users of Numerical Python

Re: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation

[Matthew B. <mat...@gm...>]

Hi,

> While I fully respect the opinion of those concerned about Travis' decisi=
on, I
> have to say that I am personally not so worried.  At scipy'05, when Travi=
s
> announced the book idea, I asked a very simple question: 'are the docstri=
ngs
> complete?'.  His answer was 'yes'.
>
> There was a good reason for my asking this question:  I personally find t=
hat
> when coding, the most productive and efficient to learn a library is by t=
yping
>
> import foo
> foo.<TAB>
>
> and then
>
> foo.bar?

Just a tiny addition.  I know what you mean, but my experience is of
someone recently trying to learn numarray / Numeric, and I suspect I
am not alone in printing out the documentation and reading a moderate
amount of it before I get started.   Call me old-fashioned (it
wouldn't be the first time!),

See you,

Matthew

Re: [Numpy-discussion] Dcoument strings for ndarray

[Colin J. W. <cj...@sy...>]

Travis Oliphant wrote:

> Colin J. Williams wrote:
>
>> With a view to exploring Numeric3, below is a script to check the 
>> availability of doc strings.
>>
>> Some are not yet available.
>>
>> There is a method __class__ module but the purpose is not clear.
>>
>> Colin W.
>>
>> # checkDocs.py
>> ''' To check the availabilty of doc strings for the ndarray class.
>>
>>  Among the attributes of ndarray is:
>> __class__ module(name[, doc])
>
>
>
> I'm not sure what you are talking about here?  Where are you getting 
> this?
>
>
> I don't know of any such attribute.  There is a __class__ attribute.  
> But I don't see a
>
> __class__ module(name[, doc])   attribute
>
> -Travis
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
It might have been clearer if I had referred to the name "__class__" in 
the multiarray module.

Please see the sequence below:

     >>> import scipy.base.multiarray as M
     >>> M.__class__
    <type 'module'>
     >>>

I am surprised that this name points to a module.  It usually is an 
attribute of a class instance which
points to the class object of that instance.

Colin W.

[Numpy-discussion] Another PEP needed for the Numeric community.

[Travis O. <oli...@ee...>]

Full, 64-bit support in Python is problematic because of the buffer and 
sequence protocols which require int's in their interfaces.

Right now, numarray support memory mapped arrays.  SciPy can easily 
support memory mapped arrays by modifying the same module that numarray 
has forged (using the frombuffer function). 

However, the module will be limited on 64-bit systems because memory-map 
support in Python is limited to 32-bit files even on 64-bit systems.    
This was an intentional limitation of the memory map module in Python so 
that the sequence and buffer protocols could be supported.  However, the 
choice to not even allow a 64-bit size memory map to be created seems 
wrong.  There is no need to go through the sequence and buffer interface 
at all times.


If the memory-map module in Python were re-written to inherit from a 
big-map object that did not export the buffer and sequence protocols (or 
did so in a limited fashion), then all that is needed is for the object 
to export it's data pointer and size.  Then a function could be written 
to create an array in scipy that used the mmap's exported buffer as its 
data region.   This would allow very big memory mapped arrays without 
waiting for Python to fix it's sequence and buffer protocols (which may 
not be fixed until Python 3.0).

Thus, a PEP stating how the mmap module should change to support 64-bit 
systems is needed. 

Another possible project idea for any lurking people desiring to help.  
The other possibility is to just copy the nice muliplatform code in the 
Python mmap module and use an ndarray as the memory-mapped object.

-Travis

Re: [Numpy-discussion] Dcoument strings for ndarray

[Travis O. <oli...@ee...>]

Colin J. Williams wrote:

> With a view to exploring Numeric3, below is a script to check the 
> availability of doc strings.
>
> Some are not yet available.
>
> There is a method __class__ module but the purpose is not clear.
>
> Colin W.
>
> # checkDocs.py
> ''' To check the availabilty of doc strings for the ndarray class.
>
>  Among the attributes of ndarray is:
> __class__ module(name[, doc])


I'm not sure what you are talking about here?  Where are you getting this?


I don't know of any such attribute.  There is a __class__ attribute.  
But I don't see a

__class__ module(name[, doc])   attribute

-Travis

Re: [Numpy-discussion] Dcoument strings for ndarray

[Travis O. <oli...@ee...>]

Colin J. Williams wrote:

> With a view to exploring Numeric3, below is a script to check the 
> availability of doc strings.
>
> Some are not yet available.
>

I don't think your script will pick up the docs for the ndarray methods.

You need to replace

eval('M.'+i+'.__doc__')

with

eval('M.ndarray.'+i+'.__doc__')

-Travis

[Numpy-discussion] Dcoument strings for ndarray

[Colin J. W. <cj...@sy...>]

With a view to exploring Numeric3, below is a script to check the 
availability of doc strings.

Some are not yet available.

There is a method __class__ module but the purpose is not clear.

Colin W.

# checkDocs.py
''' To check the availabilty of doc strings for the ndarray class.

  Among the attributes of ndarray is:
__class__ module(name[, doc])

Create a module object.
The name must be a string; the optional doc argument can have any type. 

  ndarray is referenced as a class and reported as "<type 'type'>"
  but the common Python usage is <type 'classobj'>
  see the last output line.
  '''
import scipy.base.multiarray as M
print type(M.ndarray)
print '>', type( M)
print 'module doc:', M.__doc__
for i in dir(M.ndarray):
  print i,
  try:
    print eval('M.'+i+'.__doc__')
  except: 
    print 'No doc'
print '\n', M.ndarray.__doc__   
print type(M.ndarray)
# Check normal usage
import timeit
print timeit.Timer, type(timeit.Timer)

[Numpy-discussion] Problem with Windows version of Numeric3.0

[Colin J. W. <cj...@sy...>]

Either of these import lines crashes in Pythonw, with Boa-constructor, 
SPE or PythonWin.

# subclass.py

import scipy.base as num3
import scipy.base.multiarray as M

Colin W.

[Numpy-discussion] Re: Meaghan exhibitionism

[Fa M. <man...@do...>]

Hi Do you want t
VE UP
n your Med
o SA
TO 70% o
dications?
It's easy with us - The WebSite
VCXAVL
iagialanabiealiitr
ra $is $xnum $a
134 (30 p.)169 (30 p.)  218 (180 p.) 
And much more , Good luck 

[Numpy-discussion] Problem with name numeric

[Colin J. W. <cj...@sy...>]

With the following sequence I had expected "scipy.base.numeric" to be a 
module,  instead it is reported as a type.

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\>python
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import scipy.base.numeric
 >>> dir()
['__builtins__', '__doc__', '__name__', 'scipy']
 >>> scipy
<module 'scipy' from 'C:\Python24\lib\site-packages\scipy\__init__.pyc'>
 >>> scipy.base
<module 'scipy.base' from 
'C:\Python24\lib\site-packages\scipy\base\__init__.pyc'>
 >>> scipy.base.numeric
<type 'numeric_arrtype'>
 >>>
I need to be able to access ndarray to create a subclass.

Colin W.

[Numpy-discussion] Numeric3.0 - unable to retrieve data typecode

[Colin J. W. <cj...@sy...>]

# temp.py
import scipy.base as num3
a= num3.array([[1, 2], [9, 8]])
print a.typecode

C:\>python temp.py
Traceback (most recent call last):
  File "temp.py", line 4, in ?
    print a.typecode
AttributeError: 'scipy.ndarray' object has no attribute 'typecode'

Is there any simple way to retrieve the dataType, as with numarray's 
numerictypes?

Colin W.

Re: [Numpy-discussion] Re: Purchasing Documentation

[Todd M. <jm...@st...>]

Francesc Altet wrote:

>A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure:
>  
>
>>I share these feelings and would like to express my thanks to Todd
>>Miller and to Perry Greenfield for their help over the years.
>>    
>>
>
>Of course, I did not explicitely mention Todd in my previous message,
>but I would like to note that he has always been extremely responsive
>to suggestions and very efficient in solving many, many issues that
>appeared in numarray through the years.
>
>A big thanks to you as well, Todd :-)
>  
>
Ah... you're welcome!   Working on numarray has been a great privilege.

Regards,
Todd

Re: [Numpy-discussion] Re: Purchasing Documentation

[Francesc A. <fa...@ca...>]

A Divendres 07 Octubre 2005 18:40, Colin J. Williams va escriure:
> I share these feelings and would like to express my thanks to Todd
> Miller and to Perry Greenfield for their help over the years.

Of course, I did not explicitely mention Todd in my previous message,
but I would like to note that he has always been extremely responsive
to suggestions and very efficient in solving many, many issues that
appeared in numarray through the years.

A big thanks to you as well, Todd :-)

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Re: Purchasing Documentation

[Travis O. <oli...@ee...>]

Francesc Altet wrote:

>Also, until ready-for-production scipy_core arrives, I'd ask you (and
>the maintainers of scipy_core) to provide an array protocol to share
>data as efficiently as possible between numarray and scipy_core, just
>to easy the transition between the packages. If you want, we will be
>happy to collaborate on that area as well.
>  
>

There is an array protocol in place already  (see the Array Interface 
section on numeric.scipy.org). I think it is fully supported in numarray.

Also, I have a PEP for placing a simple array object in Python that is 
nothing more than a shell for passing data around.    If somebody would 
like to push that forward they are welcome.

Right now what I have is available in an SVN server.

http://svn.scipy.org/svn/PEP

get it with

svn co http://svn.scipy.org/svn/PEP PEP


-Travis

Re: [Numpy-discussion] Re: Purchasing Documentation

[Francesc A. <fa...@ca...>]

A Divendres 07 Octubre 2005 14:51, Perry Greenfield va escriure:
> Our basic position has been and continues to be that if scipy_core provid=
es
> all the capabilities that we need, we will switch to it. We have been
> involved in installing it on the platforms we need it for (namely Solaris,
> which has proven difficult to build on) and testing it (which is ongoing).
> So far things look pretty promising and barring any unforseen problems, we
> will likely begin porting recarray to it next week. As Travis has already
> mentioned, the whole point of scipy_core was to unify the two existing
> projects, not introduce a 3rd. I have been involved all along in design a=
nd
> interface discussions with Travis on this
> project.

Ok. That's clear enough and satifies my curiosity, thank you. I must
confess, though, that I'm a bit disappointed becuase I was hoping that
numarray, now that has arrived to maturity, was going to stay for long
time (as any other good piece of software deserves). Anyway, as a
long-time user of numarray I can't express enough my gratitude towards
you and your team for suporting and enhancing a beast like numarray,
that introduced a lot of new features that missed Numeric and that
without it all of our current projects (and I guess many others around
the world) simply would not have been possible.

> We will maintain numarray until the point at which we switch all of our
> code to scipy_core (i.e., all libraries and applications that depend on
> it). This may take several months for some (likely the most work is for C
> extensions that use the numarray C-API).

Yeah, I do think so. However, what it worries me is, in my (limited)
experience, that substituting large packages like Numeric or numarray,
is not a trivial task at all (after all, how much time took until
numarray was able to be a good substitute of Numeric?). So, I forsee a
transitional time no shorter than a year before arriving to the high
standards of quality that numarray has nowadays. This is why I'm not
precisely eager to quickly switch to scipy_core until it has proven to
be, at least, as solid as it is numarray.

Having said that, we are in the mood of offering our implementation of
a NestedRecArray class that allows nested fields in RecArray objects.
This class has been included in PyTables for some months and has
proven to be stable, comes with a nice set of unit tests, and is
highly efficient (at least, as much as RecArray for most operations).

Also, until ready-for-production scipy_core arrives, I'd ask you (and
the maintainers of scipy_core) to provide an array protocol to share
data as efficiently as possible between numarray and scipy_core, just
to easy the transition between the packages. If you want, we will be
happy to collaborate on that area as well.

> While I appreciate the pain that this may cause those who have written
> C-extensions for numarray, it's hard to deny that unification will bring
> great benefits. It's as much pain for us as anyone.

Sure, but migration work is not the biggest problem (hopefully) for
us. As I said before, the key point to migrate to scipy_core is to
make sure that is stable enough, featured enough, coner-cases-proof
enough and well-documented enough as it is numarray now.

Thanks again for all your work in numarray,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] wrapping question

[Todd M. <jm...@st...>]

usr...@pr... wrote:

>>usr...@pr... wrote:
>>
>>    
>>
>>>Hi all,
>>>
>>>
>>>I'm using the numarray package. I would like to use the numarray C API
>>>to create a new array object and fill it with data without copying.
>>>Also, I
>>>
>>>
>>>
>>>      
>>>
>>The numarray C-API has a compatibility layer which supports most Numeric
>>C-API functionality.   So,  using numarray,  you can write most
>>Numeric/Numeric3/scipy_core-like code in C.  For historical reasons
>>numarray also has a wider set of native APIs (i.e. NA_vNewArray()),  but
>>since numarray is likely to be replaced by scipy_core when it matures,
>>it's wisest to use the compatibility API.
>>
>>    
>>
>>>would like numarray to take care of freeing the data once the numarray
>>>object is destructed. I've managed to find some related posts in the
>>>mailing list archive, but it is really unclear which of these posts are
>>> still accurate and what the current 'recommended' approach is. See:
>>>
>>>http://sourceforge.net/mailarchive/message.php?msg_id=11788304
>>>http://sourceforge.net/mailarchive/message.php?msg_id=2494535
>>>
>>>
>>>Currently, I first create a new PyArrayObject that is large enough to
>>>hold the data: array =
>>>NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims);
>>>
>>>
>>>then I pass array->data to a C function that reads the data from disk.
>>>Any
>>>comments on this approach?
>>>
>>>      
>>>
>>That should work fine;  'array' owns the data and will free it when
>>'array' is destructed.
>>
>>
>>A better approach,  however,  is to use numarray's PyArray_FromDims()
>>which is the Numeric/Numeric3/scipy_core API.
>>
>>    
>>
>>>Also, I was wondering if numarray frees the
>>>data using this approach (because it also allocates the memory in the
>>>first place within the NA_vNewArray function.)
>>>
>>>
>>>      
>>>
>>Yes, numarray will free the data allocated internally by
>>NA_vNewArray().  numarray will also free the data allocated by
>>PyArray_FromDims().
>>    
>>
>
>Thanx for the reply! Just to be sure I understand this correctly: to
>conform to the Numeric/Numeric3/scipy api I could/should do the following:
>
>1. allocate a new array using PyArray_FromDims()
>2. pass the ->data pointer to a C routine that reads the data
>3. use PyArray_Return to return the array to Python
>4. No DECREF's are necessary?
>
>Also, PyArray_FromDims() will not copy the data, right?
>Thanks,
>  
>
1. yes
2. yes
3. PyArray_Return is mostly important if you want rank-0 arrays 
converted into equivalent Python scalars.   Otherwise I think it's not 
really needed and is commonly omitted.
4. When you call PyArray_FromDims(),  the returned array has one 
reference.   If the array is a temporary you want to deallocate at 
extension function exit,  then DECREF;  if you want to return the 
array,  don't DECREF.

Todd

>Joris
>
>/*
>(From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html)
>PyObject* PyArray_FromDims(int nd, int *dims, int type)
>    This function will allocate a new numarray.
>    An array created with PyArray_FromDims can be used as a temporary or
>    returned using PyArray_Return.
>
>    Used as a temporary, calling Py_DECREF deallocates it.
>*/
>
>  
>
>>Regards,
>>Todd
>>
>>
>>
>>-------------------------------------------------------
>>This SF.Net email is sponsored by:
>>Power Architecture Resource Center: Free content, downloads, discussions,
>>and more. http://solutions.newsforge.com/ibmarch.tmpl
>>_______________________________________________
>>Numpy-discussion mailing list
>>Num...@li...
>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>>
>>
>>    
>>
>
>
>
>
>-------------------------------------------------------
>This SF.Net email is sponsored by:
>Power Architecture Resource Center: Free content, downloads, discussions,
>and more. http://solutions.newsforge.com/ibmarch.tmpl
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>

Re: [Numpy-discussion] wrapping question

[<usr...@pr...>]

> usr...@pr... wrote:
>
>> Hi all,
>>
>>
>> I'm using the numarray package. I would like to use the numarray C API
>> to create a new array object and fill it with data without copying.
>> Also, I
>>
>>
>>
> The numarray C-API has a compatibility layer which supports most Numeric
> C-API functionality.   So,  using numarray,  you can write most
> Numeric/Numeric3/scipy_core-like code in C.  For historical reasons
> numarray also has a wider set of native APIs (i.e. NA_vNewArray()),  but
> since numarray is likely to be replaced by scipy_core when it matures,
> it's wisest to use the compatibility API.
>
>> would like numarray to take care of freeing the data once the numarray
>> object is destructed. I've managed to find some related posts in the
>> mailing list archive, but it is really unclear which of these posts are
>>  still accurate and what the current 'recommended' approach is. See:
>>
>> http://sourceforge.net/mailarchive/message.php?msg_id=11788304
>> http://sourceforge.net/mailarchive/message.php?msg_id=2494535
>>
>>
>> Currently, I first create a new PyArrayObject that is large enough to
>> hold the data: array =
>> NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims);
>>
>>
>> then I pass array->data to a C function that reads the data from disk.
>> Any
>> comments on this approach?
>>
> That should work fine;  'array' owns the data and will free it when
> 'array' is destructed.
>
>
> A better approach,  however,  is to use numarray's PyArray_FromDims()
> which is the Numeric/Numeric3/scipy_core API.
>
>> Also, I was wondering if numarray frees the
>> data using this approach (because it also allocates the memory in the
>> first place within the NA_vNewArray function.)
>>
>>
> Yes, numarray will free the data allocated internally by
> NA_vNewArray().  numarray will also free the data allocated by
> PyArray_FromDims().

Thanx for the reply! Just to be sure I understand this correctly: to
conform to the Numeric/Numeric3/scipy api I could/should do the following:

1. allocate a new array using PyArray_FromDims()
2. pass the ->data pointer to a C routine that reads the data
3. use PyArray_Return to return the array to Python
4. No DECREF's are necessary?

Also, PyArray_FromDims() will not copy the data, right?
Thanks,

Joris

/*
(From http://stsdas.stsci.edu/numarray/numarray-1.3.html/node58.html)
PyObject* PyArray_FromDims(int nd, int *dims, int type)
    This function will allocate a new numarray.
    An array created with PyArray_FromDims can be used as a temporary or
    returned using PyArray_Return.

    Used as a temporary, calling Py_DECREF deallocates it.
*/

>
> Regards,
> Todd
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>

Re: [Numpy-discussion] wrapping question

[Todd M. <jm...@st...>]

usr...@pr... wrote:

>Hi all,
>
>I'm using the numarray package. I would like to use the numarray C API to
>create a new array object and fill it with data without copying. Also, I
>  
>
The numarray C-API has a compatibility layer which supports most Numeric 
C-API functionality.   So,  using numarray,  you can write most 
Numeric/Numeric3/scipy_core-like code in C.  For historical reasons 
numarray also has a wider set of native APIs (i.e. NA_vNewArray()),  but 
since numarray is likely to be replaced by scipy_core when it matures,   
it's wisest to use the compatibility API.

>would like numarray to take care of freeing the data once the numarray
>object is destructed. I've managed to find some related posts in the
>mailing list archive, but it is really unclear which of these posts are
>still accurate and what the current 'recommended' approach is. See:
>
>http://sourceforge.net/mailarchive/message.php?msg_id=11788304
>http://sourceforge.net/mailarchive/message.php?msg_id=2494535
>
>Currently, I first create a new PyArrayObject that is large enough to hold
>the data:
>array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims);
>
>then I pass array->data to a C function that reads the data from disk. Any
>comments on this approach? 
>
That should work fine;  'array' owns the data and will free it when 
'array' is destructed. 

A better approach,  however,  is to use numarray's PyArray_FromDims() 
which is the Numeric/Numeric3/scipy_core API.  

>Also, I was wondering if numarray frees the
>data using this approach (because it also allocates the memory in the
>first place within the NA_vNewArray function.)
>  
>
Yes, numarray will free the data allocated internally by 
NA_vNewArray().  numarray will also free the data allocated by 
PyArray_FromDims().

Regards,
Todd

[Numpy-discussion] wrapping question

[<usr...@pr...>]

Hi all,

I'm using the numarray package. I would like to use the numarray C API to
create a new array object and fill it with data without copying. Also, I
would like numarray to take care of freeing the data once the numarray
object is destructed. I've managed to find some related posts in the
mailing list archive, but it is really unclear which of these posts are
still accurate and what the current 'recommended' approach is. See:

http://sourceforge.net/mailarchive/message.php?msg_id=11788304
http://sourceforge.net/mailarchive/message.php?msg_id=2494535

Currently, I first create a new PyArrayObject that is large enough to hold
the data:
array = NA_vNewArray(NULL,__numarray_type,tmp_num_dims,tmp_dims);

then I pass array->data to a C function that reads the data from disk. Any
comments on this approach? Also, I was wondering if numarray frees the
data using this approach (because it also allocates the memory in the
first place within the NA_vNewArray function.)
Thanks,

Joris van Zwieten

RE: [Numpy-discussion] Re: Purchasing Documentation

[Perry G. <pe...@st...>]

Sorry for the tardy response. Transatlantic flights don't yet have
convenient internet service.

I've been meaning to put out a message of this sort soon anyway. This is few
days earlier than I was planning, but close enough.

Our basic position has been and continues to be that if scipy_core provides
all the capabilities that we need, we will switch to it. We have been
involved in installing it on the platforms we need it for (namely Solaris,
which has proven difficult to build on) and testing it (which is ongoing).
So far things look pretty promising and barring any unforseen problems, we
will likely begin porting recarray to it next week. As Travis has already
mentioned, the whole point of scipy_core was to unify the two existing
projects, not introduce a 3rd. I have been involved all along in design and
interface discussions with Travis on this
project.

We will maintain numarray until the point at which we switch all of our code
to scipy_core (i.e., all libraries and applications that depend on it). This
may take several months for some (likely the most work is for C extensions
that use the numarray C-API). When this is complete (including testing), we
may support numarray for a while in the sense of answering questions, but it
would not likely see any more new releases after that point. We do not have
the resources to support two different numeric packages indefinitely. We
will report on our progress in this switchover (most likely our libraries
and applications will be set up to work with both in the interim).

While I appreciate the pain that this may cause those who have written
C-extensions for numarray, it's hard to deny that unification will bring
great benefits. It's as much pain for us as anyone. It should not be as
painful to port python code since scipy_core should support all
capabilities. But some changes will be needed since things are not always
done the same way (e.g., dtype for type).

Perry

Francesc Altet wrote:
>
> Exactly, I'm wondering the same questions. Provided that numarray is
> an excellent and mature piece of code, I'd like to hear a declaration
> of principles on that subject.
>
> Regards,
>
> A Dijous 06 Octubre 2005 01:55, Colin J. Williams va escriure:
> > Perry,
> >
> > It would be helpful if you gave some indication of your organization's
> > intention with respect to numarray.
> >
> > Will it be maintained, as in the past, fully in the public domain and
> > without charge for either the code or the API information needed to make
> > use of the code?
> >
> > New versions have been produced every few months, will this continue?
> >
> > Colin W.
> >
>

Re: [Numpy-discussion] A comparative test

[Chris B. <Chr...@no...>]

Rick White wrote:

> You might wonder why everyone didn't notice when their numarray
> programs started to run blazingly fast after 1.3.0. 

Maybe because those of us for whom small array performance is important 
don't use numarray? I know I haven't tested my wxPython FloatCanvas with 
numarray since I discovered painfully slow performance a while back.


-Chris




-- 
Christopher Barker, Ph.D.
Oceanographer
                                      		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation

[Arnd B. <arn...@we...>]

On Thu, 6 Oct 2005, Fernando Perez wrote:

[...]

> To reiterate before my reputation is tarnished forever: the bet was with John,
> who is the doughnuts-eating one.  Being a man of taste, I eat bagels :) [and I
> wasn't present when the bet was made]
>
> With that critical point out of the way, on to minor details.

Thanks for the clarification - this definitively made my day ;-)

[...]
> In fact, I'd like docstrings to provide (simple) examples in them,

+(some large positive number ;-)

> and one thing on my radar now that
> ipython is growing GUI/XML legs is to add support for latex in docstrings.

Are you thinking of restructured text
  http://docutils.sourceforge.net/
together with the mathhack
  http://docutils.sourceforge.net/sandbox/cben/rolehack/
or the tools used in the LaTeXwiki,
  http://mcelrath.org/Notes/LatexWiki
?

Independent of this, I see two main points:
- how should the examples be added
- who should add the examples
to the doc-strings?
Getting "end-users" to help with this task sounds quite attractive to me.

Would it make sense to revive the pynotes idea
http://www.scipy.org/documentation/mailman?fn=scipy-dev/2004-November/002536.html
which would make it possible to add remarks to any python command?

However, I don't know if there is a good solution on how to
submit these notes and how they finally
should get merged into the doc-string
(One possibility would be to combine this in some way
with Travis' live-docs).
It might be unavoidable that at the end some knowledgable person has
to go over suggested additions and incorporate them
on a case-by-case basis?

Best,

Arnd

Re: [SciPy-user] [SciPy-dev] Re: [Numpy-discussion] Purchasing Documentation

[Fernando P. <Fer...@co...>]

Travis Oliphant wrote:

> Yes, yes.  Fernando knows me pretty well in that regard. I'll probably 
> contribute to it with useful selections from the book (once I have the 
> book finished).    It really motivates me to produce more  (of 
> everything) when I see people actually purchasing the documentation.

To reiterate before my reputation is tarnished forever: the bet was with John, 
who is the doughnuts-eating one.  Being a man of taste, I eat bagels :) [and I 
wasn't present when the bet was made]

With that critical point out of the way, on to minor details.

> It would be great if somebody could at least post the pydoc output for 
> the core.  That is a good start.  I'll add basic C-API calls to free 
> documentation and we'll be ready to go.

While I fully respect the opinion of those concerned about Travis' decision, I 
have to say that I am personally not so worried.  At scipy'05, when Travis 
announced the book idea, I asked a very simple question: 'are the docstrings 
complete?'.  His answer was 'yes'.

There was a good reason for my asking this question:  I personally find that 
when coding, the most productive and efficient to learn a library is by typing

import foo
foo.<TAB>

and then

foo.bar?

or

foo.bar??

if I want to see the source (for python-implemented things).

I understand that some people prefer to read manuals/books, and there are 
topics beyond the single-function API that are best discussed in a book 
format.  But given the instantaneous response of the above, I would rather use 
this than wade through an index in most cases.  In fact, I'd like docstrings 
to provide (simple) examples in them, and one thing on my radar now that 
ipython is growing GUI/XML legs is to add support for latex in docstrings.

I think that a library where all individual methods/functions have full, 
accurate and example-loaded docstrings, and where class declarations and 
top-level module docstrings provide the higher-level overview, is extremely 
usable as is.  Especially when you can also keep pydoc running in html server 
mode (pydoc -p) to browse at your leisure, and trivially dump that HTML info 
to disk for static reference if needed.

If this is done, I think that the space for a 'real book' is better defined, 
providing less of a detail reference and more of a teaching coverage.  I sense 
that this is Travis' approach, and I personally have no beef with it 
whatsoever.  If he had deliberately stripped or crippled all docstrings from 
new_scipy to artificially create a need for his book, that would be a 
different story.  But he certainly didn't do such a thing.

I do think that there's a lot of room for higher-level books on python for 
scientific computing.  Langtangen's is already out there, and Perry's 
excellent tutorial is already free for all to use (while slightly focused on 
astronomy, I find it a very useful document).

I guess what I'm trying to say is that the _library_ documentation can be 
considered to be a (good) set of docstrings.  As long as scipy ships with such 
a thing (which can be improved with the easiest of all patches by users, since 
no code is touched), I am not so worried that it will be perceived as lacking 
real docs.

Please note that I am not discussing the price/time constraints of his 
licensing, which is strictly his choice to make.

Again, this is just my _personal_ opinion.  And while I have great 
appreciation for Travis, he's not paying me to say any of this :)

Regards,

f

[Numpy-discussion] Simple optimization

[Travis O. <oli...@ee...>]

Colin J. Williams wrote:

> # NumericNumarrayTest.py
> ''' A test, provided by Francesc Altet
>    
> http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 
>
>    '''
> from timeit import Timer
> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)"
> print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", 
> setup).timeit(number=100)
>
> #
> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)"
> print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): 
> row=a[i]", setup).timeit(number=100)
>
> # scipy/Numeric3 added
> setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)"
> print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", 
> setup).timeit(number=100)
>
> RESULTS:
> >pythonw -u "NumericNumarrayTest.py"


> Timer-Numeric23.8: 0.179712784725
> Timer-numarray1.3.3: 0.21674933546
> Timer-Numeric3: 0.253077136899


After a simple optimization in the array_subscript function (to check 
for scalar selection up front): new results are (for 1000 runs on my 
system):

Numeric:  0.56431102752685547
numarray:  1.0897960662841797
scipy:    0.57508182525634766    (it's now executing nearly the 
identical code as Numeric)


Any more optimization ideas?

-Travis

Re: [Numpy-discussion] A comparative test

[Fernando P. <Fer...@co...>]

Rick White wrote:
> IMHO the numarray developers put their emphasis in the right place
> by focusing on large array performance and improved functionality,
> and all the noise around small array indexing performance was just
> a red herring that convinced some folks not to try numarray because
> they heard it was slow.  I hope Travis doesn't devote a lot of
> effort to this optimization right now.  I'd be much more interested
> in seeing a large array benchmark.

Except for codes like our PDE solvers, which need to create 10s of millions of 
small arrays very, very fast.  So no, it's not a red herring [1].  Just 
because you don't need something doesn't mean there's no valid need for it.

I am not disputing in any way the value of the many, many improvements made by 
numarray, nor the importance of fixing large array performance for many 
applications (such as I imagine is the typical workflow of astronomical data 
analysis).  The fact that Travis tried to incorporate all of numarrays' 
improvements into the new library is a recognition of the value of all this work.

But calling the small array performance problem a 'red herring' is inaccurate 
and unfair, at best.

Regards,

f


[1] Yes, I could preallocate pools of memory for this, manage them manually, 
etc.  But then I wouldn't be writing my code in python, would I?  The whole 
point of using python is to have my cake and eat it too: I want high-level 
code that gives me near-hardware max speeds.  With carefully tuned (python) 
code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of 
thinking about algorithm design, I get that.

Re: [Numpy-discussion] A comparative test

[Rick W. <rl...@st...>]

On Thu, 6 Oct 2005, Francesc Altet wrote:

> A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure:
> > RESULTS:
> >  >pythonw -u "NumericNumarrayTest.py"
> >
> > Timer-Numeric23.8: 0.179712784725
> > Timer-numarray1.3.3: 0.21674933546
> > Timer-Numeric3: 0.253077136899
>
> Is that really true? From my original post:
>
> >>> from timeit import Timer
> >>> setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)"
> >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100)
>  0.12782907485961914
> >>> setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)"
> >>> Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100)
>  1.2013700008392334
>
> My post was from January, so, in the interim numarray has improved *a
> lot* it's object creation time. In that case, why the numarray team
> hasn't publisized that more? Well, I think I should be missing
> something. Time for nap.

Well, this seems pretty clear:

On Thu, 14 Apr 2005, Todd Miller wrote:

> Subject: [Numpy-discussion] ANN: numarray-1.3.0
>
> I. ENHANCEMENTS
>
> 2. Removal of dictionary update from array view creation improves
> performance of view/slice/subarray creation.  This should e.g. improve
> the performance of wxPython sequence protocol access to Nx2 arrays.

You might wonder why everyone didn't notice when their numarray
programs started to run blazingly fast after 1.3.0.  My (jaded)
view is that it is very rare to have an application where the overall
execution time is dominated by the time to create small arrays.
When you're in that limit, there are a million other overheads that
are also slowing you down.  In the vast majority of applications
you could reduce the array creation time to zero and probably
wouldn't notice the difference without running timing tests.

IMHO the numarray developers put their emphasis in the right place
by focusing on large array performance and improved functionality,
and all the noise around small array indexing performance was just
a red herring that convinced some folks not to try numarray because
they heard it was slow.  I hope Travis doesn't devote a lot of
effort to this optimization right now.  I'd be much more interested
in seeing a large array benchmark.
				Rick