numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] HTML manual display problems in Mozilla

[Edward C. J. <edc...@er...>]

Mozilla .0.9.3 is not correctly displaying the HTML version of the 
manual. The problem appears to be in footnotes. Netscape 4.76 had 
similar problems with the previous version of the manual. I use RedHat 
7.1 Linux on a PC.

Thanks,
Ed Jones

Re: [Numpy-discussion] manual update

[Rob <eu...@ho...>]

Re Matlab,

I've never used it before, but I was very suprised to see that its
arrays work much like Numpy.  I have some Matlab FDTD code here and the
arrays are sliced instead of indexed.  I might be able to rope a few
more people at work into Python based on that :)    Rob.

[Numpy-discussion] manual update

[Paul F. D. <pa...@pf...>]

I have updated the Numeric manual on the web site. I added some to the
MLab section to reflect the axis arguments that were added some time ago
but never reflected in the manual. The manual also documents a change to
the behavior of the attribute "masked" in package MA. The document
corresponds to CVS now rather than the last release. I hope to release
Numeric-20.2 sometime soon but want more exercise of the MA and MLab
changes first.
 
I changed MLab.mean to return a floating point result when the argument
is integer. I'm sorry if this breaks anything but the prior behavior has
no mathematical meaning and then, worse, was being used by std to get
the wrong standard deviation too. I changed std to work for axis > 0, as
it was previously broken.
 
I don't own Matlab and so maintaining a package that emulates something
I don't have makes me nervous. I wish one of the other developers with
more knowledge in this area would pitch in.
 
Thanks to Chris Barker and others who sent in typos and suggestions.

Re: [Numpy-discussion] C API questions and tricks.

[<ro...@ho...>]

>>>>> "CB" == Chris Barker <chr...@ho...> writes:

I wrote my own specialized clip function too, only for 'f' and 'i'
types, and only for contiguous arrays....

 CB> A) How do I loop through all the elements of a discontiguous
 CB> array of arbitraty dimensions? If I knew the rank ahead of time,
 CB> I could just nest some for loops and use the strides[]
 CB> values. Not knowing before hand, it seems that I should be able
 CB> to do some nesting of loops using nd and dimensions[], but I
 CB> can't seem to work it out. Someone must have come up with a nifty
 CB> way to do this. Is there an existing macro or function to do it?

Never done this in C for numpy yet, but I normally program this
along the following lines:

n=[2,3,2,3]
x=[0,0,0,0]
while 1:
    print x
    i=len(n)-1
    while i>=0 and x[i]+1==n[i]:
       x[i]=0
       i=i-1
    if i<0:
       break
    x[i]=x[i]+1

It is always going to be slower than the straight loop for contiguous
arrays. So if you want to do it properly, you'd have to check whether
the array is contiguous, and if it is, use the fast way.

Regards,

Rob Hooft

-- 
=====   ro...@ho...                 http://www.hooft.net/people/rob/  =====
=====   R&D, Bruker Nonius BV, Delft  http://www.nonius.nl/             =====
===== PGPid 0xFA19277D ================================= Use Linux! =========

Re: [Numpy-discussion] For the next round on the manual

[Chris B. <chr...@ho...>]

While you're at it, I found what appears to be another small typo:

In the API section, description of the ArrayObject structure, is the
line:

a.shape = (dimensions[0]...dimensions[nd])


shouldn't this be dimensions[nd-1] ?


-Chris


-- 
Christopher Barker,
Ph.D.                                                           
Chr...@ho...                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------

RE: [Numpy-discussion] For the next round on the manual

[Paul F. D. <pa...@pf...>]

Fixed in my manual source, thank you. 
 
-----Original Message-----
From: num...@li...
[mailto:num...@li...] On Behalf Of Eric
Hagemann
Sent: Saturday, August 18, 2001 5:21 PM
To: Num...@li...
Subject: [Numpy-discussion] For the next round on the manual
 
Type-o in the manual
 
    Check the definition for the 'convolve' function.  Manual tells of
mode=0 as default when code and experiment point to mode=2 being default
 
    No biggie -- but it might be nice to correct if anybody is editing
the manual any time soon
 
Cheers
Eric
 

[Numpy-discussion] C API questions and tricks.

[Chris B. <chr...@ho...>]

Hi all,

I recently discovered the "clip()" function, and thought it was just
what I'd been wanting, because I need to do that a lot, and it provided
a nice notation, and I was hoping speed improvement over a couple of
where() statements.

I did some experiments, and discovered that it was, in fact, slower than
the where statements, and that the fastest way to do it is to use two
putmask() calls. (note: this changes the array in place, rather than
creating a new one)

The reason it is not faster is because it is written in Python, calling
choose(), similarly to how where() does. I decided that I could use a
fast version, so I set out to write one in C, resulting in the following
questions:

A) How do I loop through all the elements of a discontiguous array of
arbitraty dimensions? If I knew the rank ahead of time, I could just
nest some for loops and use the strides[] values. Not knowing before
hand, it seems that I should be able to do some nesting of loops using
nd and dimensions[], but I can't seem to work it out. Someone must have
come up with a nifty way to do this. Is there an existing macro or
function to do it?

B) How can I write a function that can act on any of the NumPy data
types? I currently have it written to only work with contiguous Float
arrays, which is what i need at the moment, but I'd much rather have one
for the general case.

C) I'd also like any feedback on other elements of my code (enclosed
with this email). A few comments on what the function (I've called it
fastclip) is supposed to do:

"""
fastclip(A,min,max)

changes the array, A, in place, so that all the elements less than min
are replaced by min, and all the elements greater that max are replaced
by max.

min and max can be either scalars, or anything that can be converted to
an array with the same number of elements as A (using
PyArray_ContiguousFromObject() ). If min and/or max is an array, than
the coresponding elements are used. This allows, among other things, a
way to clip to just a min or max value by calling it as:
fastclip(A,A,max) or fastclip(A,min,A).

"""

I wrote a little test script to benchmark the function, and it is much
faster that the alternatives that I have thought of:

#!/usr/bin/env python
# testing speed of where vs clip vs fastclip

from Numeric import *
from RandomArray import uniform
from NumericExtras import fastclip
import time

n = 5000

a = uniform(0,100,(n,))
b = uniform(0,100,(n,))
c = uniform(0,100,(n,))

min = 20.0
max = 80.0

print "n = %i"%(n,)

start = time.clock()
for i in range(100):
    a = clip(a,min,max)
print "clip took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    putmask(a,a < min,min)
    putmask(a,a > max,max)
print "putmask took %f seconds"%(time.clock()-start)

start = time.clock()
for i in range(100):
    fastclip(a,min,max)
print "fastclip took %f seconds"%(time.clock()-start)

Here are some results:

n = 50
clip took 0.020000 seconds
putmask took 0.050000 seconds
fastclip took 0.010000 seconds

n = 5000
clip took 0.300000 seconds
putmask took 0.230000 seconds
fastclip took 0.030000 seconds

As expected the large the array is, the better the improvement.

I'd love to here any feedbackyou can give me: I've new to writing Python
extensions, using the  Numeric API, and C itself, for that matter.

-thanks, Chris



















-- 
Christopher Barker,
Ph.D.                                                           
Chr...@ho...                 ---           ---           ---
http://members.home.net/barkerlohmann ---@@       -----@@       -----@@
                                   ------@@@     ------@@@     ------@@@
Oil Spill Modeling                ------   @    ------   @   ------   @
Water Resources Engineering       -------      ---------     --------    
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------

[Numpy-discussion] patch to speed up LinearAlgebra module

[Jeff W. <js...@cd...>]

I've uploaded a patch to LinearAlgebra.py and lapack_litemodule.c that
speeds up some computations (svd, linear least-squares, and hermitian
eigenanalysis) by 3-5 times when linking with external (preferably
optimized) blas and lapack libs.  The patch replaces calls to dgesvd,
dgelss and dsyev with the newer, faster lapack routines dgesdd, dgelsd and
dsyevd.  Here are my timing results before and after (G4 350mhz powermac,
optimized ATLAS blas lib and lapack 3.0 from netlib.org):

Before:

svd of 1000X1000 matrix takes 609.19
generalized inverse of 1000X1000 matrix takes 744.36
linear least-squares solution of 1000X1000 matrix takes
451.68
eigenvectors of 1000X1000 symmetric matrix takes 210.08

After:

svd of 1000X1000 matrix takes 142.55
generalized inverse of 1000X1000 matrix takes 237.08
linear least-squares solution of 1000X1000 matrix takes
81.91
eigenvectors of 1000X1000 symmetric matrix takes 56.23


Note that since these newer lapack routines are not in lapack_lite, you
must link external blas and lapack3 libs.

-Jeff

-- 
Jeffrey S. Whitaker         Phone  : (303)497-6313
Meteorologist               FAX    : (303)497-6449
NOAA/OAR/CDC  R/CDC1        Email  : js...@cd...
325 Broadway                Web    : www.cdc.noaa.gov/~jsw
Boulder, CO, USA 80303-3328 Office : Skaggs Research Cntr 1D-124

Re: [Numpy-discussion] Matlab_NumPy_Array?

[Greg B. <gb...@cf...>]

Try replacing step 4 with

A_string=fromstring(read_file, 'd')

'f' is for 32 bit floats, 'd' is for 64 bit floats.
Matlab uses 64 bit (double) floats.

HTH

--Greg Ball



On Sun, 19 Aug 2001, Karshi Hasanov wrote:

> 
> Hi all,
> 
>  I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
> did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
>       2) file_open=open('~/A.dat', 'rb')
>       3) read_file=file_open.read()
>       4) A_string=fromstring(read_file, 'f')
>       5) A=reshape(A_string, (i,j,k))
> where i,j,k are size of A[i,j,k]
> 
> I am not getting the right. 
> What is the best/right way of convering arrays?
> Thanks
> 
> 
> 
> 

[Numpy-discussion] Matlab_NumPy_Array?

[Karshi H. <kar...@ut...>]

Hi all,

 I  don't understand how to conver arrays from Matlab to Numpy. Here what I 
did: 1) Converted Matlab A(i,j,k) to 'A.dat' binary file
      2) file_open=open('~/A.dat', 'rb')
      3) read_file=file_open.read()
      4) A_string=fromstring(read_file, 'f')
      5) A=reshape(A_string, (i,j,k))
where i,j,k are size of A[i,j,k]

I am not getting the right. 
What is the best/right way of convering arrays?
Thanks

[Numpy-discussion] For the next round on the manual

[Eric H. <eha...@ho...>]

Type-o in the manual

    Check the definition for the 'convolve' function.  Manual tells of =
mode=3D0 as default when code and experiment point to mode=3D2 being =
default

    No biggie -- but it might be nice to correct if anybody is editing =
the manual any time soon

Cheers
Eric

[Numpy-discussion] MA patch submitter, please upload file

[Paul F. D. <pa...@pf...>]

Would the person who submitted the patch concerning MA binary ops,
please upload the file to the patch, or email it to me? You have
identified a real problem whose answer is not obvious so I'd like to see
your idea of how to fix it.
 
If you log in to SourceForge before submitting bugs and patches then I
can know who you are to thank you or make followup inquiries like this
one.

RE: [Numpy-discussion] Subarray with with arbitrary index?

[Paul F. D. <pa...@pf...>]

I'm afraid that you have to do put using one-d indices. But you do *not*
have to try to ravel the source. I.e., the first arg is just the name of
the array.
>>> from Numeric import *
>>> x=array([[1,2,3],[4,5,6]])
>>> x
array([[1, 2, 3],
       [4, 5, 6]])
>>> put(x,[0,4],[100,200])
>>> x
array([[100,   2,   3],
       [  4, 200,   6]])
>>>
-----Original Message-----
From: Huaiyu Zhu [mailto:hua...@ya...] 
Sent: Friday, August 17, 2001 11:36 PM
To: Paul F. Dubois
Cc: John J. Lee
Subject: RE: [Numpy-discussion] Subarray with with arbitrary index?

Thanks, John and Paul.  That is what I was looking for.

It did not occur to me to look for verbs put and take, rather than words
line sub- index, slice and so on.  Maybe puting some of these words in
the manual could help people doing a search?

Now that this made the most costly part of my program about seven times
faster, other problems become more prominent.  One of such question is:
How do we do it on more than one axis?

Suppose a is a 2d array. Then put(a[1,:], b, c) works, but put(a[:,1],
b,
c) complains about the first argument not a continuous array.  Doing
transpose does not help.  So do I guess it right that this is
implemented
only in the representation of a linear array?

If so, there would be no hope of using put(a, ([2, 4], [1,2]), v) or
even
more exotic ones like using += on an arbitray subgrid?

Regards,

Huaiyu

On Fri, 17 Aug 2001, Paul F. Dubois wrote:

> John is right:
> >>> a=Numeric.arange(8)
> >>> b=Numeric.array([2,3,5])
> >>> c=Numeric.arange(3)+100
> >>> Numeric.put(a,b,c)
> >>> print a
> [  0   1 100 101   4 102   6   7]
> 
> Thanks for pointing out that I had left allclose out of the Numeric
part of
> the manual. I did it in the MA part and then forgot. I'm fixing it
now. BTW:
> There are changenotes at source forge that are sometimes ahead of the
> manual.
> 
> -----Original Message-----
> From: num...@li...
> [mailto:num...@li...]On Behalf Of John
> J. Lee
> Sent: Friday, August 17, 2001 6:36 AM
> To: Huaiyu Zhu
> Cc: num...@li...
> Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?
> 
> 
> On Thu, 16 Aug 2001, Huaiyu Zhu wrote:
> 
> > Hi,
> >
> > Is it possible to assign to a subarray with arbitrary index?
> >
> > Suppose I have three arrays
> >
> > a = arange(8)
> > b = array([2, 3, 5])
> > c = arange(3)+100
> >
> > I want a function f, such that calling  f(a, b, c) would change a to
> >
> > [0 1 100 101 4 102 6 7]
> 
> f = Numeric.put
> f(a, b, c)
> 
> put used to be in Python, but it's been in C since some release
17.x.x, I
> think.
> 
> I have a sinking feeling that I must have missed something (no
interpreter
> here to check it works)...
> 
> BTW, a week ago I noticed that I had reinvented the wheel in
rewriting, in
> an uglier and less efficient form, Numeric.allclose (hope I got the
name
> right).  As far as I can see, it isn't listed in the manual.  Did I
miss
> it?  All it would need is the docstring copying over.
> 
> 
> John
> 
> 
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> http://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

RE: [Numpy-discussion] Subarray with with arbitrary index?

[Paul F. D. <pa...@pf...>]

John is right:
>>> a=Numeric.arange(8)
>>> b=Numeric.array([2,3,5])
>>> c=Numeric.arange(3)+100
>>> Numeric.put(a,b,c)
>>> print a
[  0   1 100 101   4 102   6   7]

Thanks for pointing out that I had left allclose out of the Numeric part of
the manual. I did it in the MA part and then forgot. I'm fixing it now. BTW:
There are changenotes at source forge that are sometimes ahead of the
manual.

-----Original Message-----
From: num...@li...
[mailto:num...@li...]On Behalf Of John
J. Lee
Sent: Friday, August 17, 2001 6:36 AM
To: Huaiyu Zhu
Cc: num...@li...
Subject: Re: [Numpy-discussion] Subarray with with arbitrary index?


On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John


_______________________________________________
Numpy-discussion mailing list
Num...@li...
http://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Subarray with with arbitrary index?

[John J. L. <jj...@po...>]

On Thu, 16 Aug 2001, Huaiyu Zhu wrote:

> Hi,
>
> Is it possible to assign to a subarray with arbitrary index?
>
> Suppose I have three arrays
>
> a = arange(8)
> b = array([2, 3, 5])
> c = arange(3)+100
>
> I want a function f, such that calling  f(a, b, c) would change a to
>
> [0 1 100 101 4 102 6 7]

f = Numeric.put
f(a, b, c)

put used to be in Python, but it's been in C since some release 17.x.x, I
think.

I have a sinking feeling that I must have missed something (no interpreter
here to check it works)...

BTW, a week ago I noticed that I had reinvented the wheel in rewriting, in
an uglier and less efficient form, Numeric.allclose (hope I got the name
right).  As far as I can see, it isn't listed in the manual.  Did I miss
it?  All it would need is the docstring copying over.


John

[Numpy-discussion] Subarray with with arbitrary index?

[Huaiyu Z. <hua...@ya...>]

Hi,

Is it possible to assign to a subarray with arbitrary index?

Suppose I have three arrays

a = arange(8)
b = array([2, 3, 5])
c = arange(3)+100

I want a function f, such that calling  f(a, b, c) would change a to

[0 1 100 101 4 102 6 7]

This would be useful in certain applications that would otherwise
require sparse matrices.   



Huaiyu Zhu

[Numpy-discussion] SWIG + Numeric 20.1.0 = core dump: the fix

[Edward C. J. <edc...@er...>]

If
    Numeric and SWIG are used together in creating a Python extension
  and
    there are C/C++ files as part of the extension
then
  Numeric 20.1.0 will probably break your code.

This happens because changes were made to arrayobject.h between
Numeric 20.0.0 and 20.1.0.

The changes that need to be made in your code are explained
by Konrad Hinsen:
(http://www.geocrawler.com/archives/3/1329/2001/5/0/5718231/)

     FROM: Konrad Hinsen
     DATE: 05/09/2001 09:12:25
     SUBJECT:  [Numpy-discussion] Modified header files

     Recently we had a discussion about how to use NumPy arrays from
     extension modules with multiple source files, on various
     platforms.  The modified header files that are attached to this
     message provide a (hopefully!) general solution. In fact, I
     propose to make them part of the official distribution, unless
     there are objections.

     If used like before, these header files give exactly the same
     result as the ones in NumPy 20.0.0. However, they permit to define
     the name of the C API pointer array and make it globally visible.
     Under the condition that the chosen name is unique, this should
     not create problems under any platform, no matter if static or
     dynamic linking is used.

     To use NumPy features in multiple source file extension modules,
     you have to write

        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #include "Numeric/arrayobject.h"

     in the main source file (the one that contains the init function)
     and
     
        #define PY_ARRAY_UNIQUE_SYMBOL PyArrayXXX
        #define NO_IMPORT_ARRAY
        #include "Numeric/arrayobject.h"

     in all the others. The symbol you choose instead of PyArrayXXX
     should contain both the name of the imported module (array) and
     the name of the importing module (whatever your module is called)
     in order to be unique with a reasonably high probability.

     The same applies to the Ufunc module, just replace "array" by
     "ufunc" in the example. I have also applied the "static"
     correction to the Ufunc header file, there is no reason not to do
     it.

     Konrad.
     -- 
     -----------------------------------------------------------------------
     Konrad Hinsen                            | E-Mail:
     hi...@NO...
     Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
     Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
     45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
     France                                   | Nederlands/Francais
     -----------------------------------------------------------------------

I do this by modifying one header file that is included in all
the ".c" and ".i" files. In the following example the header file
is "spam.h".

------------------------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'
------------------------------------------------------------
spam.h:

#include "Python.h"
#define PY_ARRAY_UNIQUE_SYMBOL Py_Array_API_spam
#ifndef SWIG_FILE_WITH_INIT
  #define NO_IMPORT_ARRAY
#endif

#include "/usr/include/python2.1/Numeric/arrayobject.h"
void funct(void);
------------------------------------------------------------
spam.i:

%module spam

%{
#define SWIG_FILE_WITH_INIT
#include "spam.h"
%}  

%init %{
import_array();
printf("called import_array\n");
%}

void funct(void);
------------------------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}
------------------------------------------------------------
compile script:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so
------------------------------------------------------------

I use RedHat 7.1 Linux on a PC, Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, and gcc 2.96. Anyone is welcome to use
the above example in documentation.

Re: [Numpy-discussion] SWIG / Numeric core dump

[Joe V. A. <van...@at...>]

The best way of finding core dumps with a Python extension is to compile
the extension source with '-g', and then follow these steps.
(You may also have to compile the Numeric Extensions with '-g', to trace
into them).

% gdb /usr/bin/python2.1
(gdb) br _PyImport_LoadDynamicModule 
(gdb) cont   # repeat until your extension is loaded


(gdb) finish # to load your extension
(gdb) br wrap_myfunction
(gdb) disable 1   # don't want to break for more modules being loaded
(gdb) continue


Note: you can't set a breakpoint in your code until your module has been
loaded.


-- 
Joe VanAndel  	          
National Center for Atmospheric Research
http://www.atd.ucar.edu/~vanandel/
Internet: van...@uc...

[Numpy-discussion] SWIG / Numeric core dump

[Edward C. J. <edc...@er...>]

The following Python / SWIG / Numeric program dumps core.
I use RedHat Linux 7.1 on a PC. Python 2.1, Numeric 20.1.0,
SWIG 1.1 build 883, gcc 2.96. What is the problem?

The output looks like:

> doit.py
called import_array
Calling PyArray_FromDims
Segmentation fault (core dumped)
>

The program files are:
----------------------------------------------
doit.py:

#! /usr/bin/python

import Numeric, spam

spam.funct()
print 'spam.funct completed'

----------------------------------------------
spam.h:

#include "Python.h"
#include "/usr/include/python2.1/Numeric/arrayobject.h"

void funct(void);

----------------------------------------------
spam.i

%module spam

%{
#include "spam.h"
%}  

%init %{
    import_array();
    printf("called import_array\n");
%}

void funct(void);

----------------------------------------------
spammodule.c

#include "spam.h"

void funct(void)
{
    PyArrayObject *pao;
    int dims[2];

    dims[0] = 100;
    dims[1] = 200;
    printf("Calling PyArray_FromDims\n");
    pao = (PyArrayObject*) PyArray_FromDims(2, dims, PyArray_UBYTE);
    printf("Completed PyArray_FromDims\n");
    Py_DECREF(pao);
}

----------------------------------------------
swigit:

swig -python spam.i
gcc -c -Wall spammodule.c spam_wrap.c -I/usr/include/python2.1
ld -shared spammodule.o spam_wrap.o -o spam.so

RE: [Numpy-discussion] NumPy installation instructions for Windows

[Paul F. D. <pa...@pf...>]

The windows installer does everything required to import Numeric.

-----Original Message-----
From: num...@li...
[mailto:num...@li...]On Behalf Of Just
van Rossum
Sent: Friday, August 10, 2001 12:30 AM
To: num...@li...
Subject: [Numpy-discussion] NumPy installation instructions for Windows


I develop a package that depends on NumPy, and I would like to add a line or
two
to my documentation about how to install NumPy under Windows -- just the
basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The
.exe
files are  Windows installers of the sort that you are used to". Does this
mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just

_______________________________________________
Numpy-discussion mailing list
Num...@li...
http://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] NumPy installation instructions for Windows

[Just v. R. <ju...@le...>]

I develop a package that depends on NumPy, and I would like to add a line or two
to my documentation about how to install NumPy under Windows -- just the basic
set of modules, not the extensions.

I understand users should grab the .exe archive. The FAQ only mentions "The .exe
files are  Windows installers of the sort that you are used to". Does this mean
everything is explained and done by the installer itself, or is there some
additional action the user should perform? Thanks!

(I have no Windows box here so I can't see for myself...)

Just

[Numpy-discussion] swig and C/C++

[Tariq R. <tar...@li...>]

in fact ... even simpler ... can anyone point me to an example of passing
Numeric arrays to and from C ... the swig examples are fine for integers /
floats ... but as i'm a new to this stuff I need a bit if guidance with
Numeric arrays ...

any ideas / pointers would be very gratefully receieved... i'm working on
wavelet stuff which i want to give back to the Numeric community....

thanks

tariq		(tariq_rashid at lineone.net)

----------------------------

Today's Topics:

   1. NumTut (=?ks_c_5601-1987?B?wMyx1LrA?=)
   2. swig and Numeric (Paul F. Dubois)

Message: 2
From: "Paul F. Dubois" <du...@ll...>
Organization: Lawrence Livermore National Lab
To: num...@li...
Date: Tue, 7 Aug 2001 10:20:19 -0700
Subject: [Numpy-discussion] swig and Numeric

Does someone have an example of using SWIG to connect Numeric arrays to C=
 code
that expects double* arguments?

Re: [Numpy-discussion] fast constructor for arrays from byte data (unpickling?)

[Robert K. <ke...@ca...>]

On Wed, Aug 08, 2001 at 01:37:02PM -0700, Chris Barker wrote:
> Robert Kern wrote:
> 
> > > Thanks, that works, but now I am wondering: what I want is a fast and
> > > memory efficient way to get the contents of a file into a NujmPy array,
> > > this sure doesn't look any better than:
> > >
> > > a = fromstring(file.read()))
> > 
> > Depends on how large the file is. file.read() creates a temporary string the
> > size of the file. That string isn't freed until fromstring() finishes and
> > returns the array object. For a brief time, both the string and the array have
> > duplicates of the same data taking up space in memory.
> 
> Exactly. Also is the memory used for hte string guarranteed to be freed
> right away? I have no idea how Python internals work.

Once fromstring returns, the string's refcount is 0 and should be freed just
about right away. I'm not sure, but I don't think the new gc will affect that.

> Anyway, that's why I want a "fromfile()" method, like the one inthe
> library array module.

Yes, I agree, it would be nice to have.

> > I don't know the details of mmap, so it's certainly possible that the only way
> > that fromstring knows how to access the data is to pull all of it into memory
> > first, thus recreating the problem. Alas.
> 
> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

Performing a very cursory, very non-scientific study, I created a 110M file,
mmap'ed it, then made a string from some of it. I'm using 64MB of RAM, 128MB
swap partition on a Linux 2.4.7 machine. According to top(1), the memory use 
didn't jump up until I made the string. Also, given that the call to mmap
returned almost instantaneously, I'd say that mmap doesn't pull the whole file
into memory when the object is created (one could just read the file for that).

I don't know what test to perform to see whether the fromstring() constructor
uses double memory, but my guess would be that memcpy won't pull in the whole
file before copying. OTOH, the accessed portions of the mmap'ed file may be kept
in memory.

Does anyone know the details on mmap? I'm shooting in the dark, here.
<reads Phil Austin's e-mail>  Ooh, nice.

> -Chris

-- 
Robert Kern
ke...@ca...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

Re: [Numpy-discussion] mmap (was: fast constructor for arrays from byte data)

[Robert K. <ke...@ca...>]

On Wed, Aug 08, 2001 at 01:56:56PM -0700, Phil Austin wrote:

[snip pointers to info]

> bottom line: you really need a vm implementation that supports
> madvise() to get the maximum benefit from mmap.  (Linux 2.4
> might have madvise, I haven't tried it out yet -- 2.2 didn't).

FWIW, there appears to be a madvise(2) system call in Linux 2.4.7 (in 
mm/filemap.c for those of you with kernel source trees lying around), but I
can't find it in the headers anywhere. Sigh. I don't know if madvise is supposed
to be internal or not, though.

-- 
Robert Kern
ke...@ca...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

[Numpy-discussion] mmap (was: fast constructor for arrays from byte data)

[Phil A. <ph...@ge...>]

Chris Barker writes:

> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...

This might be more than you need to know, but there's a 
copy of the article:

"Why aren't you using mmap() yet?"
by Kevin Sheehan at:

http://thrush.eos.ubc.ca/users/phil/mmap/mmap.pdf (note reversed page
order)

and a Solaris/SGI email debate in mhonarc format:

http://thrush.eos.ubc.ca/users/phil/mmap/threads.html

bottom line: you really need a vm implementation that supports
madvise() to get the maximum benefit from mmap.  (Linux 2.4
might have madvise, I haven't tried it out yet -- 2.2 didn't).

The way I used to use mmap was through a numpy array class that was
initiated from a pointer to a mmap'd file.  This made reading/writing
to the file as simple as indexing into the numpy array.  I haven't
been back to reimplement that class on linux since I left solaris,
though.  You can also get some of the benefits of mmap via netcdf,
which will use mmap if it exists.

Regards, Phil