numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] numarray memmap

[Sebastian H. <ha...@ms...>]

Hi,
I would like to use memmap array more often - here are some comments:
(Please look for lines with $$$$)

>>> from numarray import memmap as mm
>>> m = mm.open("C:/mm1", mode='w+', len=0)
>>> n = m.insert(0,0)
>>> a = na.array(buffer=n)
X:\PrWin\wxPython\lib\PyCrust\PyCrustApp.py:1: DeprecationWarning: The
'buffer' keyword is deprecated.  Use 'sequence' instead.
Exception exceptions.AttributeError: "Memmap instance has no attribute
'_mode'" in Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "X:\PrWin\numarray\numarraycore.py", line 279, in array
    type = getTypeObject(sequence, type, typecode)
  File "X:\PrWin\numarray\numarraycore.py", line 223, in getTypeObject
    return Py2NumType[ _maxtype(sequence) ]
KeyError

$$$$ Is this a mistake on my part or should the error message be "better"
???


>>> a = na.array(buffer=n, type=na.UInt16, shape=(0,0,0))
>>> a.shape
(0, 0, 0)
>>> m
<Memmap on file 'C:/mm1' with mode='w+', length=0, 1 slices>
>>> a.resize((100,100,100))
>>> a.shape
(100, 100, 100)
>>> m.sync()
>>> m.flush()
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "X:\PrWin\numarray\memmap.py", line 501, in flush
    self._consolidate(filename)
  File "X:\PrWin\numarray\memmap.py", line 469, in _consolidate
    f.write(self._buffer(m, mlen))
  File "X:\PrWin\numarray\memmap.py", line 317, in _buffer
    raise RuntimeError("Memmap no longer valid.  (closed?)")
RuntimeError: Memmap no longer valid.  (closed?)
>>> m
<Memmap on file 'C:/mm1' with mode='w+', length=0, 1 slices>

$$$$  Is this a bug ?

My idea is to "prepare" the memmap file before I actually know the array
dimensions that should go into it.  Is that a good idea ?

Thanks,
Sebastian Haase

Re: [Numpy-discussion] zero dimensions in numarray.strings?

[Todd M. <jm...@st...>]

On Wed, 2003-12-03 at 05:08, Francesc Alted wrote:
> A Dimarts 02 Desembre 2003 12:52, Todd Miller va escriure:
> >
> > I fixed the immediate problem with zero element string arrays this
> > morning.  I imagine there are probably other problems with that case...
> > Please let me know if and when you find them.
> 
> Thanks. Well, I've found another small issue. The next works:
> 
> >>> array([]).info()
> class: <class 'numarray.numarraycore.NumArray'>
> shape: (0,)
> strides: (4,)
> byteoffset: 0
> bytestride: 4
> itemsize: 4
> aligned: 1
> contiguous: 1
> data: <memory at 40552060 with size:0 held by object 40552040 aliasing object 
> 00000000>
> byteorder: little
> byteswap: 0
> type: Int32
> 
> 
> But this don't:
> 
> >>> strings.array([])
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 964, 
> in array
>     return fromlist(buffer, itemsize=itemsize, shape=shape, kind=kind)
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 828, 
> in fromlist
>     shape, itemsize = _slistShape(slist, itemsize=itemsize, shape=shape)
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 763, 
> in _slistShape
>     shape_items = _slistShape0(slist)
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 746, 
> in _slistShape0
>     maxs = _slistShape0(slist[0])
> IndexError: list index out of range
> 
> Perhaps (?) you want to make it equivalent to:
> 
> >>> strings.array(None, shape=(0,), itemsize=0).info()
> class: <class 'numarray.strings.CharArray'>
> shape: (0,)
> strides: (0,)
> byteoffset: 0
> bytestride: 0
> itemsize: 0
> aligned: 1
> contiguous: 1
> data: <memory at 405521e0 with size:0 held by object 405521c0 aliasing object 
> 00000000>
> type: CharArrayType(0)
> 
> I've used the numarray present in CVS to run these test.

OK.  I appended this to the 'zero dimension string' bug report so that
it doesn't get lost.

Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] numarray zeros - keyword argument 'typecode'

[Todd M. <jm...@st...>]

On Tue, 2003-12-02 at 19:44, Sebastian Haase wrote:
> Hi,
> I'm just trying to debug some PyOpenGl stuff . It seems that this is only
> available with Numeric (no numarray !?)
> 
> Drawing a 60000 vertex array takes 6 sec ( should be much less than 0.5 sec
> !! )
> I found this is because all my other code is using numarray and it is this
> conversion that takes so long.
> Switching to Numeric (and back to numarray) got this error message:
> [[[ Num  is here numarray !! ]]
> self.m_histPlotArray = Num.zeros((n,2), typecode=Num.Float32)
> TypeError: zeros() got an unexpected keyword argument 'typecode'

numarray is still weak in a few places for keyword names.  Try 'type'
instead. 

Todd
> 
> I thought numarray  suppossed to be backwards compatible ... so I just
> report this as a bug.
> 
> Regards,
> Sebastian
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] zero dimensions in numarray.strings?

[Francesc A. <fa...@op...>]

A Dimarts 02 Desembre 2003 12:52, Todd Miller va escriure:
>
> I fixed the immediate problem with zero element string arrays this
> morning.  I imagine there are probably other problems with that case...
> Please let me know if and when you find them.

Thanks. Well, I've found another small issue. The next works:

>>> array([]).info()
class: <class 'numarray.numarraycore.NumArray'>
shape: (0,)
strides: (4,)
byteoffset: 0
bytestride: 4
itemsize: 4
aligned: 1
contiguous: 1
data: <memory at 40552060 with size:0 held by object 40552040 aliasing object 
00000000>
byteorder: little
byteswap: 0
type: Int32


But this don't:

>>> strings.array([])
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 964, 
in array
    return fromlist(buffer, itemsize=itemsize, shape=shape, kind=kind)
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 828, 
in fromlist
    shape, itemsize = _slistShape(slist, itemsize=itemsize, shape=shape)
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 763, 
in _slistShape
    shape_items = _slistShape0(slist)
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 746, 
in _slistShape0
    maxs = _slistShape0(slist[0])
IndexError: list index out of range

Perhaps (?) you want to make it equivalent to:

>>> strings.array(None, shape=(0,), itemsize=0).info()
class: <class 'numarray.strings.CharArray'>
shape: (0,)
strides: (0,)
byteoffset: 0
bytestride: 0
itemsize: 0
aligned: 1
contiguous: 1
data: <memory at 405521e0 with size:0 held by object 405521c0 aliasing object 
00000000>
type: CharArrayType(0)

I've used the numarray present in CVS to run these test.

Cheers,

-- 
Francesc Alted

[Numpy-discussion] numarray zeros - keyword argument 'typecode'

[Sebastian H. <ha...@ms...>]

Hi,
I'm just trying to debug some PyOpenGl stuff . It seems that this is only
available with Numeric (no numarray !?)

Drawing a 60000 vertex array takes 6 sec ( should be much less than 0.5 sec
!! )
I found this is because all my other code is using numarray and it is this
conversion that takes so long.
Switching to Numeric (and back to numarray) got this error message:
[[[ Num  is here numarray !! ]]
self.m_histPlotArray = Num.zeros((n,2), typecode=Num.Float32)
TypeError: zeros() got an unexpected keyword argument 'typecode'

I thought numarray  suppossed to be backwards compatible ... so I just
report this as a bug.

Regards,
Sebastian

Re: [Numpy-discussion] CVS - built can't find package directory

[Todd M. <jm...@st...>]

On Tue, 2003-12-02 at 16:56, Sebastian Haase wrote:
> > CVS builds fine for me with gcc-3.2.2.  I don't have gcc-2.95 anymore.
> > I built numarray yesterday with Sun cc so I don't think it's a compiler
> > issue.
> >
> > Looking at the code in question,  it is fairly new (last 3-4 weeks) so
> > if it didn't get updated,  or got updated with conflicts, it could cause
> > problems... but not for me.  I suggest wiping your current CVS and doing
> > a complete checkout.
> 
> I changed in Src/_ufuncmodule.c around line 40 and line 43 (removed the
> semicolons):
> 
> #if _PYTHON_CALLBACKS
> #define ConverterRebuffer(conv,arr,inb) \
>               PyObject_CallMethod(conv, "rebuffer", "(OO)", arr, inb)
> #else
> #define ConverterRebuffer(conv,arr,inb) \
>  ((PyConverterObject *)conv)->rebuffer(conv, arr, inb)
> #endif
> 
> I do not understand why this caused trouble - but in my opinion there don't
> need to be any semiconlons, because you use it like a function call.

I agree.  They're gone now in CVS.  I haven't recompiled widely though.

> 
> Is there some documentation on the nd_image package ?

In CVS, yes. 

> 
> Thanks
> - Sebastian

Regards,
Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] CVS - built can't find package directory

[Sebastian H. <ha...@ms...>]

> CVS builds fine for me with gcc-3.2.2.  I don't have gcc-2.95 anymore.
> I built numarray yesterday with Sun cc so I don't think it's a compiler
> issue.
>
> Looking at the code in question,  it is fairly new (last 3-4 weeks) so
> if it didn't get updated,  or got updated with conflicts, it could cause
> problems... but not for me.  I suggest wiping your current CVS and doing
> a complete checkout.

I changed in Src/_ufuncmodule.c around line 40 and line 43 (removed the
semicolons):

#if _PYTHON_CALLBACKS
#define ConverterRebuffer(conv,arr,inb) \
              PyObject_CallMethod(conv, "rebuffer", "(OO)", arr, inb)
#else
#define ConverterRebuffer(conv,arr,inb) \
 ((PyConverterObject *)conv)->rebuffer(conv, arr, inb)
#endif

I do not understand why this caused trouble - but in my opinion there don't
need to be any semiconlons, because you use it like a function call.

Is there some documentation on the nd_image package ?

Thanks
- Sebastian




>
> Regards,
> Todd
>
> On Tue, 2003-12-02 at 15:33, Sebastian Haase wrote:
> > <snip>
> > > cvs update -d -P
> >
> > Thanks - that was it ...
> >
> > <snip>
> >
> > > > Then I get a compiler error:
> > > >
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -IInclude/numarray
> > > > -I/usr/include/python2.2 -c Src/_ufuncmodule.c -o
> > > > build/temp.linux-i686-2.2/_ufuncmodule.o
> > > > Src/_ufuncmodule.c: In function `_slow_exec1':
> > > > Src/_ufuncmodule.c:1555: parse error before `*'
> > > > Src/_ufuncmodule.c:1556: `ro' undeclared (first use in this
function)
> > > > Src/_ufuncmodule.c:1556: (Each undeclared identifier is reported
only
> > once
> > > > Src/_ufuncmodule.c:1556: for each function it appears in.)
> > > > Src/_ufuncmodule.c: In function `_slow_exec2':
> > > > Src/_ufuncmodule.c:1876: parse error before `*'
> > > > Src/_ufuncmodule.c:1877: parse error before `*'
> > > > Src/_ufuncmodule.c:1878: `ri1' undeclared (first use in this
function)
> > > > Src/_ufuncmodule.c:1878: `ro' undeclared (first use in this
function)
> > > > Src/_ufuncmodule.c:1908: parse error before `*'
> > > > Src/_ufuncmodule.c:1909: parse error before `*'
> > > > error: command 'gcc' failed with exit status 1
> > > >
> > >
> > > I'm not sure what happened here.  CVS is currently viable though.  Try
> > > again.
> > >
> >
> > I just ran it again ... same thing. (my gcc is version 2.95)
> > Could you check this ... ?
> >
> > - Sebastian
> >
> -- 
> Todd Miller <jm...@st...>
>
>

Re: [Numpy-discussion] CVS - built can't find package directory

[Todd M. <jm...@st...>]

On Tue, 2003-12-02 at 14:48, Sebastian Haase wrote:
> Hi,
> I'm trying to test some features of the latest CVS version.
> I get:
> $ python2.2 setup.py build --gencode
> <snip>
> error: package directory 'Packages/nd_image/Lib' does not exist

This looks like a CVS update gone bad:  a directory added since
numarray-0.7 that didn't get populated by your checkout.  I suggest
doing:

cvs update -d -P  

*or* just wipe out your old CVS and do a fresh checkout.

> 
>    -- after taking out the line of addons I get:
> error: package directory 'Packages/MA/Lib' does not exist
> 

Ditto here.  One update in numarray should do it.

> Then I get a compiler error:
> gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -IInclude/numarray 
> -I/usr/include/python2.2 -c Src/_ufuncmodule.c -o 
> build/temp.linux-i686-2.2/_ufuncmodule.o
> Src/_ufuncmodule.c: In function `_slow_exec1':
> Src/_ufuncmodule.c:1555: parse error before `*'
> Src/_ufuncmodule.c:1556: `ro' undeclared (first use in this function)
> Src/_ufuncmodule.c:1556: (Each undeclared identifier is reported only once
> Src/_ufuncmodule.c:1556: for each function it appears in.)
> Src/_ufuncmodule.c: In function `_slow_exec2':
> Src/_ufuncmodule.c:1876: parse error before `*'
> Src/_ufuncmodule.c:1877: parse error before `*'
> Src/_ufuncmodule.c:1878: `ri1' undeclared (first use in this function)
> Src/_ufuncmodule.c:1878: `ro' undeclared (first use in this function)
> Src/_ufuncmodule.c:1908: parse error before `*'
> Src/_ufuncmodule.c:1909: parse error before `*'
> error: command 'gcc' failed with exit status 1
> 

I'm not sure what happened here.  CVS is currently viable though.  Try
again.

> 
> What are these packages ? 

MA is a port of Paul Dubois masked array package.  nd_image is Peter
Verveer's new multi-dimensional image processing package.

> 

Cheers,

Todd

-- 
Todd Miller <jm...@st...>

[Numpy-discussion] CVS - built can't find package directory

[Sebastian H. <ha...@ms...>]

Hi,
I'm trying to test some features of the latest CVS version.
I get:
$ python2.2 setup.py build --gencode
<snip>
error: package directory 'Packages/nd_image/Lib' does not exist

   -- after taking out the line of addons I get:
error: package directory 'Packages/MA/Lib' does not exist

Then I get a compiler error:
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -IInclude/numarray 
-I/usr/include/python2.2 -c Src/_ufuncmodule.c -o 
build/temp.linux-i686-2.2/_ufuncmodule.o
Src/_ufuncmodule.c: In function `_slow_exec1':
Src/_ufuncmodule.c:1555: parse error before `*'
Src/_ufuncmodule.c:1556: `ro' undeclared (first use in this function)
Src/_ufuncmodule.c:1556: (Each undeclared identifier is reported only once
Src/_ufuncmodule.c:1556: for each function it appears in.)
Src/_ufuncmodule.c: In function `_slow_exec2':
Src/_ufuncmodule.c:1876: parse error before `*'
Src/_ufuncmodule.c:1877: parse error before `*'
Src/_ufuncmodule.c:1878: `ri1' undeclared (first use in this function)
Src/_ufuncmodule.c:1878: `ro' undeclared (first use in this function)
Src/_ufuncmodule.c:1908: parse error before `*'
Src/_ufuncmodule.c:1909: parse error before `*'
error: command 'gcc' failed with exit status 1


What are these packages ? 

Thanks,
Sebastian Haase

Re: [Numpy-discussion] zero dimensions in numarray.strings?

[Todd M. <jm...@st...>]

On Tue, 2003-12-02 at 04:33, Francesc Alted wrote:
> Hi,
> 
> It seems like if numarray.strings module is not able to create objects when
> one of its dimensions is zero. For example, this works:
> 
> >>> import numarray
> >>> numarray.array([], shape=(1,0,2))
> array([])
> 
> but, for strings, it raises the next error:
> 
> >>> numarray.strings.array(None, itemsize=3, shape=(1,0,2))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 955, 
> in array
>     byteoffset=byteoffset, bytestride=bytestride)
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 94, 
> in __init__
>     self.fill(" ")
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 318, 
> in fill
>     self[:] = char*self._itemsize
>   File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 505, 
> in _slicedIndexing
>     retarr._copyFrom(value)
>   File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 202, 
> in _copyFrom
>     me[:] = it[..., :self._itemsize]
>   File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 505, 
> in _slicedIndexing
>     retarr._copyFrom(value)
> libnumarray.error: copy1bytes: access beyond buffer. offset=2 buffersize=0
> 
> I think it would be nice (and consistent) to support that in strings too.

I fixed the immediate problem with zero element string arrays this
morning.  I imagine there are probably other problems with that case...
Please let me know if and when you find them.


Regards,
Todd

> 
> Thanks,
-- 
Todd Miller <jm...@st...>

[Numpy-discussion] zero dimensions in numarray.strings?

[Francesc A. <fa...@op...>]

Hi,

It seems like if numarray.strings module is not able to create objects when
one of its dimensions is zero. For example, this works:

>>> import numarray
>>> numarray.array([], shape=(1,0,2))
array([])

but, for strings, it raises the next error:

>>> numarray.strings.array(None, itemsize=3, shape=(1,0,2))
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 955, 
in array
    byteoffset=byteoffset, bytestride=bytestride)
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 94, 
in __init__
    self.fill(" ")
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 318, 
in fill
    self[:] = char*self._itemsize
  File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 505, 
in _slicedIndexing
    retarr._copyFrom(value)
  File "/usr/local/lib/python2.3/site-packages/numarray/strings.py", line 202, 
in _copyFrom
    me[:] = it[..., :self._itemsize]
  File "/usr/local/lib/python2.3/site-packages/numarray/generic.py", line 505, 
in _slicedIndexing
    retarr._copyFrom(value)
libnumarray.error: copy1bytes: access beyond buffer. offset=2 buffersize=0

I think it would be nice (and consistent) to support that in strings too.

Thanks,

-- 
Francesc Alted

[Numpy-discussion] Sending Resume

[Engineer - M. <rco...@ya...>]

Hello  David

I'm looking for new project as independent CONSULTANT

Will call you tomorrow to schedule meeting as we were talking
before, possible this week or in Monday;



 
E-mail:   rco...@ya...

 
 
Tel. (408)  309-7006

Engineer
or Manager
or
INDEPENDENT CONSULTING
or
Engineering Manager
  
and/or
 
Senior Mechanical & Design ENGINEER,
 
Design Analyst
 
Program Manager
 
Project Manager, Product Manager
 
Product Development Specialist 
 
Manufacturing Engineering
 
Electro-Mechanical & Industrial Designer 
 
R&D, CADD Manager, SBIR
 
 
Over 20 years combine experience in 
Technical Managerial Positions 
performing Engineering & Design Service
Systems Design & System Integration, 
R&D, Product & Process Development, 
Projects & Products Engineering & Management, 
Manufacturing Operations & Preparation & 
Managing teams of Engineering Specialist;
 
 
08-1993 to present;
 
MECH-TRONIC DESIGN & MFG; SANTA CLARA, CA
Engineering Manager, Project & Product Manager,
SR. Mechanical & Design Engineer, CADD Manager. 
 
 
Duties included:
 
Preparing technical documentation, calculations, 
engineering, design, layouts, drawings, 
3D and Solid Modeling, development & propositions. 
 
Hard Drive Designing, testing, balancing, 
recalculating and redesign. 
Solid Works, Mechanical Desktop.
 
Manufacturing and Assembly Equipment design and  build. 
Hard Drive Testing Equipment design & mfg.  
 
Tooling and Operations Development, 
Implementation & Automation for mass production.   
 
Pro-E, CADD Management, Automation and Operations, 
Analyzing, AutoCAD 2000 & M D, Win 95/98/2000/NT, 
Creating, revising & implementing improvements to existing
and new manufacturing processes.  
 
Mfg & Assy Development & support. 
 
Cost study and cost reduction analyzing.  
Purchasing manufacturing equipment, 
Bill of Materials creation. 
 
Copy Machines & Color commercial Printers design & build. 
Paper path & Heads - dynamic orientation. 
 
Rapid Thermo Processing, 
Wafer 200 & 300 mm with automatic Door and manual inserting,
Scanners, Sensors & Cassette opening and rotation, 
 
LASER & Equipment & Motion Control. 
Fiber Optics equipment, enclosures, connectors, 
tools & manufacturing; assembly equipment design & build. 
 
Wafer Handling & Processing Equipment, Stages & Sliders, 
Thickness Measurement, Robotics. 
Network & Electronic / Computer Testing Equipment. 
Plastic & Enclosures Design & Build; 
 
Hard Drive, modems, ICT Equipment Des, Mfg & Assy. 
 
Teaching Mechanical Engineering and  Descriptive Geometry, CADD. 
Computer Graphics, Electro - Mechanical Engineering & Design, 
 
AutoCAD, Plotting, DOS, Lecture and practical assignment. 
Software and computer support. 
 
Electrical & Mechanical Projects,  
Production Equipment & Machinery. 
 
Designing Electric Cars, Analyzes & Calculations, 
Prototypes electronic and electrical schematic Layouts, 
mechanical detailing & body shell. 
CAD and Plotting Station Management, Lib, 
Analyzes & Calculations, Prototypes & Product Mfg.
 
Development, design & redesigning mechanical components
for manufacturing, Assembly draw, 
Customizing  Electronic, Mechanical and Electrical Equipment, 
AutoCAD,  Spreadsheet, Basic, dB; Lotus, Excel, Paradox, Quadro
 
Project Management & Production, scheduling, quality control,
controlling & production planning injection molding, 
plastic parts and elements design and manufacturing 
 
 
Manufacturing hydraulic & pneumatic equipment, 
machinery and control systems, mechanisms, robotics device,
precision machine elements. 
 
Inspecting and control manufacturing standards,
analyzing stresses and tolerances, selecting materials, 
engineering computations and technical improvements, 
documentation, projects development, supplying. 
 
Automation, Conveyors, Spiral Elevator, 
Fast Cannery Transportation, 
Electronic and Electrical Components, 
Master Control Center, 
Sheet Metal Oven Rebuilding Project.
 
LASER MEDICAL DEVICE, Lasers, Optics, Fiber Optics
3 SETS OF OPTICAL LENSES MOVE-AUTOMATON COLIMMATOR
 
Project Management, Design, Product Development
 
Manufacturing, Production, Operation Manager
 
Sales & Customer Service
 
Transportation & Shop Manager
 
Service Station Operating & Management
 
 
CREATIVE THINKING AND SOLUTION DEVELOPMENT
 
OPEN FOR TRAVEL (domestic /international)
 
US Citizen
 
Permanent preferred
 
INDEPENDENT CONSULTING 
 
TEL. (408)  309-7006     E-mail: rco...@ya...

Re: [Numpy-discussion] PyArray_ContiguousFromObject & PyDECREF

[Konrad H. <hi...@cn...>]

On Monday 01 December 2003 07:15, Michiel Jan Laurens de Hoon wrote:

> In Chapter 12 ("Writing a C extension") in the Numerical Python manual,=
 in
> the first example PyArray_ContiguousFromObject is used together with
> PyDECREF, while in the last example PyArray_ContiguousFromObject is use=
d
> but the arrays are not PyDECREF'ed before the function returns. From
> Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in bo=
th
> cases. Or am I missing something here? Is PyDECREF not needed for some
> reason in the last example?

It should be there, the second example as presented has a memory leak.

Konrad.
--=20
-------------------------------------------------------------------------=
------
Konrad Hinsen                            | E-Mail: hi...@cn...
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24
Rue Charles Sadron                       | Fax:  +33-2.38.63.15.17
45071 Orleans Cedex 2                    | Deutsch/Esperanto/English/
France                                   | Nederlands/Francais
-------------------------------------------------------------------------=
------

[Numpy-discussion] PyArray_ContiguousFromObject & PyDECREF

[Michiel J. L. de H. <md...@im...>]

Dear pythoneers,

In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the 
first example PyArray_ContiguousFromObject is used together with PyDECREF, while 
in the last example PyArray_ContiguousFromObject is used but the arrays are not 
PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it 
seems that a PyDECREF is needed in both cases. Or am I missing something here? 
Is PyDECREF not needed for some reason in the last example?

Thanks in advance,

Michiel de Hoon
U Tokyo

First example:
==============
static PyObject *
trace(PyObject *self, PyObject *args)
{
   PyObject *input;
   PyArrayObject *array;
   double sum;
   int i, n;
   if (!PyArg_ParseTuple(args, "O", &input))
     return NULL;
   array = (PyArrayObject *)
           PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2);
   if (array == NULL)
     return NULL;
   n = array->dimensions[0];
   if (n > array->dimensions[1])
     n = array->dimensions[1];
   sum = 0.;
   for (i = 0; i < n; i++)
     sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]);
   Py_DECREF(array);
   return PyFloat_FromDouble(sum);
}


Last example:
=============
static PyObject *
matrix_vector(PyObject *self, PyObject *args)
{
   PyObject *input1, *input2;
   PyArrayObject *matrix, *vector, *result;
   int dimensions[1];
   double factor[1];
   double real_zero[1] = {0.};
   long int_one[1] = {1};
   long dim0[1], dim1[1];
   extern dgemv_(char *trans, long *m, long *n,
                 double *alpha, double *a, long *lda,
                 double *x, long *incx,
                 double *beta, double *Y, long *incy);
   if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2))
     return NULL;
   matrix = (PyArrayObject *)
     PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2);
   if (matrix == NULL)
     return NULL;
   vector = (PyArrayObject *)
             PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1);
   if (vector == NULL)
     return NULL;
   if (matrix->dimensions[1] != vector->dimensions[0]) {
     PyErr_SetString(PyExc_ValueError,
                     "array dimensions are not compatible");
     return NULL;
   }
   dimensions[0] = matrix->dimensions[0];
   result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE);
   if (result == NULL)
     return NULL;
   dim0[0] = (long)matrix->dimensions[0];
   dim1[0] = (long)matrix->dimensions[1];
   dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1,
          (double *)vector->data, int_one,
          real_zero, (double *)result->data, int_one);
   return PyArray_Return(result);
}


-- 
Michiel de Hoon, Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
Tokyo 108-8639
Japan
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

[Numpy-discussion] ANN: PyQwt-3.8

[Gerard V. <gve...@gr...>]

PyQwt-3.8 = FAST and EASY data plotting for Python and (Py)Qt.

PyQwt is a set of Python bindings for the Qwt C++ class library which
extends the Qt framework with widgets for scientific and engineering
applications.

It supports the use of PyQt, Qt, Qwt, the Numerical Python extensions
(either Numeric, or numarray or both) and optionally SciPy in a GUI
Python application or in an interactive Python session.

The home page of PyQwt is http://pyqwt.sourceforge.net.

Main changes in PyQwt-3.8:

1. PyQwt inherits from Qwt the new classes QwtPicker, QwtPlotPicker and
   QwtPlotZoomer.
   QwtPicker can make any widget react to events by event filtering.
   QwtPlotPicker is a QwtPicker taylored to the plot canvas.
   QwtPlotZoomer implements infinite deep zooming on the plot canvas.

2. PyQwt inherits from Qwt the support for Qt's rich text format in QwtPlot
   to display equations like E=mc<sup>2</sup>.
   Caveat: in principle, a truetype font with full unicode support is better
   to typeset equations.

3. Bug fixes to support event filtering code (see EventFilterDemo.py).

4. Conversion of a QImage to a Numeric array or numarray array.

5. Better support for cloning a plot into Grace from the command line
   interpreter friendly plot classes.

6. PyQwt-3.8 works with PyQt-3.8.1 downto -3.6 and Qt-3.2.3 downto -2.3.0.

7. License change: GPL with permission for dynamic linking to commercial,
   educational and non-commercial versions of Qt, PyQt and sip.  In this
   case PyQwt becomes a free plug-in for a non-free program.


Gerard Vermeulen

[Numpy-discussion] PyMatrix: Announcement

[Colin J. W. <cj...@sy...>]

In response to a number of requests, the PyMatrix package is now 
available for download as a zip file.

Colin W.

> PyMatrix is available for test and review.
>      http://www3.sympatico.ca/cjw
> 
> PyMatrix provides access to basic matrix arithmetic, using Python and 
> numarray.
> 
> Examples:
>           A * B              =>  the product of matrices A and B
>           A.I                =>  the inverse of matrix A
>           A.EVectors         =>  the eigenvectors of A
>           A.var(0)           =>  the variances of the columns of A
>           (a.T*a).I * a.T*b  =>  the solution (x) for a * x  =  b,
>                                      where a is a matrix and b a column vector
> 
> This package was developed on a Windows XP.  I would appreciate 
> comments, particularly with respect to usage on other systems.
> 
> Colin W.
> 

Re: [Numpy-discussion] how-to "put" RAM-based numarray into memmap

[Todd M. <jm...@st...>]

> Thanks for your reply.
> Is it possible to define a memmap slice and giving it a (preinitialized !)
> memory buffer ?

It should be now.  I just added support for a "buffer" parameter to
CVS.  I tested it manually and it worked OK for me;  please try it out
or watch for it in numarray-0.8.

> I'm thinking: I have a RAM-based numarray, I just take the buffer (pointer)
> and hand it over to the memmap-slice so that it can make the association
> between the disk-space and the RAM-space.
> I guess you are calling "heap" what I call RAM.  Is memmap using something
> inherently different that heap? (I might be missing something...)

Yes and no.   When you allocate a slice of an existing memmap, you're
getting memory backed on the disk file of your choice;  mapped memory
generally comes from a different address partition than the memory from
malloc().  When you allocate memory using numarray,  or insert a slice
into a memmap without specifying the buffer,  you're getting memory from
the heap which is backed on the system swap file.  (Of course,  this
varies a little by system as well;  I'm thinking Linux/UNIX here and
Win32 may be slightly different).

Todd

-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] how-to "put" RAM-based numarray into memmap

[Sebastian H. <ha...@ms...>]

> On Mon, 2003-11-24 at 14:13, Sebastian Haase wrote:
> > > Sebastian Haase wrote:
> > > >
> > > > Hi,
> > > > Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard)
> > > > numarray.
> > > > Now I would like to "save" that onto harddrive (with a small header
up
> > > > front
> > >
> > > How about:
> > >
> > > f = open(filename, 'wb')
> > > f.write(MyHeader)
> > > A.tofile(f)
> > >
> > > To read it back in, you need to know where your header ends, by either
> > > parsing it or using one of the same size every time, then you can use
> > > fromfile() to create an array form it.
> >
> > The main reason for my question was just to find out if NUMARRAY
supports
> > it, and how ?
> > Also I have many "bookkeeping" functions already implemented for the
> > memmap'd case.
> > (That is, I have a class with member methods operting on a member
memmapped
> > array)
> > So if what I descibed is possible I could save myself form duplicating
lot's
> > of code.
> >
> > Essentially I was hoping for the most ellegant solution ;-)
> >
>
> memmap's Memmap class does support an insert() method for adding a new
> slice to the end of (or anywhere in) an existing map.  The new slice,
> however,  will exist as a block of memory allocated on the heap until
> the memmap is saved to disk.
>
> Thus, two scenarios present themselves: (1) you allocate the new slice
> ahead of time and create an array from it, avoiding data duplication (2)
> you create an array and later copy it into (a newly inserted slice of)
> the memmap, thereby duplicating your data on the heap.
>
> When you close the map,  slices on the heap are written to the map file.
>
> Todd
>

Thanks for your reply.
Is it possible to define a memmap slice and giving it a (preinitialized !)
memory buffer ?
I'm thinking: I have a RAM-based numarray, I just take the buffer (pointer)
and hand it over to the memmap-slice so that it can make the association
between the disk-space and the RAM-space.
I guess you are calling "heap" what I call RAM.  Is memmap using something
inherently different that heap? (I might be missing something...)
As I was trying to illustrate in my example, my ram-numarray might already
be using most of the available address space.

Thanks,
Sebastian

[Numpy-discussion] Re: OT:pretty print of reals (was: Community Poll: numarray default underflow handling == "ignore" ?)

[Alexander S. <a.s...@gm...>]

"Sebastian Haase" <ha...@ms...> writes:

> Hi Alexander,
> Thanks for the code - I will look into it
> I just read a few lines of your MatrixFormatter class seeing the definition
> of __repr__.
> 
> Here is a thought: Is it possible that I overload the __repr__ of float
> (that is: the buildin data type) !?

Nope. You can't screw around with methods of builtins -- and this is a (quite
sensible) conscious design decision.

> When I use pyCrust it automatically "fills in" default arguments like,
> func(a=0.1000000000001)
> So, this probably does not call the sys.displayhook - so __repr__ itself
> would need to be changed.

Nope, whatever pycrust uses needs to be changed (are you *sure* changing
sys.displayhook has no effect on pycrust (it might not change what it "fills
in", but what about actual output?) because that would seem sort of ...
suboptimal. Even so it shouldn't be too difficult to figure out how you can
customize printing in pycrust (by having a look at the pycrust code and/or
reading the manual or asking on the appropriate mailing-list).

> Do you know if that is possible ? (It would recursively fix all list and
> matrix display problems I had ...)

It wouldn't, it would just get you the float and list (tuple etc.) display you
want (arrays/matrices don't call repr on each individual array arguments to
compute their overall repr).


HTH,

'as

Re: [Numpy-discussion] how-to "put" RAM-based numarray into memmap

[Todd M. <jm...@st...>]

On Mon, 2003-11-24 at 14:13, Sebastian Haase wrote:
> > Sebastian Haase wrote:
> > >
> > > Hi,
> > > Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard)
> > > numarray.
> > > Now I would like to "save" that onto harddrive (with a small header up
> > > front
> >
> > How about:
> >
> > f = open(filename, 'wb')
> > f.write(MyHeader)
> > A.tofile(f)
> >
> > To read it back in, you need to know where your header ends, by either
> > parsing it or using one of the same size every time, then you can use
> > fromfile() to create an array form it.
> 
> The main reason for my question was just to find out if NUMARRAY supports
> it, and how ?
> Also I have many "bookkeeping" functions already implemented for the
> memmap'd case.
> (That is, I have a class with member methods operting on a member memmapped
> array)
> So if what I descibed is possible I could save myself form duplicating lot's
> of code.
> 
> Essentially I was hoping for the most ellegant solution ;-)
> 

memmap's Memmap class does support an insert() method for adding a new
slice to the end of (or anywhere in) an existing map.  The new slice,
however,  will exist as a block of memory allocated on the heap until
the memmap is saved to disk.  

Thus, two scenarios present themselves: (1) you allocate the new slice
ahead of time and create an array from it, avoiding data duplication (2)
you create an array and later copy it into (a newly inserted slice of)
the memmap, thereby duplicating your data on the heap.

When you close the map,  slices on the heap are written to the map file.

Todd

> Thanks,
> Sebastian
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller <jm...@st...>

Re: [Numpy-discussion] how-to "put" RAM-based numarray into memmap

[Sebastian H. <ha...@ms...>]

> Sebastian Haase wrote:
> >
> > Hi,
> > Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard)
> > numarray.
> > Now I would like to "save" that onto harddrive (with a small header up
> > front
>
> How about:
>
> f = open(filename, 'wb')
> f.write(MyHeader)
> A.tofile(f)
>
> To read it back in, you need to know where your header ends, by either
> parsing it or using one of the same size every time, then you can use
> fromfile() to create an array form it.

The main reason for my question was just to find out if NUMARRAY supports
it, and how ?
Also I have many "bookkeeping" functions already implemented for the
memmap'd case.
(That is, I have a class with member methods operting on a member memmapped
array)
So if what I descibed is possible I could save myself form duplicating lot's
of code.

Essentially I was hoping for the most ellegant solution ;-)

Thanks,
Sebastian

[Numpy-discussion] OT:pretty print of reals (was: Community Poll: numarray default underflow handling == "ignore" ?)

[Sebastian H. <ha...@ms...>]

Hi Alexander,
Thanks for the code - I will look into it
I just read a few lines of your MatrixFormatter class seeing the definition
of __repr__.

Here is a thought: Is it possible that I overload the __repr__ of float
(that is: the buildin data type) !?
When I use pyCrust it automatically "fills in" default arguments like,
func(a=0.1000000000001)
So, this probably does not call the sys.displayhook - so __repr__ itself
would need to be changed.
Do you know if that is possible ? (It would recursively fix all list and
matrix display problems I had ...)

Thanks again,
Sebastian


----- Original Message ----- 
From: "Alexander Schmolck" <a.s...@gm...>
To: "Sebastian Haase" <ha...@ms...>
Cc: <num...@li...>
Sent: Friday, November 21, 2003 7:23 PM
Subject: Re: [Numpy-discussion] Community Poll: numarray default underflow
handling == "ignore" ?


> "Sebastian Haase" <ha...@ms...> writes:
>
> > Hi Alexander,
> > Thanks for your reply  and yes, I would be very interested to see more
of
> > your "pretty print" code.
>
> Sure, I'll send you the code. As it is for a matrix class it only handles
up
> to (array-)rank 2 currently (and uses methods of the matrix class) but I
don't
> think changing either of these should be too hard.
>
> > I am trying to put together all kinds of useful things to make python
( with
> > numarray and scipy) a possible replacement for Matlab...
>
> <grin> Been, there done that -- actually still doing it.
>
> When I was faced with the choice between matlab and python for my
numerical
> work, I decided that despite the convince matlab offered for many things
> (libraries, interactive environment) I couldn't face the its monstrosity
as a
> programming language and thus opted for python.
>
> These were the main areas were I found python/Numeric was lagging behind:
>
> 1. Numeric.dot was much too slow (factor 40-100 and more for large arrays)
> 2. Although Numeric's arrays are quite general and powerful, I found
writing
>    down simple matrix expressions (dot(transpose(X), dot(...))) and the
visual
>    clarity of the repr of the result less attractive than matlab's.
> 3. Library support (e.g. in some fields many academics tend to make matlab
>    code corresponding to their articles available, which is very handy to
>    quickly try something).
> 4. Matlab came with very good plotting capabilities, whereas python's
support
>    for plotting was quite poor when I started using it.
> 5. Matlab also had a pretty good interactive environment.
>
> So far I've managed to come up with the following (sometimes partial)
> solutions for the above problems.
>
> 1. An optional fast dot product using blas (although I somehow still ended
up
>    spending a few days on it, all the real work was done by my supervisor
>    Richard Everson -- I've submitted a patch that made it into Numeric 23
but
>    I don't think numarray has an the option of a fast dot product yet,
>    although I haven't checked).
> 2. A powerful matrix class (mostly finished; should transparently support
>    Numeric as well as numarray (the latter is not fully there yet),
>    unreleased).
> 3. A high-level matlab interface (pretty much ready to be released).
> 4. A convenient and high-level plotting library that supports multiple
>    backends (currently only proper support for matlab as backend although
I
>    started working on/contemplating others (e.g. grace, gnuplot and
chaco);
>    unreleased)
> 5. IPython (kudos to Fernando!) + emacs + a few convenience functions for
>    things like saving variables and redefining classes makes a darn nice
>    environment once one got the hang of it (unfortunately it meant I had
to
>    write the emacs support for ipython, it's available on the ipython
>    website).
>
> Of course if anyone wants to lend a hand, he/she is more than welcome ;)
>
> > I am not hoping for a similarly big documentation and help system, but
just
>
> Why not? Nowadays pretty much all python code comes with pretty good
> docstrings and implementing something like 'lookfor' shouldn't be
particularly
> hard. Already looking up docs and jumping to code in ipython is quite
> convenient (e.g. ``some_func?``).
>
> > the fact that it's free ( doesn't need to connect to a "license server"
> > every other minute)  and that it happily supports single-precision
arrays
> > ( not to mention memmap array !)   should make it interesting.
>
> And that it is not based on an abomination of a programming language but
an
> excellent one, with plenty of high quality general purpose libs (and even
if
> only doing numerics is concerned: Numeric/numarray's (APL-ish) design also
> doesn't compare badly to matlab, IMHO).
>
> > I know that I probably should use IPython,
>
> Indeed, everyone should (from within emacs ;)
>
> 'as
>
>

Re: [Numpy-discussion] PyMatrix: Announcement

[Nadav H. <na...@Vi...>]

Don't see how can I get the sources for gnu/linux installation.

 Nadav


On Mon, 2003-11-24 at 15:17, Colin J. Williams wrote:
> PyMatrix is available for test and review.
>      http://www3.sympatico.ca/cjw
> 
> PyMatrix provides access to basic matrix arithmetic, using Python and 
> numarray.
> 
> Examples:
>           A * B                   =>               the product of 
> matrices A and B
>           A.I                       =>               the inverse of matrix A
>           A.EVectors           =>               the eigenvectors of A
>           A.var(0)               =>               the variances of the 
> columns of A
>           (a.T*a).I * a.T*b  =>               the solution (x) for a * 
> x  =  b,
>                                                         where a is a 
> matrix and b a column vector
> 
> This package was developed on a Windows XP.  I would appreciate 
> comments, particularly with respect to usage on other systems.
> 
> Colin W.
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

[Numpy-discussion] PyMatrix: Announcement

[Colin J. W. <cj...@sy...>]

PyMatrix is available for test and review.
     http://www3.sympatico.ca/cjw

PyMatrix provides access to basic matrix arithmetic, using Python and 
numarray.

Examples:
          A * B                   =>               the product of 
matrices A and B
          A.I                       =>               the inverse of matrix A
          A.EVectors           =>               the eigenvectors of A
          A.var(0)               =>               the variances of the 
columns of A
          (a.T*a).I * a.T*b  =>               the solution (x) for a * 
x  =  b,
                                                        where a is a 
matrix and b a column vector

This package was developed on a Windows XP.  I would appreciate 
comments, particularly with respect to usage on other systems.

Colin W.

[Numpy-discussion] Bulk Friendly Web Hosting & Server

[Hosting <Bul...@29...>]

Dear Sales:

We offer reliable bulk email friendly web hosting services. You can now 
have the peace of mind knowing that your web site is secure during your 
email marketing campaigns. 

Bullet Proof Web Hosting 100% Guaranteed! We guarantee that your site
will be 99% uptime. 

                   Hosting Features:

                   Your own Web address:  www.YourName.com 

                   Dedicated 100 M fiber
                  100MB of disk space 
                   Unlimited Data Transfer
                   FTP Support 
                   FrontPage Support

                   ASP Support 
                   PERL Support 
                   PHP Support
                   CGI Support

                   No Setup Fee
                   US$199.00/month


Bullet-Proof dedicated servers:

                   1. 

                   Two IPs
                  1024MB RAM DDR
                   PIIII / Two CPU
                   120 GB SCS
                   Intel 82559 LAN /(2U)
                   Unlimited Data Transfer
                   Linux / Windows / FreeBSD

                   Price:   No setup fee 
                                US$ 599.00/month


                   2.   

                   Dynamic  IP
                  1024MB RAM DDR
                   PIIII / Two CPU
                   120 GB SCS
                   Intel 82559 LAN /(2U)
                   Unlimited Data Transfer
                   Linux / Windows / FreeBSD

                   Price:   No setup fee 
                                US$ 999.00/month


Allowed Usage
You can use the server for any of the following:

Direct Bulk Mailing or Proxy Mailing 
Web Site Hosting 
Proxy, Relay or Port Scanning

Restrictions & Rules
Our Plans do NOT include the following:

No Adult Content 
No Illegal Material


We aslo offer targeted email addresses and send out targeted emails for 
you.

We looking forward to serving you in the near future.


Cheers!

Betty Jones
Support Teams
Su...@bi...
Http://www.biz-servers.com