numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] build problems on Solaris

[Mathew Y. <my...@jp...>]

Heres the problem.... The function get_flags_linker_so in 
numpy/distutils/fcompiler/gnu.py is not called anywhere. Because of 
this, g2c is not added as a library and  -mimpure-text is not set. This 
causes the "s_wsfe unresolved"  problem.

Anybody know how to fix this?

Mathew

Re: [Numpy-discussion] Back to numexpr

[Tim H. <tim...@co...>]

Francesc Altet wrote:

>A Dimarts 13 Juny 2006 20:46, Tim Hochberg va escriure:
>  
>
>>>Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre for
>>>some users (specially in 32-bit platforms), is a type with the same rights
>>>than the others and we would like to give support for it in numexpr. In
>>>fact, Ivan Vilata already has implemented this suport in our local copy
>>>of numexpr, so perhaps (I say perhaps because we are in the middle of a
>>>big project now and are a bit scarce of time resources) we can provide
>>>the patch against the latest version of David for your consideration.
>>>With this we can solve the problem with int64 support in 32-bit platforms
>>>(although addmittedly, the VM gets a bit more complicated, I really think
>>>that this is worth the effort)
>>>      
>>>
>>In addition to complexity, I worry that we'll overflow the code cache at
>>some point and slow everything down. To be honest I have no idea at what
>>point that is likely to happen, but I know they worry about it with the
>>Python interpreter mainloop.
>>    
>>
>
>That's true. I didn't think about this :-/
>
>  
>
>>Also, it becomes much, much slower to 
>>compile past a certain number of case statements under VC7, not sure
>>why. That's mostly my problem though.
>>    
>>
>
>No, this is a general problem (I'd say much more in GCC, because the optimizer 
>runs so slooooow). However, this should only affect to poor developers, not 
>users and besides, we should find a solution for int64 in 32-bit platforms.
>  
>
Yeah. This is just me whining. Under VC7, there is a very sudden change 
when adding more cases where compile times go from seconds to minutes. I 
think we're already past that now anyway, so slowing that down more 
isn't going to hurt me. Overflowing the cache is the real thing I worry 
about.

>>One idea that might be worth trying for int64 is to special case them
>>using functions. That is using OP_FUNC_LL and OP_FUNC_LLL and some
>>casting opcodes. This could support int64 with relatively few new
>>opcodes. There's obviously some exta overhead introduced here by the
>>function call. How much this matters is probably a function of how well
>>the compiler / hardware supports int64 to begin with.
>>    
>>
>
>Mmm, in my experience int64 operations are reasonable well supported by modern 
>32-bit processors (IIRC they normally take twice of the time than int32 ops).
>
>The problem with using a long for representing ints in numexpr is that we have 
>the duality of being represented differently in 32/64-bit platforms and that 
>could a headache in the long term (int64 support in 32-bit platforms is only 
>one issue, but there should be more). IMHO, it is much better to assign the 
>role for ints in numexpr to a unique datatype, and this should be int64, for 
>the sake of wide int64 support, but also for future (and present!) 64-bit 
>processors. The problem would be that operations with 32-bit ints in 32-bit 
>processors can be slowed-down by a factor 2x (or more, because there is a 
>casting now), but in exchange, whe have full portable code and int64 support. 
>  
>
This certainly makes things simpler. I think that this would be fine 
with me since I mostly use float and complex, so the speed issue 
wouldn't hit me much. But that's 'cause I'm selfish that way ;-)

>In case we consider entering this way, we have two options here: keep VM 
>simple and advertise that int32 arithmetic in numexpr in 32-bit platforms 
>will be sub-optimal, or, as we already have done, add the proper machinery to 
>support both integer separately (at the expense of making the VM more 
>complex).  Or perhaps David can come with a better solution (vmgen from 
>gforth? no idea what this is, but the name sounds sexy;-) 
>  
>
Yeah!

>>That brings up another point. We probably don't want to have casting
>>opcodes from/to everything. Given that there are 8 types on the table
>>now, if we support every casting opcode we're going to have 56(?)
>>opcodes just for casting. I imagine what we'll have to do is write a
>>cast from int16 to float as OP_CAST_Ii; OP_CAST_FI; trading an extra
>>step in these cases for keeping the number of casting opcodes under
>>control. Once again, int64 is problematic since you lose precision
>>casting to int. I guess in this case you could get by with being able to
>>cast back and forth to float and int. No need to cast directly to
>>booleans, etc as two stage casting should suffice for this.
>>    
>>
>
>Well, we already thought about this. Not only you can't safely cast an int64 
>to an int32 without loosing precistion, but what is worse, you can't even 
>cast it to any other commonly available datatype (casting to a float64 will 
>also loose precision). And, although you can afford loosing precision when 
>dealing with floating data in some scenarios (but not certainly with a 
>general-purpose library like numexpr tries to be), it is by any means 
>unacceptable loosing 'precision' in ints. So, to my mind, the only solution 
>is completely avoiding casting int64 to any type.
>  
>
I forgot that the various OP_CAST_xy opcodes only do safe casting. That 
makes the number of potential casts much less, so I guess this is not as 
big a deal as I thought. I'm still not sure, for instance, if we need 
boolean to int16, int32, int64, float32, float64, complex64 and 
complex128. It wouldn't kill us, but it's probably overkill.

-tim

Re: [Numpy-discussion] Back to numexpr

[David M. C. <co...@ph...>]

On Tue, 13 Jun 2006 21:30:41 +0200
Francesc Altet <fa...@ca...> wrote:

> A Dimarts 13 Juny 2006 20:46, Tim Hochberg va escriure:
> > >Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre
> > >for some users (specially in 32-bit platforms), is a type with the same
> > >rights than the others and we would like to give support for it in
> > >numexpr. In
> > > fact, Ivan Vilata already has implemented this suport in our local copy
> > > of numexpr, so perhaps (I say perhaps because we are in the middle of a
> > > big project now and are a bit scarce of time resources) we can provide
> > > the patch against the latest version of David for your consideration.
> > > With this we can solve the problem with int64 support in 32-bit
> > > platforms (although addmittedly, the VM gets a bit more complicated, I
> > > really think that this is worth the effort)
> >
> > In addition to complexity, I worry that we'll overflow the code cache at
> > some point and slow everything down. To be honest I have no idea at what
> > point that is likely to happen, but I know they worry about it with the
> > Python interpreter mainloop.
> 
> That's true. I didn't think about this :-/
> 
> > Also, it becomes much, much slower to 
> > compile past a certain number of case statements under VC7, not sure
> > why. That's mostly my problem though.
> 
> No, this is a general problem (I'd say much more in GCC, because the
> optimizer runs so slooooow). However, this should only affect to poor
> developers, not users and besides, we should find a solution for int64 in
> 32-bit platforms.

If I switch to vmgen, it can easily make two versions of the code: one using a
case statement, and another direct-threaded version for GCC (which supports
taking the address of a label, and doing a 'goto' to a variable). Won't solve
the I-cache problem, though. And there's always subroutine threading (each
opcode is a function, and the program is a list of function pointers).

We won't know until we try :)

> Or perhaps
> David can come with a better solution (vmgen from gforth? no idea what this
> is, but the name sounds sexy;-) 

The docs for it are at
http://www.complang.tuwien.ac.at/anton/vmgen/html-docs/

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] Back to numexpr

[Francesc A. <fa...@ca...>]

A Dimarts 13 Juny 2006 20:46, Tim Hochberg va escriure:
> >Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre f=
or
> >some users (specially in 32-bit platforms), is a type with the same righ=
ts
> >than the others and we would like to give support for it in numexpr. In
> > fact, Ivan Vilata already has implemented this suport in our local copy
> > of numexpr, so perhaps (I say perhaps because we are in the middle of a
> > big project now and are a bit scarce of time resources) we can provide
> > the patch against the latest version of David for your consideration.
> > With this we can solve the problem with int64 support in 32-bit platfor=
ms
> > (although addmittedly, the VM gets a bit more complicated, I really thi=
nk
> > that this is worth the effort)
>
> In addition to complexity, I worry that we'll overflow the code cache at
> some point and slow everything down. To be honest I have no idea at what
> point that is likely to happen, but I know they worry about it with the
> Python interpreter mainloop.

That's true. I didn't think about this :-/

> Also, it becomes much, much slower to=20
> compile past a certain number of case statements under VC7, not sure
> why. That's mostly my problem though.

No, this is a general problem (I'd say much more in GCC, because the optimi=
zer=20
runs so slooooow). However, this should only affect to poor developers, not=
=20
users and besides, we should find a solution for int64 in 32-bit platforms.

> One idea that might be worth trying for int64 is to special case them
> using functions. That is using OP_FUNC_LL and OP_FUNC_LLL and some
> casting opcodes. This could support int64 with relatively few new
> opcodes. There's obviously some exta overhead introduced here by the
> function call. How much this matters is probably a function of how well
> the compiler / hardware supports int64 to begin with.

Mmm, in my experience int64 operations are reasonable well supported by mod=
ern=20
32-bit processors (IIRC they normally take twice of the time than int32 ops=
).

The problem with using a long for representing ints in numexpr is that we h=
ave=20
the duality of being represented differently in 32/64-bit platforms and tha=
t=20
could a headache in the long term (int64 support in 32-bit platforms is onl=
y=20
one issue, but there should be more). IMHO, it is much better to assign the=
=20
role for ints in numexpr to a unique datatype, and this should be int64, fo=
r=20
the sake of wide int64 support, but also for future (and present!) 64-bit=20
processors. The problem would be that operations with 32-bit ints in 32-bit=
=20
processors can be slowed-down by a factor 2x (or more, because there is a=20
casting now), but in exchange, whe have full portable code and int64 suppor=
t.=20
In case we consider entering this way, we have two options here: keep VM=20
simple and advertise that int32 arithmetic in numexpr in 32-bit platforms=20
will be sub-optimal, or, as we already have done, add the proper machinery =
to=20
support both integer separately (at the expense of making the VM more=20
complex).  Or perhaps David can come with a better solution (vmgen from=20
gforth? no idea what this is, but the name sounds sexy;-)=20

>
> That brings up another point. We probably don't want to have casting
> opcodes from/to everything. Given that there are 8 types on the table
> now, if we support every casting opcode we're going to have 56(?)
> opcodes just for casting. I imagine what we'll have to do is write a
> cast from int16 to float as OP_CAST_Ii; OP_CAST_FI; trading an extra
> step in these cases for keeping the number of casting opcodes under
> control. Once again, int64 is problematic since you lose precision
> casting to int. I guess in this case you could get by with being able to
> cast back and forth to float and int. No need to cast directly to
> booleans, etc as two stage casting should suffice for this.

Well, we already thought about this. Not only you can't safely cast an int6=
4=20
to an int32 without loosing precistion, but what is worse, you can't even=20
cast it to any other commonly available datatype (casting to a float64 will=
=20
also loose precision). And, although you can afford loosing precision when=
=20
dealing with floating data in some scenarios (but not certainly with a=20
general-purpose library like numexpr tries to be), it is by any means=20
unacceptable loosing 'precision' in ints. So, to my mind, the only solution=
=20
is completely avoiding casting int64 to any type.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Back to numexpr

[Tim H. <tim...@co...>]

Francesc Altet wrote:

>Ei, numexpr seems to be back, wow! :-D
>
>A Dimarts 13 Juny 2006 18:56, Tim Hochberg va escriure:
>  
>
>>I've finally got around to looking at numexpr again. Specifically, I'm
>>looking at Francesc Altet's numexpr-0.2, with the idea of harmonizing
>>the two versions. Let me go through his list of enhancements and comment
>>(my comments are dedented):
>>    
>>
>
>Well, as David already said, he committed most of my additions some days 
>ago :-)
>
>  
>
>>    - Enhanced performance for strided and unaligned data, specially for
>>    lightweigth computations (e.g. 'a>10'). With this and the addition of
>>    the boolean type, we can get up to 2x better times than previous
>>    versions. Also, most of the supported computations goes faster than
>>    with numpy or numarray, even the simplest one.
>>
>>Francesc, if you're out there, can you briefly describe what this
>>support consists of? It's been long enough since I was messing with this
>>that it's going to take me a while to untangle NumExpr_run, where I
>>expect it's lurking, so any hints would be appreciated.
>>    
>>
>
>This is easy. When dealing with strided or unaligned vectors, instead of 
>copying them completely to well-behaved arrays, they are copied only when the 
>virtual machine needs the appropriate blocks. With this, there is no need to 
>write the well-behaved array back into main memory, which can bring an 
>important bottleneck, specially when dealing with large arrays. This allows a 
>better use of the processor caches because data is catched and used only when 
>the VM needs it. Also, I see that David has added support for byteswapped 
>arrays, which is great! 
>  
>
I'm looking at this now. I imagine it will become clear eventually. I've 
clearly forgotten some stuff over the last few months. Sigh.

First I need to get it to compile here. It seems that a few GCCisms have 
crept back in.


[SNIP]

>>rarely used.
>>    
>>
>
>Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre for 
>some users (specially in 32-bit platforms), is a type with the same rights 
>than the others and we would like to give support for it in numexpr. In fact, 
>Ivan Vilata already has implemented this suport in our local copy of numexpr, 
>so perhaps (I say perhaps because we are in the middle of a big project now 
>and are a bit scarce of time resources) we can provide the patch against the 
>latest version of David for your consideration. With this we can solve the 
>problem with int64 support in 32-bit platforms (although addmittedly, the VM 
>gets a bit more complicated, I really think that this is worth the effort)
>  
>
In addition to complexity, I worry that we'll overflow the code cache at 
some point and slow everything down. To be honest I have no idea at what 
point that is likely to happen, but I know they worry about it with the 
Python interpreter mainloop. Also, it becomes much, much slower to 
compile past a certain number of case statements under VC7, not sure 
why. That's mostly my problem though.

One idea that might be worth trying for int64 is to special case them 
using functions. That is using OP_FUNC_LL and OP_FUNC_LLL and some 
casting opcodes. This could support int64 with relatively few new 
opcodes. There's obviously some exta overhead introduced here by the 
function call. How much this matters is probably a function of how well 
the compiler / hardware supports int64 to begin with.

That brings up another point. We probably don't want to have casting 
opcodes from/to everything. Given that there are 8 types on the table 
now, if we support every casting opcode we're going to have 56(?) 
opcodes just for casting. I imagine what we'll have to do is write a 
cast from int16 to float as OP_CAST_Ii; OP_CAST_FI; trading an extra 
step in these cases for keeping the number of casting opcodes under 
control. Once again, int64 is problematic since you lose precision 
casting to int. I guess in this case you could get by with being able to 
cast back and forth to float and int. No need to cast directly to 
booleans, etc as two stage casting should suffice for this.

-tim

Re: [Numpy-discussion] Back to numexpr

[Francesc A. <fa...@ca...>]

A Dimarts 13 Juny 2006 19:47, Francesc Altet va escriure:
> >     - Support for both numpy and numarray (use the flag --force-numarray
> >     in setup.py).
> >
> > At first glance this looks like it doesn't make things to messy, so I'm
> > in favor of incorporating this.
>
> Yeah. I thing you are right. It's only that we need this for our own thin=
gs
> :)

Ooops! small correction here. I thought that you were saying that you were=
=20
*not* in favour of supporting numarray as well, but you clearly was. Sorry=
=20
about the misunderstanding.

Anyway, if David's idea of providing a thin numpy-compatible numarray layer=
 is=20
easy to implement, then great.

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Back to numexpr

[Francesc A. <fa...@ca...>]

Ei, numexpr seems to be back, wow! :-D

A Dimarts 13 Juny 2006 18:56, Tim Hochberg va escriure:
> I've finally got around to looking at numexpr again. Specifically, I'm
> looking at Francesc Altet's numexpr-0.2, with the idea of harmonizing
> the two versions. Let me go through his list of enhancements and comment
> (my comments are dedented):

Well, as David already said, he committed most of my additions some days=20
ago :-)

>     - Enhanced performance for strided and unaligned data, specially for
>     lightweigth computations (e.g. 'a>10'). With this and the addition of
>     the boolean type, we can get up to 2x better times than previous
>     versions. Also, most of the supported computations goes faster than
>     with numpy or numarray, even the simplest one.
>
> Francesc, if you're out there, can you briefly describe what this
> support consists of? It's been long enough since I was messing with this
> that it's going to take me a while to untangle NumExpr_run, where I
> expect it's lurking, so any hints would be appreciated.

This is easy. When dealing with strided or unaligned vectors, instead of=20
copying them completely to well-behaved arrays, they are copied only when t=
he=20
virtual machine needs the appropriate blocks. With this, there is no need t=
o=20
write the well-behaved array back into main memory, which can bring an=20
important bottleneck, specially when dealing with large arrays. This allows=
 a=20
better use of the processor caches because data is catched and used only wh=
en=20
the VM needs it. Also, I see that David has added support for byteswapped=20
arrays, which is great!=20

>     - Support for both numpy and numarray (use the flag --force-numarray
>     in setup.py).
>
> At first glance this looks like it doesn't make things to messy, so I'm
> in favor of incorporating this.

Yeah. I thing you are right. It's only that we need this for our own things=
 :)

>     - Add types for int16, int64 (in 32-bit platforms), float32,
>       complex64 (simple prec.)
>
> I have some specific ideas about how this should be accomplished.
> Basically, I don't think we want to support every type in the same way,
> since this is going to make the case statement blow up to an enormous
> size. This may slow things down and at a minimum it will make things
> less comprehensible. My thinking is that we only add casts for the extra
> types and do the computations at high precision. Thus adding two int16
> numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then
> a OP_CAST_fF.  The details are left as an excercise to the reader ;-).
> So, adding int16, float32, complex64 should only require the addition of
> 6 casting opcodes plus appropriate modifications to the compiler.
>
> For large arrays, this should have most of the benfits of giving each
> type it's own opcode, since the memory bandwidth is still small, while
> keeping the interpreter relatively simple.

Yes, I like the idea as well.

> Unfortunately, int64 doesn't fit under this scheme; is it used enough to
> matter? I hate pile a whole pile of new opcodes on for something that's
> rarely used.

Uh, I'm afraid that yes. In PyTables, int64, while being a bit bizarre for=
=20
some users (specially in 32-bit platforms), is a type with the same rights=
=20
than the others and we would like to give support for it in numexpr. In fac=
t,=20
Ivan Vilata already has implemented this suport in our local copy of numexp=
r,=20
so perhaps (I say perhaps because we are in the middle of a big project now=
=20
and are a bit scarce of time resources) we can provide the patch against th=
e=20
latest version of David for your consideration. With this we can solve the=
=20
problem with int64 support in 32-bit platforms (although addmittedly, the V=
M=20
gets a bit more complicated, I really think that this is worth the effort).

Cheers,

=2D-=20
>0,0<   Francesc Altet =A0 =A0 http://www.carabos.com/
V   V   C=E1rabos Coop. V. =A0=A0Enjoy Data
 "-"

Re: [Numpy-discussion] Back to numexpr

[Tim H. <tim...@co...>]

David M. Cooke wrote:

>On Tue, Jun 13, 2006 at 09:56:37AM -0700, Tim Hochberg wrote:
>  
>
>>[SNIP]
>>    
>>
>
>All the above is checked in already :-)
>  
>
So I see. Oops!

>  
>
>>   - Support for both numpy and numarray (use the flag --force-numarray
>>   in setup.py).
>>
>>At first glance this looks like it doesn't make things to messy, so I'm 
>>in favor of incorporating this.
>>    
>>
>
>... although I had ripped this all out. I'd rather have a numpy-compatible
>numarray layer (at the C level, this means defining macros like PyArray_DATA)
>than different code for each.
>  
>
Okey dokey. I don't feel strongly about this either way other than I'd 
rather have one version of numexpr around rather than two almost 
identical versions. Whatever makes that work would makes me happy.

>  
>
>>   - Added a new benchmark for testing boolean expressions and
>>   strided/unaligned arrays: boolean_timing.py
>>
>>Benchmarks are always good.
>>    
>>
>
>Haven't checked that in yet.
>
>  
>
>>   Things that I want to address in the future:
>>
>>   - Add tests on strided and unaligned data (currently only tested
>>   manually)
>>
>>Yep! Tests are good.
>>
>>   - Add types for int16, int64 (in 32-bit platforms), float32,
>>     complex64 (simple prec.)
>>
>>I have some specific ideas about how this should be accomplished. 
>>Basically, I don't think we want to support every type in the same way, 
>>since this is going to make the case statement blow up to an enormous 
>>size. This may slow things down and at a minimum it will make things 
>>less comprehensible.
>>    
>>
>
>I've been thinking how to generate the virtual machine programmatically,
>specifically I've been looking at vmgen from gforth again. I've got other
>half-formed ideas too (separate scalar machine for reductions?) that I'm
>working on too.
>
>But yes, the # of types does make things harder to redo :-)
>
>  
>
>>My thinking is that we only add casts for the extra 
>>types and do the computations at high precision. Thus adding two int16 
>>numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then 
>>a OP_CAST_fF.  The details are left as an excercise to the reader ;-). 
>>So, adding int16, float32, complex64 should only require the addition of 
>>6 casting opcodes plus appropriate modifications to the compiler.
>>    
>>
>
>My thinking too.
>  
>
Yeah! Although I'm not in a hurry on this part.

 I'm remembering now that the next item on my agenda was to work on 
supporting broadcasting. I don't exactly know how this is going to work, 
although I recall having something of a plan at some point. Perhaps the 
easiest way to start out is to just test the shapes of the input array 
for compatibility. If they're compatible and don't require broadcasting, 
proceed as now. If they are incompatible, raise a "ValueError: shape 
mismatch: objects cannot be broadcast to a single shape" as numpy does. 
If they are compatible, but require broadcasting, raise a 
NotImplementedError. This should be relatively easy and makes the 
numexpr considerably more congruent with numpy.

I'm hoping that, while working on that, my plan will pop back into my 
head ;-)


[SNIP]

Regards,

-tim

Re: [Numpy-discussion] Back to numexpr

[David M. C. <co...@ph...>]

On Tue, Jun 13, 2006 at 09:56:37AM -0700, Tim Hochberg wrote:
> 
> I've finally got around to looking at numexpr again. Specifically, I'm 
> looking at Francesc Altet's numexpr-0.2, with the idea of harmonizing 
> the two versions. Let me go through his list of enhancements and comment 
> (my comments are dedented):
> 
>    - Addition of a boolean type. This allows better array copying times
>    for large arrays (lightweight computations ara typically bounded by
>    memory bandwidth).
> 
> Adding this to numexpr looks like a no brainer. Behaviour of booleans 
> are different than integers, so in addition to being more memory 
> efficient, this enables boolean &, |, ~, etc to work properly.
> 
>    - Enhanced performance for strided and unaligned data, specially for
>    lightweigth computations (e.g. 'a>10'). With this and the addition of
>    the boolean type, we can get up to 2x better times than previous
>    versions. Also, most of the supported computations goes faster than
>    with numpy or numarray, even the simplest one.
> 
> Francesc, if you're out there, can you briefly describe what this 
> support consists of? It's been long enough since I was messing with this 
> that it's going to take me a while to untangle NumExpr_run, where I 
> expect it's lurking, so any hints would be appreciated.
> 
>    - Addition of ~, & and | operators (a la numarray.where)
> 
> Sounds good.

All the above is checked in already :-)

>    - Support for both numpy and numarray (use the flag --force-numarray
>    in setup.py).
> 
> At first glance this looks like it doesn't make things to messy, so I'm 
> in favor of incorporating this.

... although I had ripped this all out. I'd rather have a numpy-compatible
numarray layer (at the C level, this means defining macros like PyArray_DATA)
than different code for each.

>    - Added a new benchmark for testing boolean expressions and
>    strided/unaligned arrays: boolean_timing.py
> 
> Benchmarks are always good.

Haven't checked that in yet.

> 
>    Things that I want to address in the future:
> 
>    - Add tests on strided and unaligned data (currently only tested
>    manually)
> 
> Yep! Tests are good.
> 
>    - Add types for int16, int64 (in 32-bit platforms), float32,
>      complex64 (simple prec.)
> 
> I have some specific ideas about how this should be accomplished. 
> Basically, I don't think we want to support every type in the same way, 
> since this is going to make the case statement blow up to an enormous 
> size. This may slow things down and at a minimum it will make things 
> less comprehensible.

I've been thinking how to generate the virtual machine programmatically,
specifically I've been looking at vmgen from gforth again. I've got other
half-formed ideas too (separate scalar machine for reductions?) that I'm
working on too.

But yes, the # of types does make things harder to redo :-)

> My thinking is that we only add casts for the extra 
> types and do the computations at high precision. Thus adding two int16 
> numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then 
> a OP_CAST_fF.  The details are left as an excercise to the reader ;-). 
> So, adding int16, float32, complex64 should only require the addition of 
> 6 casting opcodes plus appropriate modifications to the compiler.

My thinking too.

> For large arrays, this should have most of the benfits of giving each 
> type it's own opcode, since the memory bandwidth is still small, while 
> keeping the interpreter relatively simple.
> 
> Unfortunately, int64 doesn't fit under this scheme; is it used enough to 
> matter? I hate pile a whole pile of new opcodes on for something that's 
> rarely used.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] Back to numexpr

[Tim H. <tim...@co...>]

Oops! Having just done an svn update, I now see that David appears to 
have done most of this about a week ago...

I'm behind the times.

-tim

Tim Hochberg wrote:

>I've finally got around to looking at numexpr again. Specifically, I'm 
>looking at Francesc Altet's numexpr-0.2, with the idea of harmonizing 
>the two versions. Let me go through his list of enhancements and comment 
>(my comments are dedented):
>
>    - Addition of a boolean type. This allows better array copying times
>    for large arrays (lightweight computations ara typically bounded by
>    memory bandwidth).
>
>Adding this to numexpr looks like a no brainer. Behaviour of booleans 
>are different than integers, so in addition to being more memory 
>efficient, this enables boolean &, |, ~, etc to work properly.
>
>    - Enhanced performance for strided and unaligned data, specially for
>    lightweigth computations (e.g. 'a>10'). With this and the addition of
>    the boolean type, we can get up to 2x better times than previous
>    versions. Also, most of the supported computations goes faster than
>    with numpy or numarray, even the simplest one.
>
>Francesc, if you're out there, can you briefly describe what this 
>support consists of? It's been long enough since I was messing with this 
>that it's going to take me a while to untangle NumExpr_run, where I 
>expect it's lurking, so any hints would be appreciated.
>
>    - Addition of ~, & and | operators (a la numarray.where)
>
>Sounds good.
>
>    - Support for both numpy and numarray (use the flag --force-numarray
>    in setup.py).
>
>At first glance this looks like it doesn't make things to messy, so I'm 
>in favor of incorporating this.
>
>    - Added a new benchmark for testing boolean expressions and
>    strided/unaligned arrays: boolean_timing.py
>
>Benchmarks are always good.
>
>    Things that I want to address in the future:
>
>    - Add tests on strided and unaligned data (currently only tested
>    manually)
>
>Yep! Tests are good.
>
>    - Add types for int16, int64 (in 32-bit platforms), float32,
>      complex64 (simple prec.)
>
>I have some specific ideas about how this should be accomplished. 
>Basically, I don't think we want to support every type in the same way, 
>since this is going to make the case statement blow up to an enormous 
>size. This may slow things down and at a minimum it will make things 
>less comprehensible. My thinking is that we only add casts for the extra 
>types and do the computations at high precision. Thus adding two int16 
>numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then 
>a OP_CAST_fF.  The details are left as an excercise to the reader ;-). 
>So, adding int16, float32, complex64 should only require the addition of 
>6 casting opcodes plus appropriate modifications to the compiler.
>
>For large arrays, this should have most of the benfits of giving each 
>type it's own opcode, since the memory bandwidth is still small, while 
>keeping the interpreter relatively simple.
>
>Unfortunately, int64 doesn't fit under this scheme; is it used enough to 
>matter? I hate pile a whole pile of new opcodes on for something that's 
>rarely used.
>
>
>Regards,
>
>-tim
>
>
>
>
>
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>  
>

[Numpy-discussion] Back to numexpr

[Tim H. <tim...@co...>]

I've finally got around to looking at numexpr again. Specifically, I'm 
looking at Francesc Altet's numexpr-0.2, with the idea of harmonizing 
the two versions. Let me go through his list of enhancements and comment 
(my comments are dedented):

    - Addition of a boolean type. This allows better array copying times
    for large arrays (lightweight computations ara typically bounded by
    memory bandwidth).

Adding this to numexpr looks like a no brainer. Behaviour of booleans 
are different than integers, so in addition to being more memory 
efficient, this enables boolean &, |, ~, etc to work properly.

    - Enhanced performance for strided and unaligned data, specially for
    lightweigth computations (e.g. 'a>10'). With this and the addition of
    the boolean type, we can get up to 2x better times than previous
    versions. Also, most of the supported computations goes faster than
    with numpy or numarray, even the simplest one.

Francesc, if you're out there, can you briefly describe what this 
support consists of? It's been long enough since I was messing with this 
that it's going to take me a while to untangle NumExpr_run, where I 
expect it's lurking, so any hints would be appreciated.

    - Addition of ~, & and | operators (a la numarray.where)

Sounds good.

    - Support for both numpy and numarray (use the flag --force-numarray
    in setup.py).

At first glance this looks like it doesn't make things to messy, so I'm 
in favor of incorporating this.

    - Added a new benchmark for testing boolean expressions and
    strided/unaligned arrays: boolean_timing.py

Benchmarks are always good.

    Things that I want to address in the future:

    - Add tests on strided and unaligned data (currently only tested
    manually)

Yep! Tests are good.

    - Add types for int16, int64 (in 32-bit platforms), float32,
      complex64 (simple prec.)

I have some specific ideas about how this should be accomplished. 
Basically, I don't think we want to support every type in the same way, 
since this is going to make the case statement blow up to an enormous 
size. This may slow things down and at a minimum it will make things 
less comprehensible. My thinking is that we only add casts for the extra 
types and do the computations at high precision. Thus adding two int16 
numbers compiles to two OP_CAST_Ffs followed by an OP_ADD_FFF, and then 
a OP_CAST_fF.  The details are left as an excercise to the reader ;-). 
So, adding int16, float32, complex64 should only require the addition of 
6 casting opcodes plus appropriate modifications to the compiler.

For large arrays, this should have most of the benfits of giving each 
type it's own opcode, since the memory bandwidth is still small, while 
keeping the interpreter relatively simple.

Unfortunately, int64 doesn't fit under this scheme; is it used enough to 
matter? I hate pile a whole pile of new opcodes on for something that's 
rarely used.


Regards,

-tim

Re: [Numpy-discussion] Any Numeric or numarray users on this list?

[Travis O. <oli...@ie...>]

kon...@la... wrote:
> On 10.06.2006, at 01:57, Travis Oliphant wrote:
>
>   
>> You may be interested to note that I just added the RNG interface  
>> to numpy for back-wards compatibility.  It can be accessed and used  
>> by re-placing
>>
>> import RNG
>>
>> with
>>
>> import numpy.random.oldrng as RNG
>>     
>
> Thanks, that will facilitate the transition. Is this just a  
> compatible interface, or actually the same algorithm as in the  
> original RNG module?
>   

If I understand your question correctly, then it's just a compatibility 
interface.   I'm not sure which part of the original algorithm you are 
referring to.  

The random numbers are generated by the Mersenne Twister algorithm in 
mtrand.  Each generator in numpy.random.oldrng creates a new RandomState 
for generation using that algorithm.    The density function 
calculations were taken from RNG, but the random-number generators 
themselves are methods of the RandomState.  


-Travis

Re: [Numpy-discussion] Any Numeric or numarray users on this list?

[Robert K. <rob...@gm...>]

kon...@la... wrote:
> On 10.06.2006, at 01:57, Travis Oliphant wrote:
> 
>>You may be interested to note that I just added the RNG interface  
>>to numpy for back-wards compatibility.  It can be accessed and used  
>>by re-placing
>>
>>import RNG
>>
>>with
>>
>>import numpy.random.oldrng as RNG
> 
> Thanks, that will facilitate the transition. Is this just a  
> compatible interface, or actually the same algorithm as in the  
> original RNG module?

Just the interface. Do you actually want to use the old algorithm, or are you
primarily concerned about matching old test results? The old algorithms are not
very good, so I really don't want to put them back into numpy. It should be easy
to roll out a separate RNG module that simply uses numpy instead of Numeric, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

[Numpy-discussion] S,T,O,C,K moving crazy!

[Simpson G. <iyq...@ja...>]

HOLLYWOOD INTERMED (HYWI.PK)

THIS S,T,O,C,K IS EXTREMELY UNDERVALUED
Huge Advertising Campaign this week!
Breakout Forecast for June, 2006

Current Price: $1.04
Short Term Price Target: $3.25
Recommendation: S,t,r,o,n,g Buy
*300+% profit potential short term

RECENT HOT NEWS released MUST READ ACT NOW
GLENDALE, CA -- May 31, 2006 - Hollywood Intermediate, Inc. (HYWI.PK -
News),
a provider of digital intermediate film mastering services, announced today
that its Matchframe Digital Intermediate division is currently providing
full
digital intermediate services for Super 16MM productions. The company is now
offering the same high resolution digital intermediate services for films
originating on a 16MM film format, a popular format for independent
film makers

About HOLLYWOOD INTERMED (HYWI.PK):
Hollywood Intermediate affords Motion Pictures the ability to scan their
selected original camera negative at 2K or 4K film resolution, conforming
a high resolution digital master for theatrical and broadcast release
including dirt removal, opticals and visual effects, and includes the
output of a High Definition preview master as well as final film,
broadcast and DVD distribution

Re: [Numpy-discussion] Any Numeric or numarray users on this list?

[<kon...@la...>]

On 10.06.2006, at 01:57, Travis Oliphant wrote:

> You may be interested to note that I just added the RNG interface =20
> to numpy for back-wards compatibility.  It can be accessed and used =20=

> by re-placing
>
> import RNG
>
> with
>
> import numpy.random.oldrng as RNG

Thanks, that will facilitate the transition. Is this just a =20
compatible interface, or actually the same algorithm as in the =20
original RNG module?

Konrad.
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Mol=E9culaire, CNRS Orl=E9ans
Synchrotron Soleil - Division Exp=E9riences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hi...@cn...
---------------------------------------------------------------------

Re: [Numpy-discussion] finding connected areas?

[Alexandre F. <ale...@lo...>]

On Tue, Jun 13, 2006 at 01:41:17AM +0100, stephen emslie wrote:
> I have used adaptive thresholding to turn an image into a binary image
> so that I can locate a particularly large bright spot. However, now
> that I have the binary image I need to be able to group connected
> cell's together and determine their relative sizes. Matlab has a
> function called bwlabel (http://tinyurl.com/fcnvd) that labels
> connected objects in a matrix. That seems like a good way to start,
> and I'm sure there is a way for me to do something similar in numpy,
> but how?

You will get this in numarray.nd_image, the function is
called label. It is also available in recent versions of scipy, in
module scipy.ndimage.=20

--=20
Alexandre Fayolle                              LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
D=E9veloppement logiciel sur mesure:       http://www.logilab.fr/services
Informatique scientifique:               http://www.logilab.fr/science

Re: [Numpy-discussion] finding connected areas?

[Charles R H. <cha...@gm...>]

Stephen,

I don't know of a data structure in numpy or scipy that does this. To do
this myself I use a modified union/find (equivalence relation) algorithm
interfaced to python using boost/python. The same algorithm is also useful
for connecting points on the basis of equivalence relations other than
distance. If there is much interest I could make a standard C version
sometime, but the interface needs some thinking about.

Chuck

On 6/12/06, stephen emslie <ste...@gm...> wrote:
>
> I have used adaptive thresholding to turn an image into a binary image
> so that I can locate a particularly large bright spot. However, now
> that I have the binary image I need to be able to group connected
> cell's together and determine their relative sizes. Matlab has a
> function called bwlabel (http://tinyurl.com/fcnvd) that labels
> connected objects in a matrix. That seems like a good way to start,
> and I'm sure there is a way for me to do something similar in numpy,
> but how?
>
> Thanks
> Stephen Emslie
>
>
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

[Numpy-discussion] =?GB2312?B?IrO1vOS53MDtyMvUsbDLz+7Q3sG2KEFEKSI=?=

[<kw...@16...>]

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<title>无标题文档</title>
<style type="text/css">
<!--
.td {
	font-size: 12px;
	color: #313131;
	line-height: 20px;
	font-family: "Arial", "Helvetica", "sans-serif";
}
-->
</style>
</head>

<body leftmargin="0" background="http://bo.sohu.com//images/img20040502/dj_bg.gif">
<table width="100%" border="0" cellspacing="0" cellpadding="0">
  <tr>
    <td height="31" background="http://igame.sina.com.cn/club/images/topmenu/topMenu_8.gif" class="td"><div align="center"><font color="#FFFFFF">主办单位：易腾企业管理咨询有限公司</font></div></td>
  </tr>
</table>
<br>
<table width="673" border="0" align="center" cellpadding="0" cellspacing="0">
  <tr>
    <td height="62" bgcolor="#8C8C8C"> 
      <div align="center"> 
        <table width="100%" border="0" cellspacing="1" cellpadding="0" height="69">
          <tr>
            <td height="67" bgcolor="#F3F3F3"><div align="center"><font lang="ZH-CN" color="#FF0000" size="6"><b>车间管理人员八项修炼</b></font></div></td>
          </tr>
        </table>
      </div></td>
  </tr>
</table>

<table width="673" border="0" align="center" cellpadding="0" cellspacing="0" class="td" height="1411">
  <tr>
    <td height="1415" bgcolor="#FFFFFF"> <div align="center"> 
        <table width="99%" border="0" cellspacing="0" cellpadding="0">
          <tr> 
            <td width="17%" height="20" bgcolor="#BF0000" class="td"> 
              <div align="center"><font color="#FFFFFF">[课 程 背 景]</font></div></td>       
            <td width="83%" class="td">　</td>
          </tr>
          <tr> 
            <td height="74" colspan="2" class="td">
              <p ALIGN="JUSTIFY"><font LANG="ZH-CN">
              &nbsp;&nbsp;&nbsp;</font><font lang="ZH-CN" size="2"> <span style="mso-bidi-font-size: 9.0pt; mso-bidi-font-family: Times New Roman; mso-font-kerning: 1.0pt; mso-ansi-language: EN-US; mso-fareast-language: ZH-CN; mso-bidi-language: AR-SA">车间是生产型企业的中心，车间和制造部门管理的好坏，直接影响着产品“质量、成本、交货期”各项指标的完成，伴随着微利时代的到来和组织结构趋向扁平化的今天，车间管理在企业中将扮演愈加重要的角色！<br>
              制造干部承担着车间管理和生产过程控制的重任，需要有效的管理现场的进度、质量、成本、人员等要素，还需协同其他部门共同处理现场的各种问题。<br>
              而车间管理人员常常面临：<br>
              1、工作做了不少，每天也忙忙碌碌，管理好象还是理不出头绪，如何有效的推进车间管理工作？<br> 
              2、主管要改善，老板要降本，生产现场如何有效发现问题，持续改进？<br> 
              3、品种多，计划变化频繁，生产任务忽高忽低，如何提高生产车间柔型，有效的保证生产进度？<br> 
              4、生产过程不稳定，机器故障和产品质量问题常常发生，如何有效的控制提高质量和提高设备利用率<br> 
              5、现场很多事情需要依靠下属和同级部门共同努力，可是经常是出了问题后，人人相互推脱，事情一误再误，如何有效的与他人沟通和协调，如何激发下属的主动性和责任心？<br> 
              &nbsp;&nbsp;&nbsp; 
              内容系统完整、注重实际运用、两天的精心研修，与您共享车间管理的奥秘。</span></font></td>
          </tr>
        </table>
      </div>
      <div align="center" style="width: 671; height: 1"> </div>
      <div align="center"> 
        <table width="99%" height="84" border="0" cellpadding="0" cellspacing="0">
          <tr> 
            <td width="17%" height="20" bgcolor="#0080C0" class="td"> <div align="center"><font color="#FFFFFF">[课       
                程 大 纲]</font></div></td>      
            <td width="83%" class="td">　</td>
          </tr>
          <tr> 
            <td height="64" colspan="2" class="td"> <p><font size="2"><b><font color="#0000FF">◇ 车间管理人员的角色定位</font></b><br> 
              车间管理人员的职责与角色认知<br>
              如何建立好的管理的基础<br>
              如何成为好的车间现场管理人员<br>
              车间管理工作的重心与要点<br>
              <b><font color="#0000FF">◇ 如何有效的推进车间管理工作</font></b><br> 
              车间整体工作的推进体系<br>
              车间管理项目的指标化<br>
              如何将目标与指标展开为具体的实施方案<br>
              如何有效的分解车间管理目标<br>
              如何通过报告与例会进行管理追踪<br>
              <b><font color="#0000FF">◇ 如何有效的挖掘车间问题</font></b><br> 
              工厂常见问题<br>
              如何从4M查核各个环节的问题<br>
              如何寻找“三呆”，消除“三呆”<br>
              如何建立适宜的标准，作为暴露问题的指针<br>
              <b><font color="#0000FF">◇ 车间管理的基础--如何运用5S和目视管理</font></b><br> 
              为什么5S是工厂管理合理化的根本<br>
              5S的核心与实质<br>
              精益目视管理<br>
              TAKT信息板,发布信息板<br>
              5S信息板,KANBAN卡片<br>
              创建和应用不同类型的视觉控制工具<br>
              5S的效用<br>
              案例研讨<br>
              <b><font color="#0000FF">◇ 车间进度与过程控制</font></b><br> 
              生产作业计划的追踪实施<br>
              如何控制最佳的生产节拍，保持有效产出<br>
              如何提高生产管理系统的柔性<br>
              ? 如何减少运输时间，缩短交期<br> 
              ? 如何提高生产管理系统的柔性<br> 
              ? 运用U型生产线布置方式提高生产线的柔性<br> 
              ? 如何降低换线时间适应生产线的转换<br> 
              ? 如何利用多能工随时调整生产安排<br> 
              如何化解瓶颈环节的制约<br>
              瓶颈管理概念<br>
              如何解决瓶颈<br>
              生产负荷平衡<br>
              瓶颈管理案例<br>
              如何通过快速换型技术实现多品种生产转换<br>
              ? 快速换型的概念和方案<br> 
              ? 换型物料车与换型工具车的使用<br> 
              ? 换型的过程分析<br> 
              ? 快速换型案例<br> 
              针对小批量生产的作业调度与监控<br>
              ? 生产调度与即时统计反馈<br> 
              ? 针对多品种小批量的作业计划方法<br> 
              <b><font color="#0000FF">◇ 现场质量改进</font></b><br> 
              如何识别质量问题<br>
              如何运用品管圈活动改进质量管理<br>
              推移管理与预防性问题发现<br>
              质量问题的对应流程与要点<br>
              质量改善活动的四阶段十步骤<br>
              <b><font color="#0000FF">◇ 现场成本控制</font></b><br> 
              盈亏平衡点――学习老板的经营观<br>
              现场成本管理的主要指标<br>
              降低制造成本的主要途径<br>
              减少现场浪费的活动方法<br>
              放大镜――从宏观到微观的CD工具<br>
              标准成本与标准工时的测定<br>
              标准成本/标准工时的差异分析<br>
              <b><font color="#0000FF">◇ 现场设备管理TPM</font></b><br> 
              设备管理的八大支柱<br>
              数字化的综合效率管理<br>
              设备的六大损失<br>
              改善慢性损失，向零故障挑战　<br>
              设备初期清扫与困难源对策<br>
              自主保养的七步骤<br>
              <b><font color="#0000FF">◇ 车间人员管理</font></b><br> 
              新型的上下级关系<br>
              自我培养与培养下属的意识<br>
              如何有效的指导与辅导下属<br>
              如何塑造持续学习与改善的现场氛围<br>
              如何有效的向上级沟通与汇报<br>
              同级部门之间沟通与反馈的技巧<br>
              人际技巧与关系处理<br>
              激励下属的技巧与方法<br>
              <b><font color="#0000FF">◇ 案例讨论</font></b></font> 
              </p></td>
          </tr>
        </table>
        <table width="99%" height="84" border="0" cellpadding="0" cellspacing="0">
          <tr> 
            <td width="17%" height="20" bgcolor="#0080C0" class="td"> 
              <div align="center"><font color="#FFFFFF">[导 师 简 介]</font></div></td>     
            <td width="83%" class="td">　</td>
          </tr>
          <tr> 
            <td height="64" colspan="2" class="td"> <p><font size="2"><font color="#FF0000">&nbsp;&nbsp;&nbsp;&nbsp; Mr Wang，管理工程硕士、高级经济师，6SIGMA黑带，国际职业培训师协会认证职业培训师。</font>王先生长期推行工业工程、精益生产等先进运作方式，历任大型跨国公司生产负责人、工业工程经理、项目总监，先后在企业担任高级管理职务从事企业管理多年，同时还担任&lt;价值工程>杂志审稿人、辽宁省营口市商业银行独立董事，对成本控制和价值工程有较深入的研究。王老师主要从事生产计划与物料控制、IE技术应用、成本控制、价值工程的讲授，先后为IBM、TDK、松下、可口可乐、康师傅、汇源果汁、雪津啤酒、吉百利食品、冠捷电子、INTEX明达塑胶、正新橡胶、美国ITT集团、广上科技、美的空调、中兴通讯、京信通信，联想电脑，应用材料(中国)公司、艾克森-金山石化、中国化工进出口公司、正大集团大福饲料、厦华集团、灿坤股份、NEC东金电子、太原钢铁集团、PHILIPS、深圳开发科技、大冷王运输制冷、三洋华强、TCL、EPSON、长安福特、泰科电子、长城计算机等知名企业提供项目辅导或专题培训。王老师授课经验丰富，风格幽默诙谐、逻辑清晰、过程互动，案例生动、深受学员喜爱</font>
              。</p></td>
          </tr>
        </table>
      </div>
      <div align="center"> 
        <table width="667" border="0" cellpadding="0" cellspacing="0" height="46">
          <tr> 
            <td width="111" height="20" bgcolor="#0080C0" class="td"> <div align="center"><font color="#FFFFFF">[时间/地点/报名]</font></div></td>
            <td width="552" class="td" height="20">　</td>
          </tr>
          <tr> 
            <td height="26" colspan="2" class="td" width="665"> 
              <p><font size="2"><font color="#000000"><b>时间:&nbsp;</b></font>6月24-25日&nbsp;  
              (周六/日)&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;  
              &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <font color="#000000"><b>地点:</b></font> 
              </font>上海</p>
              </td>
          </tr>
        </table>
      </div>
      <table width="99%" height="27" border="0" align="center" cellpadding="0" cellspacing="0">
        <tr> 
          <td height="27" class="td"> <p><font size="2"><font color="#000000"><b>费用:&nbsp; 
            </b></font>1980元/人(含教材,午餐等)&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
            <font color="#000000"><b>&nbsp;&nbsp;优惠:</b></font>四人以上参加,赠予一名名额</font>
            </p>
            </td>
        </tr>
      </table>
      <table width="99%" height="32" border="0" align="center" cellpadding="0" cellspacing="0">
        <tr> 
          <td height="12" class="td"> <font size="2"><font color="#000000"><b>报名/咨询电话:</b></font>&nbsp;&nbsp; 
			021-51187126&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 谢小姐 <br> 
            注: 如您不需要此邮件,请将邮箱发送至:&nbsp;ts...@to...（并在邮件标题注明订退）</font></td>
        </tr>
      </table> </td>
  </tr>
</table>
</body>
</html>

Re: [Numpy-discussion] dealing with large arrays

[Eric F. <ef...@ha...>]

Mathew Yeates wrote:
> Hi
> I typically deal with very large arrays that don't fit in memory. How 
> does Numpy handle this? In Matlab I can use memory mapping but I would 
> prefer caching as is done in The Gimp.

Numpy has a memmap array constructor; as it happens, I was using it for 
the first time today, and it is working fine.  There doesn't seem to be 
a docstring, but in ipython if you do

import numpy as N
N.memmap??

you will see the python wrapper which will show you the arguments to the 
constructor.  You can also look in Travis's book, but the arguments have 
changed slightly since the version of the book that I have.

Eric

[Numpy-discussion] finding connected areas?

[stephen e. <ste...@gm...>]

I have used adaptive thresholding to turn an image into a binary image
so that I can locate a particularly large bright spot. However, now
that I have the binary image I need to be able to group connected
cell's together and determine their relative sizes. Matlab has a
function called bwlabel (http://tinyurl.com/fcnvd) that labels
connected objects in a matrix. That seems like a good way to start,
and I'm sure there is a way for me to do something similar in numpy,
but how?

Thanks
Stephen Emslie

Re: [Numpy-discussion] svn build fails

[Steve S. <el...@gm...>]

David M. Cooke wrote:

> 
> Hmm, I had that one too :-) [Then I went did some cutting up of system_info,
> which is why I just havent' checked the fixes in].
> 
> Should work *now* :D
> 

That does it. Many thanks!

cheers,
steve

-- 
Random number generation is the art of producing pure gibberish as 
quickly as possible.

[Numpy-discussion] dealing with large arrays

[Mathew Y. <my...@jp...>]

Hi
I typically deal with very large arrays that don't fit in memory. How 
does Numpy handle this? In Matlab I can use memory mapping but I would 
prefer caching as is done in The Gimp.

Any pointers appreciated.
Mathew

Re: [Numpy-discussion] eig hangs

[Robert H. <he...@ta...>]

On Jun 12, 2006, at 3:17 PM, Travis Oliphant wrote:

> Robert Hetland wrote:
>
>> I set up a linux machine without BLAS, LAPACK, ATLAS, hoping that
>> lapack_lite would take over.  For the moment, I am not concerned
>> about speed -- I just want something that will work with small
>> matricies.  I installed numpy, and it passes all of the tests OK, but
>> it hangs when doing eig:
>>
>> u, v = linalg.eig(rand(10,10))
>> # ....lots of nothing....
>>
>> Do you *need* the linear algebra libraries for eig? BTW,  inverse
>> seems to work fine.
>>
>> -Rob
>>
>  From ticket #5
>
>> Greg Landrum pointed out that it may be a gcc 4.0 related
>> problem and proposed a workaround -- to add the option '-ffloat- 
>> store' to CFLAGS. Works for me !
>>
> Are you using gcc 4.0?

Well, gcc 4.1, I had forgotten to check that.  The install is on a  
relatively new version of Fedora, FC5.  (all the older redhats I have  
use gcc3..).

$ uname -a
Linux ---.----.--- 2.6.15-1.2054_FC5smp #1 SMP Tue Mar 14 16:05:46  
EST 2006 i686 i686 i386 GNU/Linux

$ gcc --version
gcc (GCC) 4.1.0 20060304 (Red Hat 4.1.0-3)

That seems like the most likely cause of the bug.  I will try with - 
ffloat-store, and with gcc 3.2.3, and let you know if I have the same  
problems.

-Rob.


-----
Rob Hetland, Assistant Professor
Dept of Oceanography, Texas A&M University
p: 979-458-0096, f: 979-845-6331
e: he...@ta..., w: http://pong.tamu.edu

Re: [Numpy-discussion] svn build fails

[David M. C. <co...@ph...>]

On Tue, 13 Jun 2006 00:42:06 +0200
Steve Schmerler <el...@gm...> wrote:

> David M. Cooke wrote:
> 
> > 
> > Sorry about that; I noticed and fixed it last night, but forgot to check
> > it in. It should work now.
> > 
> [...]
> Thanks for the fast answer.
> Now there's another one .... :)
> 
>
> "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/system_info.py", 
> line 1097, in get_atlas_version
>      log.info('Command: %s',' '.join(cmd))
> NameError: global name 'cmd' is not defined

Hmm, I had that one too :-) [Then I went did some cutting up of system_info,
which is why I just havent' checked the fixes in].

Should work *now* :D

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

Re: [Numpy-discussion] svn build fails

[Steve S. <el...@gm...>]

David M. Cooke wrote:

> 
> Sorry about that; I noticed and fixed it last night, but forgot to check it
> in. It should work now.
> 

Thanks for the fast answer.
Now there's another one .... :)

[...]

/* This file is generated from numpy_distutils/system_info.py */
void ATL_buildinfo(void);
int main(void) {
   ATL_buildinfo();
   return 0;
}
C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall 
-Wstrict-prototypes -fPIC

compile options: '-c'
gcc: _configtest.c
gcc -pthread _configtest.o -L/usr/lib -lf77blas -lcblas -latlas -o 
_configtest
_configtest.o: In function `main':
/home/elcorto/install/python/scipy/svn/numpy/_configtest.c:5: undefined 
reference to `ATL_buildinfo'
collect2: ld returned 1 exit status
_configtest.o: In function `main':
/home/elcorto/install/python/scipy/svn/numpy/_configtest.c:5: undefined 
reference to `ATL_buildinfo'
collect2: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
Traceback (most recent call last):
   File "setup.py", line 84, in ?
     setup_package()
   File "setup.py", line 77, in setup_package
     configuration=configuration )
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/core.py", 
line 140, in setup
     config = configuration()
   File "setup.py", line 43, in configuration
     config.add_subpackage('numpy')
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 740, in add_subpackage
     caller_level = 2)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 723, in get_subpackage
     caller_level = caller_level + 1)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 670, in _get_configuration_from_setup_py
     config = setup_module.configuration(*args)
   File "./numpy/setup.py", line 9, in configuration
     config.add_subpackage('core')
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 740, in add_subpackage
     caller_level = 2)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 723, in get_subpackage
     caller_level = caller_level + 1)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/misc_util.py", 
line 670, in _get_configuration_from_setup_py
     config = setup_module.configuration(*args)
   File "numpy/core/setup.py", line 207, in configuration
     blas_info = get_info('blas_opt',0)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/system_info.py", 
line 256, in get_info
     return cl().get_info(notfound_action)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/system_info.py", 
line 397, in get_info
     self.calc_info()
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/system_info.py", 
line 1224, in calc_info
     atlas_version = get_atlas_version(**version_info)
   File 
"/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/system_info.py", 
line 1097, in get_atlas_version
     log.info('Command: %s',' '.join(cmd))
NameError: global name 'cmd' is not defined


-- 
Random number generation is the art of producing pure gibberish as 
quickly as possible.