Re: [Numpy-discussion] mysql -> record array

[John H. <jdh...@ac...>]

>>>>> "Erin" == Erin Sheldon <eri...@gm...> writes:

    Erin> The question I have been asking myself is "what is the
    Erin> advantage of such an approach?".  It would be faster, but by

In the use case that prompted this message, the pull from mysql took
almost 3 seconds, and the conversion from lists to numpy arrays took
more that 4 seconds.  We have a list of about 500000 2 tuples of
floats.

Digging in a little bit, we found that numpy is about 3x slower than
Numeric here

  peds-pc311:~> python test.py
  with dtype: 4.25 elapsed seconds
  w/o dtype 5.79 elapsed seconds
  Numeric  1.58 elapsed seconds
  24.0b2
  1.0.1.dev3432

Hmm... So maybe the question is -- is there some low hanging fruit
here to get numpy speeds up?

import time
import numpy
import numpy.random
rand = numpy.random.rand

x = [(rand(), rand()) for i in xrange(500000)]
tnow = time.time()
y = numpy.array(x, dtype=numpy.float_)
tdone = time.time()
print 'with dtype: %1.2f elapsed seconds'%(tdone - tnow)

tnow = time.time()
y = numpy.array(x)
tdone = time.time()
print 'w/o dtype %1.2f elapsed seconds'%(tdone - tnow)

import Numeric
tnow = time.time()
y = Numeric.array(x, Numeric.Float)
tdone = time.time()
print 'Numeric  %1.2f elapsed seconds'%(tdone - tnow)

print Numeric.__version__
print numpy.__version__