Re: [matplotlib-devel] Patch suggestion

[John H. <jdh...@ac...>]

>>>>> "Nicholas" == Nicholas Young <su...@su...> writes:

    Nicholas> I've attempted to implement this code myself (see
    Nicholas> attached patch to src/_image.cpp) but I'm not a regular
    Nicholas> c++ or even c programmer so it's fairly likely there
    Nicholas> will be memory leaks in the code. For a 1024x2048 array
    Nicholas> using the GTKAgg backend and with plenty of memory free
    Nicholas> this change results in show() taking <0.7s rather than
    Nicholas> >4.6s; if there is a memory shortage and swapping
    Nicholas> becomes involved the change is much more noticeable. I
    Nicholas> haven't made any decent Python wrapping code yet - but
    Nicholas> would be happy do do so if someone familiar with c++
    Nicholas> could tidy up my attachment.

Hi Nicholas, 

Thanks for the suggestions and patch.  I incorporated frombuffer and
have been testing it.  I've been testing the performance of frombuffer
vs fromarray, and have seen some 2-3x speedups but nothing like the
numbers you are reporting.  [Also, I don't see any detectable memory
leaks so I don't think you have any worries there]

Here is the test script I am using - does this look like a fair test?

You can uncomment report_memory on unix like systems to get a memory
report on each pass through the loop, and switch out fromarray vs
frombuffer to compare your function with mine

On a related note, below I'm pasting in a representative section the
code I am currently using in fromarray for MxNx3 and MxNx4 arrays --
any obvious performance gains to be had here numerix gurus?

Another suggestion for Nicholas -- perhaps you want to support MxN,
MxNx3 and MxNx4 arrays in your frombuffer function?

And a final question -- how are you getting your function into the
matplotlib image pipeline.  Did you alter the image.py
AxesImage.set_data function to test whether A is a buffer object?  If
so, you might want to post these changes to the codebase as well.


// some fromarray code

   //PyArrayObject *A = (PyArrayObject *) PyArray_ContiguousFromObject(x.ptr(), PyArray_DOUBLE, 2, 3); 
  PyArrayObject *A = (PyArrayObject *) PyArray_FromObject(x.ptr(), PyArray_DOUBLE, 2, 3); 

    int rgba = A->dimensions[2]==4;    
    double r,g,b,alpha;
    int offset =0;
    
    for (size_t rownum=0; rownum<imo->rowsIn; rownum++) {
      for (size_t colnum=0; colnum<imo->colsIn; colnum++) {
	offset = rownum*A->strides[0] + colnum*A->strides[1];
	r = *(double *)(A->data + offset);
	g = *(double *)(A->data + offset + A->strides[2] );
	b = *(double *)(A->data + offset + 2*A->strides[2] );
 	
	if (rgba) 
	  alpha = *(double *)(A->data + offset + 3*A->strides[2] );
	else
	  alpha = 1.0;
	
	*buffer++ = int(255*r);         // red
	*buffer++ = int(255*g);         // green
	*buffer++ = int(255*b);         // blue
	*buffer++ = int(255*alpha);     // alpha
	
      }
    }


## ... and here is the profile script ....

import sys, os, time, gc


from matplotlib._image import fromarray, fromarray2, frombuffer
from matplotlib.numerix.mlab import rand
from matplotlib.numerix import UInt8

def report_memory(i):
    pid = os.getpid()
    a2 = os.popen('ps -p %d -o rss,sz' % pid).readlines()
    print i, '  ', a2[1],
    return int(a2[1].split()[1])

N = 1024
#X2 = rand(N,N)
#X3 = rand(N,N,3)
X4 = rand(N,N,4)

start = time.time()

b4 = (X4*255).astype(UInt8).tostring()
for i in range(50):

    im = fromarray(X4, 0)        
    #im = frombuffer(b4, N, N, 0)
    #val = report_memory(i)

end = time.time()
print 'elapsed: %1.3f'%(end-start)