Re: [Swig-devel] Re: ready for 1.3.29...

[Marcelo M. <mm...@ac...>]

Robin Dunn wrote:

> William S Fulton wrote:
>
>> William S Fulton wrote:
>>
>>> Marcelo Matus wrote:
>>>
>>>> Here we are ready for 1.3.29.
>>>>
>>>> All reported errors are fixed and we reached the 2,000 downloads 
>>>> already...
>>>>
>>>> so, William, whenever you have time.....
>>>>
>>>>
>>> I'm a bit short of bandwidth right now. Early next week should be 
>>> good. There still seem to be a few bugs rolling in so probably okay 
>>> to wait a few more days:)
>>>
>>
>> Unless there are any objections, I'm going to try and make the 
>> release on Sunday. I don't think we need a release candidate. Please 
>> start testing from cvs and please no more code changes so that 
>> testing is less drawn out.
>
>
> This isn't an objection, just a concern.  I was working with Andi 
> Vadja to get PyLucene working with 1.3.28 and he noticed some big 
> runtime performance problems.  Using Python's timeit module he tested 
> like this:
>
>     python -m timeit -s "from PyLucene import Document" "Document()"
>
> Here are his results:
>
>     with 1.3.28: 100 loops, best of 3: 14.5 msec per loop
>     with 1.3.24 (and my hacks): 10000 loops, best of 3: 88.2 usec per 
> loop
>     ie, 1.3.24 is 164 times faster :(
>
>
> He noticed the new try/except in the __init__ of the proxy classes and 
> tested again after hacking them out, with these results:
>
>     without the try/except : 10000 loops, best of 3: 21 usec per loop
>     this is now 4 times *faster* than 1.3.24, hence 640 times
>     faster than with the try/except
>
>
> I tried to duplicate the problem with wxPython, but wasn't able to. 
> Part of it may be that I was not able to use 1.3.28 because of a bogus 
> syntax error that I couldn't find a workaround for, so I updated to 
> 1.3.29-cvs and used that instead.  Here are my results in timing the 
> creation of a wx.Size object:
>
> SWIG 1.3.24:
>     10000 loops, best of 3: 40.5 usec per loop
>
> SWIG 1.3.27:
>     10000 loops, best of 3: 46.8 usec per loop
>
> SWIG 1.3.29, with try/except:
>     10000 loops, best of 3: 30.8 usec per loop
>
> SWIG 1.3.29 without try/except:
>     100000 loops, best of 3: 11.7 usec per loop
>
>
> First, does anybody recall a change since 1.3.28 that might have 
> gotten rid of some unnecessary overhead in proxy object construction 
> and explain Andi's poor timings?  Second, the improvement in my times 
> without the try/except in the __init__ is an attractive boost, is 
> there perhaps a way to do it that doesn't have as much cost as a 
> try/except?
>
Also, talking about PyLucene performance, I noticed they call 
SWIG_TypeQuery directly or  indirectly
without any 'caching', which is very expensive, for example, instead of:

PyObject *jo2pr_type(jobject obj, const char *type)
{
    return jo2pr(obj, SWIG_TypeQuery(type));
}

%extend org::osafoundation::util::IndexReaderEnumeration {

    PyObject *next()
    {
        if (self->hasMoreElements())
        {
            jint id = self->current;
            ::org::apache::lucene::document::Document *doc = 
(::org::apache::lucene::document::Document *) self->nextElement();

            PythonGIL gil;
            PyObject *pyTuple = PyTuple_New(2);
            PyObject *pyId = PyInt_FromLong((int) id);
            PyObject *pyDoc =
                jo2pr_type(doc, "org::apache::lucene::document::Document 
*");

            PyTuple_SET_ITEM(pyTuple, 0, pyId);
            PyTuple_SET_ITEM(pyTuple, 1, pyDoc);

            return pyTuple;
        }
}

is better to use:

%extend org::osafoundation::util::IndexReaderEnumeration {

    PyObject *next()
    {
        if (self->hasMoreElements())
        {
            jint id = self->current;
            ::org::apache::lucene::document::Document *doc = 
(::org::apache::lucene::document::Document *) self->nextElement();

            PythonGIL gil;
            PyObject *pyTuple = PyTuple_New(2);
            PyObject *pyId = PyInt_FromLong((int) id);
            static swig_type_info *desc =  
SWIG_TypeQuery("org::apache::lucene::document::Document *");
            PyObject *pyDoc =  jo2pr(doc, desc);

            PyTuple_SET_ITEM(pyTuple, 0, pyId);
            PyTuple_SET_ITEM(pyTuple, 1, pyDoc);

            return pyTuple;
        }
}

note the 'static' use in 'static swig_type_info *desc'.

The problem is that SWIG_TypeQuery is a linear expensive search.


Marcelo