#19 Performance boost: methods using hash_map<> + lookup corr.

closed-out-of-date
None
5
2009-07-19
2008-03-25
No

Hello :)

In respect with continued development of PyCXX and being nice project I would like to submit following patch that improves PyCXX method's calling speed/performance by almost 2 times (tested in our projects with C/C++ core working with around 2304 python script files). Due to intensive interaction of our cor with Python scripting subsystem it was interesting to find areas for performance improvement in PyCXX method calling routines. As result solution based on hash_map<> hashed container implementation was chosen and small corrections to lookup procedures in Extensions.hxx file were done to avoid double map<> lookups. Custom string hashing module was chosen based on well-known algorythm designed by Paul Hsieh (http://www.azillionmonkeys.com/qed/hash.html) to benefit from better performance and hash distribution for safety sake.

Although to provide compatibility with compilers having specific hash_map<> implementation and etc special defines provided in Config.hxx, which could also b used to return PyCXX to traditional std::map<> usage, so users have choice.

Following patch is not new and was used in our project for some time and as result had thorough real-life testing.

Tested/worked on platforms: WIN64/WIN32 compiled with MS VS 7.x - 9.0 and ICC, 64-/32-bit Linux compiled with GCC 4.x.

Performance increase was for about 2 times in some cases and resulted in noticable application relaxation of CPU usage.

Best regards,
Dmitry Kostjuchenko.

Discussion

  • Anonymous - 2008-03-25
    • assigned_to: nobody --> barry-scott
     
  • Anonymous - 2008-03-25

    Logged In: YES
    user_id=1684258
    Originator: YES

    File Added: pycxx_hashmap_impl.patch

     
  • Anonymous - 2008-03-25

    Logged In: YES
    user_id=1684258
    Originator: YES

    File Added: pycxx_hashmap_impl.patch

     
  • Anonymous - 2008-03-25

    hash_map<> implementaton for PyCXX (for rev. 100, clean)

     
  • Anonymous - 2008-03-25

    Logged In: YES
    user_id=1684258
    Originator: YES

    Warning free SVN patch included for rev.100.

     
  • Barry Alan Scott

    Logged In: YES
    user_id=28665
    Originator: NO

    This looks very interesting. Can I get you fix some simple problems with the patch please?

    1. No tabs use spaces
    2. (define XXX) should be define( XXX )
    3. rename __HM_PYCXX_NAMESPACE to __PYCXX_HM_NAMESPACE

    Can you suggest a benchmark that I can used to comfirm the performance improvement?

    I can use PySVN to check that the API does not break.

    Barry

     
  • Barry Alan Scott

    Logged In: YES
    user_id=28665
    Originator: NO

    cleaned up version of the patch has been committed as r101

    Many thanks.

     
  • Barry Alan Scott

    • status: open --> open-accepted
     
  • Barry Alan Scott

    This patch has been superseded by patch in 2797240

     
  • Barry Alan Scott

    • status: open-accepted --> closed-out-of-date
     

Log in to post a comment.