#21 Increase Performance by 4X on v20branch

closed
None
5
2006-01-17
2005-12-15
No

You can probably collapse the 'reduce super type
complexity feature' into this. Anywhere we do
reflection we should only have to do it once per bean
mapper instance. Many other things we can take out.
Matt you are the man.

Discussion

  • Franz Garsombke

    Franz Garsombke - 2005-12-15
    • summary: Increase Performance by 4X --> Increase Performance by 4X on v20branch
     
  • Matt Tierney

    Matt Tierney - 2005-12-30

    Logged In: YES
    user_id=1236069

    1st phase of performance improvements have been
    implemented. Performance has been improved 3x - 4x in the
    v20branch. But this should be an ongoing feature request
    as we can always find additional performance improving
    changes to make.

     
  • Matt Tierney

    Matt Tierney - 2005-12-30
    • status: open --> pending
     
  • Franz Garsombke

    Franz Garsombke - 2006-01-04
    • status: pending --> open
     
  • Franz Garsombke

    Franz Garsombke - 2006-01-04

    Logged In: YES
    user_id=550744

    Summary of changes from Matt. I have also attached his
    spreadsheet with all of the statistics.

    --------------------------

    It was a piece of cake. I switched out ehcache with a real
    simple custom cache impl. Performance is still good. I
    need to put the finishes touches on it and then I will check
    the changes in. Should be 2night or tomorrow.

    -------------- Original message --------------
    From: mhtierney@comcast.net
    I think having our own cache implementation would be
    pretty easy. It wouldn't be nearly as robust or flexible as
    a full blown caching framework but it would elimate the
    ehcache dependency. We probably don't need all the bells
    and whistles for Dozer anyways. It would be just a matter
    of switching out the implementation of the
    DozerCacheManager. I think the API of the CacheManagerIF
    would stay the same for the most part. We could just have
    the DozerCacheManager backed by a Map or something. I
    don't know how a custom cache would perform vs. ehcache but
    it will be easy to find out. I wouldn't expect a big
    difference but I havent dug into the ehcache src code so I
    am not familiar with their underlying implementation.

    In jakarta commons Collections there is a FastHashMap
    that might work good for the backing store of the custom
    cache. The jakarta commons PropertyUtils caches property
    descriptors and it currently uses the FastHashMap to
    implement it's caching. I will research other options as well.

    I will try to switch out ehcache and compare
    performance. I dont think storing the data on the actual
    FieldMap instances is the way to go.

    It was just easiest to use ehcache to get things rolling
    and to test out the improvements to see if they even helped.
    I didn't want to have to reinvent the wheel of implementing
    a custom cache without even knowing whether it was going to
    help or what shape it would take. I figured ehcache has any
    performance, synchronization, etc issues already worked out
    and I didn't want to have to deal with those types of issues
    when testing out whether we need some level of caching.

    -------------- Original message --------------
    From: Franz Garsombke <fgarsombke@yahoo.com>
    That is great Matt. I will attach the spreadsheet to
    the feature request and all of these notes when you are done.

    As far as ehcache...I just realized that we will
    need to have that as a dependency. After removing the castor
    dependency...I'm hesitant to add any more. Is there any way
    to have a HashMap or something similiar that gets the
    caching done for us? Can this information be stored on the
    instantiated FieldMaps themselves? So my question is how
    hard would it be to have our own cache implementation...

    Looking at your notes you are definitely hitting
    things I spotted as "expensive". Great job.

    Thanks Matt.

    Franz

    mhtierney@comcast.net wrote:

    I made some more changes after profiling an
    inheritance use case. I also profiled a deep field mapping
    scenario. I will rerun all of the scenarios to get the
    updated timings and send out the updated spreadsheet.

    Here are the additional performance related mods:

    - Modified Hint.getHints() to only determine
    hints one time. And modified getHint so that it doesn't
    call getHints() within an Iterator. This small change made
    a big improvement because getHints() was called for every
    loop and it was getting computed each time. It improved the
    overall inheritance test scenario by 30% - 40%

    - In the mapping processor, removed extra calls
    to CustomConverterContainer.getConverterByDestType()

    - Moved some logic out of the Iterate loop in
    the processSuperTypeMapping method

    - In mapOrRecurseObject(), removed extra calls
    to mappingUtils.isSupportedMap()

    - In DozerCacheManager use instance variable
    for ehcache singleton instance instead of calling
    CacheManager.getInstance() all the time

    - Use StringBuffers for String concats

    - Looked at all the iterate methods and moved
    anything out of them that I could.

    -------------- Original message --------------
    From: mhtierney@comcast.net
    I checked in some performance related
    changes to the 2.0 branch. I got some pretty good gains
    with these changes. I attached a spreadsheet with before
    and after results. I will try to profile some additional
    scenarios. When you have time can someone review my changes
    and make sure everything looks ok thus far? Also, does
    anyone have any other ideas for some scenarios to profile?
    I want to profile one of the inheritance scenarios. Just
    let me know if you see any issues or noticed something I
    missed or overlooked. The build looks good and all is
    running green. Oh yeah, we should probably give some
    thought to how we want to present performance test results
    on the dozer home page.

    Here is a summary of the mods I made and
    checked in(in no particular order):

    - Removed uneccesary calls to
    FieldMap.getSrcFieldValue(). We were making at least 2
    calls for every field when we only need to make 1.

    - Added simple caching framework that uses
    Ehcache 1.1. Added dozer cache manager class and IF. The
    cache manager is set up so that you can cache at the VM
    level or at an object instance level.

    - Added caching to
    CustomConverterContainer.getConverterByDestinationType().
    The caching is at the DozerBeanMapper instance level. I saw
    big gains just by doing this simple thing

    - Added caching to
    MappingProcessor.checkForSuperMappings(). The caching is at
    the DozerBeanMapper instance level. Also broke this method
    up into 2 different methods. 1 that checks and the other
    that processes super mappings if they exist

    - Just a note that these are the only two
    places I added caching. Both of these caches are per
    DozerBeanMapper instance. There is no VM wide caching as of
    right now.

    - Changes around the getDateFormat() method.
    I saw this was getting called for every field regardless of
    whether it even needed to. The constructor for
    SimpleDateFormat looks expensive. So I just modified the
    logic only to create a DateFormat object when needed.

    - Changed the MappingUtils class to non
    static. I didnt create an IF for this but we can if we want.

    - Created MapperTestBase that creates a
    default static mapper instance.

     
  • Matt Tierney

    Matt Tierney - 2006-01-04

    Logged In: YES
    user_id=1236069

    Just a couple notes...

    All timing in the spreadsheet are in milliseconds.

    The timings in the spreadsheet are a good measurement of
    relative performance gains in v2. But you should expect
    even better performance on a production like environment.
    I ran the before and after tests on my grandmas computer so
    the computer horsepower wasn't the greatest:) I reran the
    same tests on my computer at home and saw better v2.0
    timings. Here are those results and I will also update the
    spreadsheet

    Test Scenario #Iterations Total Time(ms) Avg Time(ms)

    1 10000 11000 1.1
    2 25000 3400 .136
    3 25000 3500 .14
    4 10000 5400 .54
    5 10000 2600 .26

     
  • Franz Garsombke

    Franz Garsombke - 2006-01-11
    • status: open --> pending
     
  • Franz Garsombke

    Franz Garsombke - 2006-01-17
    • status: pending --> closed
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks