Cache: return expired object if can't get new

bhafer
2009-07-15
2013-05-14
  • bhafer
    bhafer
    2009-07-15

    I am trying to implement a cache that will continue to return an expired object if it is unable to obtain an updated version.  (I.e., getting a stale element back is better than getting nothing.)  This seems to me like it might be a commonly required pattern, and I'm curious if anyone has come up with a solution using ehcache.

    I want to use the SelfPopulatingCache to avoid having the client API dealing with the task of putting elements into the cache.  But most importantly, elements would never be evicted from the cache for being stale unless a newer replacement element can be obtained first.  Obviously, elements could still be evicted for size constraints, but in my application, the cache will be sized to avoid this.  This seems like it would dovetail nicely with the SelfPopulatingCache notion of a CacheEntryFactory.

    I am considering using the SelfPopulatingCache code as a starting point to make my own subclass of BlockingCache that implements the desired behavior.  During get(), it would check if the retrieved element was expired and if so invoke CacheEntryFactory.createEntry(key) to obtain a new entry.  But if createEntry() fails, get() would still return the expired element.

    Most importantly for this to work, though, I need to figure out how to prevent the backing BlockingCache from removing the element when I call super.get(key) so that the element is available to future get() calls by other clients as well.  Unfortunately, there seems to be no good extension point to make this change.  I have a few competing ideas I'm currently entertaining for this solution:

    1)  I could leverage the "eternal" attribute in the cache configuration for an unintended purpose.  Luckily "eternal" appears to be superfluous since you can also set the TTL and TTI parameters to 0 instead.  So I could set my cache to eternal=true and configure the TTL and TTI to set the timeouts I actually desire.  Then, the backing cache will never remove an element, and I would write an isExpired(Element) method in my custom cache class based on Element.isExpired() and to use instead of it.  In this method, I will ignore the eternal setting but still evaluate the other expiry rules.  I'm a little concerned about how to properly handle potential threading issues though, as I see that Cache synchronizes on element before calling element.isExpired().

    2)  It seems unfortunate that there isn't an actual configuration flag on Cache to specify whether objects are expired on get() calls or not.  Since the get() caller always has the ability to perform element.isExpired() and cache.remove(key) if needed, it seems like a simple configuration flag like this to turn off eviction during get() calls would allow more usage patterns to be created.  I would consider this a reasonable design solution, but for obvious reasons I wouldn't want to tackle such an invasive change to the core source code as I would be cutoff from the future codeline.  Unless, of course, the developers thought this would be a useful feature and would consider allowing me to commit it.  Performance-wise, it would add only one instance variable boolean check to each get() call.

    3)  Finally, I know that I can use a CacheEventListener to detect when an element is evicted from the cache.  I would imagine I could use this functionality to trap the eviction and put the expired element back into the cache if I couldn't replace it with a better version.  But this seems like not the cleanest/easiest solution for a couple reasons.  First, there would be a few practical difficulties around coordinating the CacheEventListener thread/object that gets the evicted element and the CacheEntryFactory thread/object that is trying to populate a new element.  Certainly, it could be done but it seems awkward.  Second, I'm not sure how to put the expired element back into the cache and make sure all of its state is still in place such that it isn't now considered non-expired for the next get() call.

    Anyhow, I'd love to get thoughts or suggestions from others who may have tackled this.

     
    • Andrew Liles
      Andrew Liles
      2009-07-19

      I had a similar requirement and agree that we are lacking some hooks to achieve what you want to do.  It seems a quite common requirement to get a callback upon eviction. 

      Anyway, here's a way you can achieve what you want.

      1. Make all my elements eternal, then you never need to worry about the core caching technology evicting a "good" result.

      2. Create a subclass of SelfPopulatingCache

      Override :

         protected void refreshElement(final Element element, Ehcache backingCache) throws Exception
                 
      The incoming Element is the old value, this method is responsible for decidig what to do and updating the backingCache.  You should implement your own expiry mechanism here, and if it is not time to expire the element, merely return from the method.  If it is time to refresh the element, call super.refreshElement(..) and let the default implementation occur.  I think that if an Exception is raised during this process the backing store value is NOT removed.  The "public void refresh()" is tolerant to Exceptions per element and will try to do all elements; - all that will happen is that the outer "refresh()" will throw the last Exception.  If you care about this you could catch the Exception yourself in your call to super.refreshElement(..).

      I required finer grain control on how the cache is refreshed; this may be useful to you as an alternate approach. I (externally) created a ThreadExecutor which gets all the keys in the cache and then works through them one by one (or may be not every cache key), refreshing each element.  To this end, I created new public methods in SelfPopulatingCache like

          public Element refresh(Object key, boolean quiet) throws CacheException

      (note that this feature was committed into the Ehcache codebase at SVN Revision 995 after the 1.6.0 release).

       
    • bhafer
      bhafer
      2009-07-23

      Thanks Andrew.

      This is pretty much along the lines of what I was thinking but with some good tips in there.

      One thing is throwing me off though:

      If I understand this correctly, it looks like my SelfPopulatingCache subclass would never attempt to refresh an element on its own (i.e., during a get() call).  This is why I was going to subclass BlockingCache as SelfPopulatingCache does, so I can override get() and implement a custom expiry check that ignores "eternal".

       
      • Andrew Liles
        Andrew Liles
        2009-07-24

        Ah yes, I see - my previous suggestion only did an expiry check during the refresh cycle on the "refresh thread". 

        If you want to ensure every cache GET runs an expiry check you need to override get(..) (but of course you can still do that if you subclass SelfPopulatingCache  rather than BlockingCache)

         
    • bhafer
      bhafer
      2009-07-27

      Thank you for your assistance.  It's great to be able to validate my design with one of the project's experts.

       
  • bhafer
    bhafer
    2010-01-28

    Hello again.  It took a long time to get the time for it, but I've finally implemented an ExpiredBetterThanNullSelfPopulatingCache and some other features. 

    I've started a SourceForge project: http://sourceforge.net/projects/ehcache-zen/

    Please check it out and let me know on the forums there your feedback.

     
  • Jeroen Borgers
    Jeroen Borgers
    2010-09-23

    We have similar issues and ehcache-zen seems a great solution to them. I think it would be good for acceptance to include zen into standard eh-cache. It would make it much easier at least for me to put this into production.

    Is this being considered?

    Regards, Jeroen.

     
  • Greg Luck
    Greg Luck
    2010-09-24

    Jeroen

    I have read the quickstart and yes, this seems useful. I have just sent a request to bhafer's sourceforge asking for a patch to Ehcache.

    Greg Luck
    Maintainer, Ehcache

     
  • Jeroen Borgers
    Jeroen Borgers
    2010-09-24

    Thanks, Greg!

    Regards,
    Jeroen.

     
  • bhafer
    bhafer
    2010-09-24

    I had always thought these additions would be great in the product, but of course the logistical hurdles are a bit higher with such an undertaking on an active source tree like ehcache.  I currently maintain ehcache 1.6 and 1.7 compatible versions of the zen codebase, which are drastically different implementations due to the corresponding ecache API changes.  I will hope to attempt a patch of the latter at some point in the near future once time permits.

    In the meantime, Jeroen, if you'd like to use zen, it is as simple as adding the jar to the classpath and declaring the cache spring bean as described in the Quick Start.  If you have any questions, please post them on the zen project and I will be glad to assist.

     
  • Jeroen Borgers
    Jeroen Borgers
    2010-09-24

    Okay, good to hear, bhafer!
    If you need a hand pls let me know.
    -Jeroen.

     
  • Jeroen Borgers
    Jeroen Borgers
    2010-09-24

    Brian,

    Do I understand it correctly if I conclude the following:

    If I want to return the stale/expired element while the new value is being fetched from file/database by one thread,
    I should use a SelfManagingCache wrapping a SelfPopulatinggCache?

    Thanks,
    Jeroen.

     
  • Jeroen Borgers
    Jeroen Borgers
    2011-11-07

    Hi bhafer,

    How are things going with the patch on ehcache ?

    Regards,
    Jeroen.

     
  • bhafer
    bhafer
    2011-11-08

    Unfortunately, the ehcache codebase changed significantly with the migration to Terracotta, and the library implementation I created would require significant rework to work with later versions of ehcache.  We are no longer utilizing ehcache in my organization, and I don't have the spare bandwidth to continue working on this project on the side.  Please feel free to utilize and/or update the sourceforge project source code if it is helpful to you.

    Regards,
    Brian

     
  • Jeroen Borgers
    Jeroen Borgers
    2011-11-10

    Hi Brian,

    Thanks for letting me know. I.think I'll utilize your source code.
    Thanks,
    Jeroen.

     

  • Anonymous
    2012-02-14

    Just as a bump to this thread, I'm currently implementing this functionality in Grails.  The approach I've taken is to create a cache with eternal elements and have a background job refresh on a regular basis.  I had to create a custom cache key to be able to recall the method based on the key (since you can't recreate the call from the information in a regular cache key).  If there is any interest in this, I may be able to publish it to the grails plugins repo.