Fine-grained cache invalidation

[Paul Barry]

Say you have the following sqlmap:

<cacheModel id="itemCache" type="MEMORY">
  <flushOnExecute statement="insertItem"/>
  <flushOnExecute statement="updateItem"/>
</cacheModel>

<select id="getItem" parameterClass="my.package.Item" resultClass="my.package.Item" cacheModel="itemCache">
  SELECT * FFROM items WHERE id = #id:NUMERIC#
</select>

<insert id="insertItem" parameterClass="my.package.Item">
  INSERT INTO items (id,name) VALUES (#id:NUMERIC#,#name:VARCHAR#)
</insert>

<update id="updateItem" parameterClass="my.package.Item">
  UPDATE items SET name = #name:VARCHAR# WHERE id = #id.NUMERIC#
</update>

As I understand it, if you insert or update and item, the entire cache is flushed.  Couldn't iBatis be extended to only invalidate the item in the cache that is equal to the parameter?  I am not sure how the cache is implemented, but I would assume it is something like a HashMap where the parameter is the key and the ResultSet is the object.  If so, just remove the entry in the HashMap that corresponds to the value of the parameter being updated/inserted.

[Clinton Begin]

Hi Paul,

This limitation has little to do with the way the cache is implemented, and everything to do with something called Object Identity...which iBATIS doesn't support.

In short, iBATIS cannot uniquely identify objects.  This might be something we add in the future, but with it may come more constraints and less flexibility.

Cheers,
Clinton

[Kris Jenkins]

(This isn't an iBatis answer, but...) For some of the tasks I have, I've found I really need fine-grained cache invalidation, with the rules covering activity over several DAOs.  I've used AspectJ AOP to get it working, and found it a fairly painless solution.

Just a thought,
Kris

[Paul Barry]

Then out of curiosity, how does ibatis know if something is cached?  How does it retrieve it from the cache?  It must be different then my simplified idea of have HashMap with the parameter as the key and the results as the value, otherwise Object Equality would be good enough.

[Larry Meadors]

If you want the gruesome details, check out the package com.ibatis.sqlmap.engine.cache, it has the core interfaces.

They are actually pretty clear...CB does a good job at simplifying complexity. ;-)

If you want, you can also look at the implementations (they are in packages under that one).

Larry

[Brandon Goodin]

THIS IS A USER SUPPORT QUESTION AND IS NOT RELATED TO THE DEVELOPMENT OF THE IBATIS FRAMEWORK. PLEASE POST ON THE USER SUPPORT FORUM IN THE FUTURE.

[Paul Barry]

So I took a look at the code in com.ibatis.sqlmap.engine.cache.  Wouldn't it be possible for CacheController to have a method like this:

public void flush(CacheModel cacheModel, CacheKey key);

And it could be implemented like this for MemoryCacheController:

public void flush(CacheModel cacheModel, CacheKey key) {
      cache.remove(key);
  }

[Brandon Goodin]

yes, that would certainly be possible. The only challenge is getting your hands on the correct CacheKey. If you look a little deeper as to how the CacheKey is actually generated it's not a simple task. If the goal is to programatically clear a particular cache then you'd have to get your hands on several internal sqlmap components like ParamaterMap, RequestScope, and the parameterObject. I suppose we could expose the cacheKey for the currently executing statement. But, the benefit would be nominal at best considering the cache management handles that kind of scenario already. I would think a more sophisticated need would be when a developer wants to clear a cache that is related to the currently executing statement based upon some discovered condition. In this case it would be very difficult to clear the cache using a cachekey. I believe this is a matter that requires more thought to insure clarity and simplicity. If you want to continue examining the code for a solution, we are always excited to hear ideas on how to improve ibatis.

Thanks for your diligence!
Brandon

[Paul Barry]

I haven't figured out how you indicate that you want to use fine-grained cache invalidation in the mapping yet, but I think this might work:

Modify the method com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeUpdate to call notifListeners like this:

      notifyListeners(getCacheKey(request,parameterObject));

Add a method to com.ibatis.sqlmap.engine.mapping.statement.MappedStatement like this:

public void notifyListeners(CacheKey key);

Implement it like this in BaseStatement:

  public void notifyListeners(CacheKey key) {
    for (int i = 0, n = executeListeners.size(); i < n; i++) {
       ((ExecuteListener) executeListeners.get(i)).onExecuteStatement(this,key);
    }
  }
 
Add a method to com.ibatis.sqlmap.engine.mapping.statement.ExecuteListener like this:

public void onExecuteStatement(MappedStatement statement, CacheKey key);

Implement it in com.ibatis.sqlmap.engine.cache.CacheModel:

public void onExecuteStatement(MappedStatement statement, CacheKey key) {
  flush(key);
}

Create a flush(CacheKey key) method to CacheModel to remove the element from the cache based on the key

[Vic Cekvenich]

If we do get a nice working sample, I will try to extend it to work over JMX (via Commons-Modeler).

Imagine 6 application servers, and you want to fine grain invalidate a chache for a certain key, JMX would let you monitor (memory and cache hits) and invalidate something.

.V

[Paul Barry]

Vic,  I agree that would be a great feature, but you aren't going to be able to do it without a complete re-write of the way the caching works.  The objects are stored in the cache with a CacheKey object, and the CacheKey object doesn't mean anything to humans.

Clinton, why isn't it implemented like a HashMap, where the key is just an Object?