[Perl-cache-users] Cache::Cache and locking

[Ethan R. <et...@en...>]

Hi.

It looks to me from the README file that the various Cache::Cache 
modules make a point of avoiding locking of caches; for instance, the 
README indicating that Cache::FileCache avoids file locking entirely by 
using temporary files.  Given that the basic assumption of $cache->set() 
is that the data/object to be cached is already built, it makes perfect 
sense to take this approach.

However, in systems where you cache on demand, it can be desirable to 
lock things so that multiple processes don't concurrently build the 
target structure.  One approach we've used is to use a locked file 
during the build process, and only the process with the lock on that 
file enters the actual build routine, during which the expensive, 
application-specific logic takes place.  Any other processes that need 
the cache will wait while the lock file remains in place, and once the 
lock file is free, they'll check for the actual cache file again and 
presumably find it present and up-to-date.  This causes multiple 
processes to potentially wait on the outcome of one, but at least the 
extra processes aren't wasting system resources doing redundant, 
arbitrarily-complex builds.

I'm working on a project in which this kind of locking during the build 
process is essential.  While I would like to use the Cache::Cache family 
(Cache::FileCache for now), this approach seems not only at odds with 
the storage mechanisms available but the very interface of Cache::Cache 
itself -- for the sort of build process I've described, get() and set() 
essentially need to be a single atomic operation (from the perspective 
of the caller).  My current intention is to make a new Cache::Cache 
subclass that does file-based caching but with locks, with the special 
addition that you can pass a subref as the item to be cached, such that 
set() would execute the subref and cache the result and thus bring the 
build process into the portion of code in which the file is locked.

But, before pursuing this, I thought I'd check in and see if anybody 
else has tackled this kind of issue within Cache::Cache and come away 
with satisfactory solutions.  Or if anybody has any guidance or 
suggestions to offer.  Would I be better off starting from scratch for 
my Cache::Cache subclass, or could I perhaps find it helpful to start 
from Cache::FileCache?

Any suggestions will be much-appreciated.

Thanks.
- Ethan

-- 
Ethan Rowe
End Point Corporation
et...@en...