Hi Simon:
See remarks below...
Thanks,
Richard R
On Thu, 2008-03-13 at 16:23 +0000, Simon Brown wrote:
> On 13 Mar 2008, at 15:51, Richard Rodgers wrote:
>
> > Hi Simon:
> >
> > While I don't doubt for a moment that there are undiscovered memory
> > leaks in DSpace, I'm not sure I follow the case you describe. By
> > 'object
> > cache' I'm guessing you mean the cache that is held by the Context
> > object. This cache is private to the Context instance, and Contexts
> > as a rule don't live very long (typically a single HTTP request), so I
> > don't see how spidering activity could accumulate objects in it.
>
> Fair enough, I guess I didn't correctly understand how the Context
> object is used. I had assumed that it would be shared across multiple
> requests, largely because of the existence of the cache.
>
> So what happens is, whenever an individual http request accesses an
> Item, that Item is loaded into the HashMap in the Context, then
> discarded when the request is completed? Is it the case that an
> individual Item object is often requested from the database multiple
> times in the course of a single HTTP request?
>
> I'm still curious about the necessity of the cache, as our removing it
> had no noticeable impact on performance and in fact increased the
> responsiveness of the site when we did before-and-after tests with
> Siege.
>
Most of these questions/observations stem from your assumption that the
cache is there for performance, which it is not (except incidentally).
The primary purpose is (roughly) transactional integrity - ensuring that
there is only a single copy of an Item, etc, in play before the Context
commits. In this light, it makes sense that there might be a slight
performance penalty.
> > There are other cases - like ItemImport or MediaFilter runs - that
> > use a
> > single context instance (therefore cache) and might iterate over the
> > whole repository, and *could* suffer from what you describe, but as of
> > 1.4 at least, these apps were all recoded to flush their caches.
>
> Browse.indexAll() and DSIndexer.indexAllItems(), on the other hand,
> don't seem to flush cache. I apciate that it's not an often-used
> case, but it would mean that broken indexes on large databases will
> probably fail to rebuild due to the cache filling up the heap.
>
Well observed - these are fixed in current DSpace trunk I believe,
and a 1.4.X patch
'#1659841 Add option to clear context object cache'
is available.
> --
> Simon Brown <stb28@cam.ac.uk> - Cambridge University Computing Service
> +44 1223 3 34714 - New Museums Site, Pembroke Street, Cambridge CB2 3QH
>
>
|
