Re: [Bigdata-developers] on test protocol memory leaks trac 1015

[Jeremy J C. <jj...@sy...>]

What about the com.bigdata.bop.fed.QueryEngineFactory.standaloneQECache ?

Was I mistaken to read that as a real leak? The code would be cleaner to not have that method and its call, maybe the destroy and the nulling out is sufficient?

Jeremy J Carroll
Principal Architect
Syapse, Inc.



On Sep 29, 2014, at 12:12 PM, Bryan Thompson <br...@sy...> wrote:

> I have looked through those issues and verified that (a) the references are eventually released; and (b) you can force yourkit to take them through a GC and release many of them.  There is a ConcurrentWeakValueHashMapWithTimeout that has a ring buffer which gets cleared down as things age out. Until that happens some kinds of references will remain in CI, but they do get cleared out eventually.
> 
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> br...@sy...
> http://bigdata.com
> http://mapgraph.io
> CONFIDENTIALITY NOTICE:  This email and its contents and attachments are for the sole use of the intended recipient(s) and are confidential or proprietary to SYSTAP. Any unauthorized review, use, disclosure, dissemination or copying of this email or its contents or attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and permanently delete all copies of the email and its contents and attachments.
> 
> 
> 
> On Mon, Sep 29, 2014 at 3:03 PM, Jeremy J Carroll <jj...@sy...> wrote:
> 
> 
> Congrats on getting latest release out.
> 
> Better late than never, I got to looking once more at the memory leaks in TestProtocolAll and have made commit r8670 which I believe addresses them.
> 
> This adds a method to tear down the 'mode' quads/triples/sids and its related index etc at the right point.
> 
> Some concerns:
> - yourkit was suggesting leaks through
> com.bigdata.bop.fed.QueryEngineFactory.standaloneQECache
> 
> and I somewhat inelegantly added a method:
> com.bigdata.bop.fed.QueryEngineFactory.clearStandAloneQECacheDuringTesting()
> which I call again at the end of each mode of the protocol tests
> 
> However, the test issue in this area may be indicative of a real memory leak here (which i believe is fairly new code).
> My own, fairly casual assessment, is that the strong reference to the key in this cache may well be problematic. If so, fixing that and removing my clearStandAloneQECacheDuringTesting() would be better.
> 
> Further I was having some issues with clearing references through the WeakHashMap at
> 
> com.bigdata.btree.ReadWriteLockManager.locks
> 
> I slightly convinced myself that these were artifacts of my test runs, and that the finalizers would eventually run and the memory would eventually be released. That is certainly the theory, but I have had bad experiences with finalizers and the finalizer queue in the past, and I am pretty suspicious of WeakHashMaps as a result. However, despite these concerns, I made no code changes in this area.
> 
> Jeremy
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Slashdot TV.  Videos for Nerds.  Stuff that Matters.
> http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>