Menu
▾
▴
Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature
|
From: Parjanya V. <par...@uw...> - 2022-09-02 19:03:07
|
Hi Manu, Thank you very much for your response. Sure, I will try using that particular tool to check if I can find a memory leak within my code. However, during multiple runs of the code the stack trace always points to the getSignature method whenever this exception is thrown. Moreover, the below StackOverflow answer states that this exception with 'GC overhead limit exceeded' message is thrown when GC is running excessively without being able to claim enough memory: https://stackoverflow.com/questions/1393486/error-java-lang-outofmemoryerror-gc-overhead-limit-exceeded Do you think there might be a chance the issue is indeed within WALA code? Thanks, Parjanya ________________________________ From: Manu Sridharan <ma...@sr...> Sent: Friday, September 2, 2022 2:33:36 PM To: WALA discussion and Q&A <wal...@li...> Subject: Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature Hi Parjanya, An OutOfMemoryError occurs when the GC is unable to collect enough memory to allow for future allocations. This does not occur due to frequent allocation; it happens because too much memory is being retained by the program. Also, the stack trace associated with an OutOfMemoryError usually does not give good guidance as to what is causing the problem. Based on what you wrote below, I suspect the issue here is within your own code. Do you have a large List or Map for caching analysis results that keeps growing? That’s often the root cause of a leak. When I want to debug this sort of issue I use the Eclipse MAT tool to analyze a heap snapshot: https://www.eclipse.org/mat/ To collect a heap snapshot when the program runs out of memory, pass the -XX:+HeapDumpOnOutOfMemoryError parameter to the JVM. I would test this with a much smaller heap than 12000m, since the heap dump will be roughly the size of the heap. Once you open the heap snapshot in MAT, you should be able to see what data structure is consuming a lot of memory. Thanks, Manu On Sep 2, 2022 at 11:07:00, Parjanya Vyas <par...@uw...<mailto:par...@uw...>> wrote: Hi Manu, Thanks a lot for your response. It is the JVM that throws the 'GC overhead limit exceeded' exception that stems from a WALA method. Here is the stack trace: Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68) at java.lang.StringBuilder.<init>(StringBuilder.java:89) at com.ibm.wala.types.TypeName$TypeNameKey.toUnicodeString(TypeName.java:319) at com.ibm.wala.types.TypeName.toUnicodeString(TypeName.java:123) at com.ibm.wala.types.Descriptor$Key.appendSemicolonIfNeeded(Descriptor.java:227) at com.ibm.wala.types.Descriptor$Key.toString(Descriptor.java:200) at com.ibm.wala.types.Descriptor.toString(Descriptor.java:109) at java.lang.String.valueOf(String.java:2994) at java.lang.StringBuilder.append(StringBuilder.java:131) at com.ibm.wala.types.MethodReference.getSignature(MethodReference.java:223) My guess is that the issue is due to the method 'UTF8Convert::fromUTF8' allocating a new Array every time it needs to construct the string. Rapid repeated invocation of this method can probably result in a frequent allocation/deallocation of the array resulting in the GC being invoked very frequently. However, I might be wrong. For your queries: 1. The total size of all the dex files included is 87.6 MB 2. No, I am not constructing a callgraph, only the class hierarchy. Note that creating the class hierarchy is not an issue. The exception occurs only when I am analyzing the constructed class hierarchy during which I invoke the 'CallSiteReference.getDeclaredTarget.getSignature' repeatedly for different methods. Thanks, Parjanya ________________________________ From: Manu Sridharan <ma...@sr...<mailto:ma...@sr...>> Sent: Friday, September 2, 2022 11:25 AM To: WALA discussion and Q&A <wal...@li...<mailto:wal...@li...>> Subject: Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature After Googling I see the error message is likely due to an OutOfMemoryError. WALA class hierarchy construction has been battle-tested with large classpaths of jars, but I haven’t done much with large dex files. It’s possible there is a memory leak somewhere. I’d say the simplest thing to try for you is to increase the max heap size, running on a machine with more RAM if possible; do you still get an OOM with 24G of heap? If you do still get an OOM with more heap, we can try to dig deeper. Out of curiosity, what is the total size of the dex files in your class hierarchy? And are you just trying to build a class hierarchy, or also a call graph? --Manu On Sep 2, 2022 at 07:48:13, Manu Sridharan <ma...@sr...<mailto:ma...@sr...>> wrote: Hi Parjanya, are you saying that WALA is throwing an exception, or that there is an error from the JVM? If it’s the former can you link to the line of code throwing the exception and paste a stack trace excerpt? Thanks, Manu On Fri, Sep 2, 2022 at 07:46 Parjanya Vyas <par...@uw...<mailto:par...@uw...>> wrote: Hi Raffi, Thanks very much for your response. I use '12000m' with -Xmx right now. Thanks, Parjanya ________________________________ From: Raffi T Khatchadourian <raf...@hu...<mailto:raf...@hu...>> Sent: Thursday, September 1, 2022 10:44 AM To: wal...@li...<mailto:wal...@li...> <wal...@li...<mailto:wal...@li...>> Subject: Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature Parjanya, What is the value that you are giving to -Xmx when you run java? On Thu, 2022-09-01 at 03:28 +0000, Parjanya Vyas wrote: Hi all, I apologize in advance if this question has been already addressed or is too naive. When using class hierarchy analysis with a large code base, I encounter the 'GC overhead limit exceeded' exception that stems from MethodReference::getCallsiteSignature method when it is invoked frequently. The exception only occurs when analyzing a large analysis scope that consists of multiple large dex files. There is no fixed point that I could observe when this exception is triggered. Can someone please help me out with this? Any guidance will be highly appreciated! I can provide more details on the analysis data/processing that I am attempting, if needed. Thanks, Parjanya Vyas _______________________________________________ Wala-wala mailing list Wal...@li...<mailto:Wal...@li...> https://lists.sourceforge.net/lists/listinfo/wala-wala _______________________________________________ Wala-wala mailing list Wal...@li...<mailto:Wal...@li...> https://lists.sourceforge.net/lists/listinfo/wala-wala -- Sent from my phone. Please pardon brevity _______________________________________________ Wala-wala mailing list Wal...@li...<mailto:Wal...@li...> https://lists.sourceforge.net/lists/listinfo/wala-wala |
