Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature

[Parjanya V. <par...@uw...>]

Hi Manu,

Thanks a lot for your response.
It is the JVM that throws the 'GC overhead limit exceeded' exception that stems from a WALA method.
Here is the stack trace:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
      at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
      at java.lang.StringBuilder.<init>(StringBuilder.java:89)
      at com.ibm.wala.types.TypeName$TypeNameKey.toUnicodeString(TypeName.java:319)
      at com.ibm.wala.types.TypeName.toUnicodeString(TypeName.java:123)
      at com.ibm.wala.types.Descriptor$Key.appendSemicolonIfNeeded(Descriptor.java:227)
      at com.ibm.wala.types.Descriptor$Key.toString(Descriptor.java:200)
      at com.ibm.wala.types.Descriptor.toString(Descriptor.java:109)
      at java.lang.String.valueOf(String.java:2994)
      at java.lang.StringBuilder.append(StringBuilder.java:131)
      at com.ibm.wala.types.MethodReference.getSignature(MethodReference.java:223)

My guess is that the issue is due to the method 'UTF8Convert::fromUTF8' allocating a new Array every time it needs to construct the string. Rapid repeated invocation of this method can probably result in a frequent allocation/deallocation of the array resulting in the GC being invoked very frequently. However, I might be wrong.

For your queries:

  1.  The total size of all the dex files included is 87.6 MB
  2.  No, I am not constructing a callgraph, only the class hierarchy.

Note that creating the class hierarchy is not an issue. The exception occurs only when I am analyzing the constructed class hierarchy during which I invoke the 'CallSiteReference.getDeclaredTarget.getSignature' repeatedly for different methods.

Thanks,
Parjanya

________________________________
From: Manu Sridharan <ma...@sr...>
Sent: Friday, September 2, 2022 11:25 AM
To: WALA discussion and Q&A <wal...@li...>
Subject: Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature

After Googling I see the error message is likely due to an OutOfMemoryError.  WALA class hierarchy construction has been battle-tested with large classpaths of jars, but I haven’t done much with large dex files.  It’s possible there is a memory leak somewhere.  I’d say the simplest thing to try for you is to increase the max heap size, running on a machine with more RAM if possible; do you still get an OOM with 24G of heap?

If you do still get an OOM with more heap, we can try to dig deeper.  Out of curiosity, what is the total size of the dex files in your class hierarchy?  And are you just trying to build a class hierarchy, or also a call graph?

--Manu

On Sep 2, 2022 at 07:48:13, Manu Sridharan <ma...@sr...<mailto:ma...@sr...>> wrote:
Hi Parjanya, are you saying that WALA is throwing an exception, or that there is an error from the JVM? If it’s the former can you link to the line of code throwing the exception and paste a stack trace excerpt?

Thanks,
Manu

On Fri, Sep 2, 2022 at 07:46 Parjanya Vyas <par...@uw...<mailto:par...@uw...>> wrote:
Hi Raffi,

Thanks very much for your response.
I use '12000m' with -Xmx right now.

Thanks,
Parjanya
________________________________
From: Raffi T Khatchadourian <raf...@hu...<mailto:raf...@hu...>>
Sent: Thursday, September 1, 2022 10:44 AM
To: wal...@li...<mailto:wal...@li...> <wal...@li...<mailto:wal...@li...>>
Subject: Re: [Wala-wala] GC overhead limit exceeded when frequently invoking getCallsiteSignature

Parjanya,

What is the value that you are giving to -Xmx when you run java?

On Thu, 2022-09-01 at 03:28 +0000, Parjanya Vyas wrote:
Hi all,

I apologize in advance if this question has been already addressed or is too naive. When using class hierarchy analysis with a large code base, I encounter the 'GC overhead limit exceeded' exception that stems from MethodReference::getCallsiteSignature method when it is invoked frequently.

The exception only occurs when analyzing a large analysis scope that consists of multiple large dex files. There is no fixed point that I could observe when this exception is triggered.

Can someone please help me out with this? Any guidance will be highly appreciated! I can provide more details on the analysis data/processing that I am attempting, if needed.

Thanks,
Parjanya Vyas
_______________________________________________
Wala-wala mailing list
Wal...@li...<mailto:Wal...@li...>
https://lists.sourceforge.net/lists/listinfo/wala-wala
_______________________________________________
Wala-wala mailing list
Wal...@li...<mailto:Wal...@li...>
https://lists.sourceforge.net/lists/listinfo/wala-wala
--
Sent from my phone. Please pardon brevity