#631 java.lang.VerifyError: Bad access to protected data in invokevirtual

[Jason Lohrey]

Hi Eric et. al,

We have used ProGuard for many, many years in a pretty large application. After updating to 5.3.1 (and now reproduced in 5.3.2) we have found a serious issue with obfuscated binaries that violate JVM rules for invoking protected virtual methods.

I have reproduced an analysis and a workaround that we have found. Whilst there is a workaround, we wil have to identify each and every case (and there is a lot of cases that from the past 15 years of development) in order provide work arounds, so this is a serious issue.

The issue is this:

We were trying to instantiate a sub-class of this class:

public abstract class DataStorePolicy {

public static final StringType POLICY_NAME  = new StringType(128);

public static class ExNotFound extends Throwable {
    public ExNotFound(String name) {
        super("The data store policy '" + name + "' does not exist");
    }
}

public static class ExUnableToAllocateStore extends Throwable {
    public ExUnableToAllocateStore(String name,List<String> stores) {
        super(message(name,stores));
    }

    private static String message(String name,List<String> stores) {
        String msg = "No usable store found by data store policy '" + name + "'";
        if ( stores != null ) {
            msg += ": candidate stores=" + stores;
        }

        return msg;
    }
}

….

Because this class contains nested exceptions, the obfuscator kept the full class path for this class (DataStorePolicy).

The derived classes were however, obfuscated and moved into a new package “arc”.

The problem is, the JVM was doing validation of attempts to access protected methods from another package (runtime). It appears the the obfuscator (Proguard 5.3.1) did not properly update the mapping tables in this case .. and the sub-classes were in a different package to the parent class (DataStorePolicy) and hence the error:

java.lang.VerifyError: Bad access to protected data in invokevirtual

Was the result (see full stack below).

Solution:

Move the nested exceptions into their own separate classes (which is what we want anyway), and the DataStorePolicy is now fully obfuscated and in the same package as the sub-classes.

Jason

java.lang.VerifyError: Bad access to protected data in invokevirtual
Exception Details:
Location:
arc/asX.a(Larc/mf/modules/asset/xodb/store/policy/DataStorePolicy;Larc/bVj;Ljava/util/List;Ljava/util/List;)Ljava/lang/String; @56: invokevirtual
Reason:
Type 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy' (current frame, stack[0]) is not assignable to 'arc/asX'
Current Frame:
bci: @56
flags: { }
locals: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'arc/bVj', top, top, 'java/util/ArrayList', integer, 'java/util/List' }
stack: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'java/util/List' }
Bytecode:
0000000: 2cb8 0029 9900 0501 b0bb 0010 592c b700
0000010: 2d3a 0404 3605 1505 9900 792b 1904 b600
0000020: 1f3a 0619 06c7 0006 a700 6919 06b9 0032
0000030: 0100 9e00 4f2a 1906 b600 283a 0719 07b8
0000040: 0023 3a08 1908 b600 22b8 0019 3a09 1909
0000050: c600 0d2b 1907 b600 2057 1907 b019 0419
0000060: 07b9 0031 0200 5719 04b9 0030 0100 9900
0000070: 06a7 0010 1906 1907 b900 3102 0057 a7ff
0000080: ad19 04b9 0030 0100 9900 06a7 0006 a7ff
0000090: 8801 b0
Stackmap Table:
chop_frame(@9,1)
full_frame(@22,{Object[#12],Object[#10],Top,Top,Object[#16],Integer},{})
append_frame(@43,Object[#18])
append_frame(@93,Object[#15])
same_frame(@116)
chop_frame(@129,2)
same_frame(@142)
full_frame(@145,{},{})

at arc.asV.a(SourceFile:23)
at arc.mf.modules.asset.DataStore.c(SourceFile:2030)
at arc.yw.a(SourceFile:129)
at arc.mf.modules.asset.Asset.a(SourceFile:1327)
at arc.adX.a(SourceFile:374)
at arc.mf.modules.asset.xodb.XODBAssetModule.a(SourceFile:206)
at arc.mf.server.JavaModules.e(SourceFile:289)
at arc.mf.server.JavaModules.d(SourceFile:48)
at arc.btJ.a(SourceFile:203)
at arc.btJ.execute(SourceFile:200)
at arc.xodb.Transaction.a(SourceFile:1317)
at arc.xodb.Transaction.b(SourceFile:1300)
at arc.btH.a(SourceFile:200)
at arc.btH.a(SourceFile:177)
at arc.bMi.a(SourceFile:11)
at arc.mf.server.JavaModules.a(SourceFile:177)
at arc.buG.doExecute(SourceFile:2425)
at arc.utils.Task.a(SourceFile:698)
at arc.utils.Task.run(SourceFile:670)
at arc.bWB.a(SourceFile:511)
at arc.bWB.run(SourceFile:459)
at arc.bWA.run(SourceFile:303)

[Jason Lohrey]

With another case just occuring, I can say the recipe is as follows:

• Create an abstract class
• Have nested classes that are maked as "keep" (excplicitly or by inheritance)
• Configure repackageclasses (e.g. repackageclasses="arc")
• Have another class in the same package call protected methods in the abstract class

The problem (as stated in the first submission) is that the classes that make the call to the protected methods are repackaged into another package, which is no longer the same package as the class being accessed which has to be retained because if implicit "keep" requirements.

This isues to work in 4.x versions of ProGuard. With the 5.x versions it causes issues.

Jason

[Eric Lafortune]

Thanks for your report. I'll see if I can create a small sample based on your description.

[Jason Lohrey]

Thanks Eric. I’m happy to help — I found the pattern below from a large application / code base, and did not go as far as creating a sample set of code that demonstrates the issue — hoping it might be obvious, removing the need for a sample!

On 19 Dec 2016, at 10:46 am, Eric Lafortune lafortune@users.sf.net wrote:

Thanks for your report. I'll see if I can create a small sample based on your description.

[bugs:#631] https://sourceforge.net/p/proguard/bugs/631/ java.lang.VerifyError: Bad access to protected data in invokevirtual

Status: open
Group: v5.3
Created: Tue Dec 13, 2016 04:29 AM UTC by Jason Lohrey
Last Updated: Tue Dec 13, 2016 05:06 AM UTC
Owner: nobody

Hi Eric et. al,

We have used ProGuard for many, many years in a pretty large application. After updating to 5.3.1 (and now reproduced in 5.3.2) we have found a serious issue with obfuscated binaries that violate JVM rules for invoking protected virtual methods.

I have reproduced an analysis and a workaround that we have found. Whilst there is a workaround, we wil have to identify each and every case (and there is a lot of cases that from the past 15 years of development) in order provide work arounds, so this is a serious issue.

The issue is this:

We were trying to instantiate a sub-class of this class:

public abstract class DataStorePolicy {

public static final StringType POLICY_NAME = new StringType(128);

public static class ExNotFound extends Throwable {
public ExNotFound(String name) {
super("The data store policy '" + name + "' does not exist");
}
}

public static class ExUnableToAllocateStore extends Throwable {
public ExUnableToAllocateStore(String name,List<string> stores) {
super(message(name,stores));
}</string>

private static String message(String name,List<String> stores) {
    String msg = "No usable store found by data store policy '" + name + "'";
    if ( stores != null ) {
        msg += ": candidate stores=" + stores;
    }

    return msg;
}

}
….

Because this class contains nested exceptions, the obfuscator kept the full class path for this class (DataStorePolicy).

The derived classes were however, obfuscated and moved into a new package “arc”.

The problem is, the JVM was doing validation of attempts to access protected methods from another package (runtime). It appears the the obfuscator (Proguard 5.3.1) did not properly update the mapping tables in this case .. and the sub-classes were in a different package to the parent class (DataStorePolicy) and hence the error:

java.lang.VerifyError: Bad access to protected data in invokevirtual

Was the result (see full stack below).

Solution:

Move the nested exceptions into their own separate classes (which is what we want anyway), and the DataStorePolicy is now fully obfuscated and in the same package as the sub-classes.

Jason

java.lang.VerifyError: Bad access to protected data in invokevirtual
Exception Details:
Location:
arc/asX.a(Larc/mf/modules/asset/xodb/store/policy/DataStorePolicy;Larc/bVj;Ljava/util/List;Ljava/util/List;)Ljava/lang/String; @56: invokevirtual
Reason:
Type 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy' (current frame, stack[0]) is not assignable to 'arc/asX'
Current Frame:
bci: @56
flags: { }
locals: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'arc/bVj', top, top, 'java/util/ArrayList', integer, 'java/util/List' }
stack: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'java/util/List' }
Bytecode:
0000000: 2cb8 0029 9900 0501 b0bb 0010 592c b700
0000010: 2d3a 0404 3605 1505 9900 792b 1904 b600
0000020: 1f3a 0619 06c7 0006 a700 6919 06b9 0032
0000030: 0100 9e00 4f2a 1906 b600 283a 0719 07b8
0000040: 0023 3a08 1908 b600 22b8 0019 3a09 1909
0000050: c600 0d2b 1907 b600 2057 1907 b019 0419
0000060: 07b9 0031 0200 5719 04b9 0030 0100 9900
0000070: 06a7 0010 1906 1907 b900 3102 0057 a7ff
0000080: ad19 04b9 0030 0100 9900 06a7 0006 a7ff
0000090: 8801 b0
Stackmap Table:
chop_frame(@9,1)
full_frame(@22,{Object[#12] https://sourceforge.net/p/proguard/bugs/12/,Object[#10] https://sourceforge.net/p/proguard/bugs/10/,Top,Top,Object[#16] https://sourceforge.net/p/proguard/bugs/16/,Integer},{})
append_frame(@43,Object[#18] https://sourceforge.net/p/proguard/bugs/18/)
append_frame(@93,Object[#15] https://sourceforge.net/p/proguard/bugs/15/)
same_frame(@116)
chop_frame(@129,2)
same_frame(@142)
full_frame(@145,{},{})

at arc.asV.a(SourceFile:23)
at arc.mf.modules.asset.DataStore.c(SourceFile:2030)
at arc.yw.a(SourceFile:129)
at arc.mf.modules.asset.Asset.a(SourceFile:1327)
at arc.adX.a(SourceFile:374)
at arc.mf.modules.asset.xodb.XODBAssetModule.a(SourceFile:206)
at arc.mf.server.JavaModules.e(SourceFile:289)
at arc.mf.server.JavaModules.d(SourceFile:48)
at arc.btJ.a(SourceFile:203)
at arc.btJ.execute(SourceFile:200)
at arc.xodb.Transaction.a(SourceFile:1317)
at arc.xodb.Transaction.b(SourceFile:1300)
at arc.btH.a(SourceFile:200)
at arc.btH.a(SourceFile:177)
at arc.bMi.a(SourceFile:11)
at arc.mf.server.JavaModules.a(SourceFile:177)
at arc.buG.doExecute(SourceFile:2425)
at arc.utils.Task.a(SourceFile:698)
at arc.utils.Task.run(SourceFile:670)
at arc.bWB.a(SourceFile:511)
at arc.bWB.run(SourceFile:459)
at arc.bWA.run(SourceFile:303)
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/proguard/bugs/631/ https://sourceforge.net/p/proguard/bugs/631/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/ https://sourceforge.net/auth/subscriptions/

http://www.arcitecta.com/
Jason Lohrey / CTO
jason.lohrey@arcitecta.com jason.lohrey@arcitecta.com / +61 418 364 941 <tel:+61418644941>
Arcitecta
+61 3 9029 3437 <tel:+61390293437>
15 Little Bakers Lane, Northcote, Vic, 3070
www.arcitecta.com http://www.arcitecta.com/</tel:+61390293437></tel:+61418644941>

[Eric Lafortune]

I haven't been able to create a failing sample yet. My simple cases seem to work: the obfuscated classes that are involved are not moved to a different package, so the protected method remains accessible. If you have a sample, that would be helpful.

[Jason Lohrey]

OK .. I’ll see if I can reproduce — might take me a few days.

On 19 Dec 2016, at 11:28 am, Eric Lafortune lafortune@users.sf.net wrote:

I haven't been able to create a failing sample yet. My simple cases seem to work: the obfuscated classes that are involved are not moved to a different package, so the protected method remains accessible. If you have a sample, that would be helpful.

[bugs:#631] https://sourceforge.net/p/proguard/bugs/631/ java.lang.VerifyError: Bad access to protected data in invokevirtual

Status: open
Group: v5.3
Created: Tue Dec 13, 2016 04:29 AM UTC by Jason Lohrey
Last Updated: Sun Dec 18, 2016 11:46 PM UTC
Owner: nobody

Hi Eric et. al,

We have used ProGuard for many, many years in a pretty large application. After updating to 5.3.1 (and now reproduced in 5.3.2) we have found a serious issue with obfuscated binaries that violate JVM rules for invoking protected virtual methods.

I have reproduced an analysis and a workaround that we have found. Whilst there is a workaround, we wil have to identify each and every case (and there is a lot of cases that from the past 15 years of development) in order provide work arounds, so this is a serious issue.

The issue is this:

We were trying to instantiate a sub-class of this class:

public abstract class DataStorePolicy {

public static final StringType POLICY_NAME = new StringType(128);

public static class ExNotFound extends Throwable {
public ExNotFound(String name) {
super("The data store policy '" + name + "' does not exist");
}
}

public static class ExUnableToAllocateStore extends Throwable {
public ExUnableToAllocateStore(String name,List<string> stores) {
super(message(name,stores));
}</string>

private static String message(String name,List<String> stores) {
    String msg = "No usable store found by data store policy '" + name + "'";
    if ( stores != null ) {
        msg += ": candidate stores=" + stores;
    }

    return msg;
}

}
….

Because this class contains nested exceptions, the obfuscator kept the full class path for this class (DataStorePolicy).

The derived classes were however, obfuscated and moved into a new package “arc”.

The problem is, the JVM was doing validation of attempts to access protected methods from another package (runtime). It appears the the obfuscator (Proguard 5.3.1) did not properly update the mapping tables in this case .. and the sub-classes were in a different package to the parent class (DataStorePolicy) and hence the error:

java.lang.VerifyError: Bad access to protected data in invokevirtual

Was the result (see full stack below).

Solution:

Move the nested exceptions into their own separate classes (which is what we want anyway), and the DataStorePolicy is now fully obfuscated and in the same package as the sub-classes.

Jason

java.lang.VerifyError: Bad access to protected data in invokevirtual
Exception Details:
Location:
arc/asX.a(Larc/mf/modules/asset/xodb/store/policy/DataStorePolicy;Larc/bVj;Ljava/util/List;Ljava/util/List;)Ljava/lang/String; @56: invokevirtual
Reason:
Type 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy' (current frame, stack[0]) is not assignable to 'arc/asX'
Current Frame:
bci: @56
flags: { }
locals: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'arc/bVj', top, top, 'java/util/ArrayList', integer, 'java/util/List' }
stack: { 'arc/mf/modules/asset/xodb/store/policy/DataStorePolicy', 'java/util/List' }
Bytecode:
0000000: 2cb8 0029 9900 0501 b0bb 0010 592c b700
0000010: 2d3a 0404 3605 1505 9900 792b 1904 b600
0000020: 1f3a 0619 06c7 0006 a700 6919 06b9 0032
0000030: 0100 9e00 4f2a 1906 b600 283a 0719 07b8
0000040: 0023 3a08 1908 b600 22b8 0019 3a09 1909
0000050: c600 0d2b 1907 b600 2057 1907 b019 0419
0000060: 07b9 0031 0200 5719 04b9 0030 0100 9900
0000070: 06a7 0010 1906 1907 b900 3102 0057 a7ff
0000080: ad19 04b9 0030 0100 9900 06a7 0006 a7ff
0000090: 8801 b0
Stackmap Table:
chop_frame(@9,1)
full_frame(@22,{Object[#12] https://sourceforge.net/p/proguard/bugs/12/,Object[#10] https://sourceforge.net/p/proguard/bugs/10/,Top,Top,Object[#16] https://sourceforge.net/p/proguard/bugs/16/,Integer},{})
append_frame(@43,Object[#18] https://sourceforge.net/p/proguard/bugs/18/)
append_frame(@93,Object[#15] https://sourceforge.net/p/proguard/bugs/15/)
same_frame(@116)
chop_frame(@129,2)
same_frame(@142)
full_frame(@145,{},{})

at arc.asV.a(SourceFile:23)
at arc.mf.modules.asset.DataStore.c(SourceFile:2030)
at arc.yw.a(SourceFile:129)
at arc.mf.modules.asset.Asset.a(SourceFile:1327)
at arc.adX.a(SourceFile:374)
at arc.mf.modules.asset.xodb.XODBAssetModule.a(SourceFile:206)
at arc.mf.server.JavaModules.e(SourceFile:289)
at arc.mf.server.JavaModules.d(SourceFile:48)
at arc.btJ.a(SourceFile:203)
at arc.btJ.execute(SourceFile:200)
at arc.xodb.Transaction.a(SourceFile:1317)
at arc.xodb.Transaction.b(SourceFile:1300)
at arc.btH.a(SourceFile:200)
at arc.btH.a(SourceFile:177)
at arc.bMi.a(SourceFile:11)
at arc.mf.server.JavaModules.a(SourceFile:177)
at arc.buG.doExecute(SourceFile:2425)
at arc.utils.Task.a(SourceFile:698)
at arc.utils.Task.run(SourceFile:670)
at arc.bWB.a(SourceFile:511)
at arc.bWB.run(SourceFile:459)
at arc.bWA.run(SourceFile:303)
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/proguard/bugs/631/ https://sourceforge.net/p/proguard/bugs/631/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/ https://sourceforge.net/auth/subscriptions/

http://www.arcitecta.com/
Jason Lohrey / CTO
jason.lohrey@arcitecta.com jason.lohrey@arcitecta.com / +61 418 364 941 <tel:+61418644941>
Arcitecta
+61 3 9029 3437 <tel:+61390293437>
15 Little Bakers Lane, Northcote, Vic, 3070
www.arcitecta.com http://www.arcitecta.com/</tel:+61390293437></tel:+61418644941>