#745 ConstantPoolSorter.visitProgramClass: "Comparison method violates its general contract" (6.1.0beta2)

[Peter Burka]

ProGuard 6.1.0beta2, running on OpenJDK 11 against classes compiled on OpenJDK 11, crashes with the following stack trace:

java.lang.IllegalArgumentException: Comparison method violates its general contract!
    at java.base/java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:748)
    at java.base/java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:485)
    at java.base/java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:413)
    at java.base/java.util.ComparableTimSort.sort(ComparableTimSort.java:213)
    at java.base/java.util.Arrays.sort(Arrays.java:1315)
    at proguard.classfile.editor.ConstantPoolSorter.visitProgramClass(ConstantPoolSorter.java:75)
    at proguard.classfile.ProgramClass.accept(ProgramClass.java:430)
    at proguard.classfile.editor.ClassElementSorter.visitProgramClass(ClassElementSorter.java:47)
    at proguard.classfile.ProgramClass.accept(ProgramClass.java:430)
    at proguard.classfile.ClassPool.classesAccept(ClassPool.java:125)
    at proguard.ProGuard.sortClassElements(ProGuard.java:514)
    at proguard.ProGuard.execute(ProGuard.java:216)
    at proguard.ant.ProGuardTask.execute(ProGuardTask.java:335)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
    ...

I'm afraid I don't have a simple reproducer, but I can try to dig into it a bit if the cause isn't obvious.

[Eric Lafortune]

Thanks for your report. I can see a bug that could be triggered by code with a statically initialized array of floats or doubles, where one of the elements is an unusual value like NaN, +Inf, or -Inf. I'll start by fixing that.

[Peter Burka]

Thanks Eric. The codebase that demonstrated this definitely has static arrays of doubles with non-finite values.

[Eric Lafortune]

The bug has now been fixed for the upcoming ProGuard 6.1 beta3. You could already try the attached patch.

[Scott Rich]

This is great news. Any idea when 6.1beta3 will release? Or 6.1?

[Peter Burka]

I tried the patch but the error still occurred. However, the patch did give me some clues as to where to start investigating, and I think I found a second cause: at the top of ComparableConstant.java, InvokeDynamic and MethodHandle are both assigned priority=10. I think this causes them to sort inconsistently (see line 116). I changed MethodHandle's priority to 17 and my codebase now builds successfully with ProGuard.

[Eric Lafortune]

Great find, thanks! I had checked and double-checked the list badly, This is the sort of bugs where extra pairs of eyes are so useful.