#1461 [core] RuleSetFactory: Possible threading issue due to PR#75

PMD-5.3.7
closed
core (10)
PMD
2-Critical
Bug
2016-06-25
2016-02-15
No

Discussion

  • Ryan Gustafson

    Ryan Gustafson - 2016-02-16

    I was able to trivially verify that multi-threading support was not working using the -t option. The RuleChain died fairly quickly with ConcurrentModificationExceptions.

    It might be worth adding a Test around this, to try to catch regressions via CI. A forced multi-threaded test of the Java rules against the PMD source aught to 1) not fail, 2) produce consistent results if run 2+ times. Not exactly a robust test, but hopefully stands a chance of catching obvious issues before a release happens.

    Alternatively, a series of high volume, high concurrency stress tests, could be done pre-release to try to actively shake out such problems. Running all known Rules against a baseline of source from different open source projects and comparing with previous release numbers/reports could be used to spot 1) threading issues, 2) performance problems, 3) false-positive Rule regressions. The -t and -b options are useful here. I had tried doing this once, but I never completed the effort. I'm not sure if something like this now exists.

    I have noticed some performance problems while doing analysis on this bug, I'll try to submit some PR around them when I get a chance.

     
  • Alexey Yudichev

    Alexey Yudichev - 2016-03-02

    I am using maven-pmd-plugin 3.6 which defaults to PMD 5.3.5. When switching from PMD 5.3.5 to 5.4.1 I start to get a lot of ConcurrentModificationExceptions (first one below) during the build and eventually the build may hang.

    Is there a workaround here? I need 5.4.1 as this is what the current version of PMD plugin for IntelliJ IDEA uses.

    [08:37:59][Step 2/6] Mar 02, 2016 3:37:59 AM net.sourceforge.pmd.RuleSet apply
    [08:37:59][Step 2/6] WARNING: Exception applying rule NPathComplexity on file /data/cmetrading/teamcity_home/agent6/work/d8e9b2b2b3ffbf5f/xasf-container/src/test/java/com/baml/gmt/xasf/container/agent/lifecycle/events/TestAgentLifecycleChangeEvent.java, continuing with next rule
    [08:37:59][Step 2/6] java.util.ConcurrentModificationException
    [08:37:59][Step 2/6]    at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1115)
    [08:37:59][Step 2/6]    at java.util.TreeMap$KeyIterator.next(TreeMap.java:1169)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.lang.rule.stat.StatisticalRuleHelper.applyMinimumValue(StatisticalRuleHelper.java:116)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.lang.rule.stat.StatisticalRuleHelper.apply(StatisticalRuleHelper.java:68)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.lang.java.rule.AbstractStatisticalJavaRule.apply(AbstractStatisticalJavaRule.java:29)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.lang.rule.AbstractDelegateRule.apply(AbstractDelegateRule.java:215)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.RuleSet.apply(RuleSet.java:307)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.RuleSets.apply(RuleSets.java:125)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.SourceCodeProcessor.processSource(SourceCodeProcessor.java:146)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.SourceCodeProcessor.processSourceCode(SourceCodeProcessor.java:76)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.SourceCodeProcessor.processSourceCode(SourceCodeProcessor.java:43)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.processor.PmdRunnable.call(PmdRunnable.java:79)
    [08:37:59][Step 2/6]    at net.sourceforge.pmd.processor.PmdRunnable.call(PmdRunnable.java:25)
    [08:37:59][Step 2/6]    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    [08:37:59][Step 2/6]    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    [08:37:59][Step 2/6]    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    [08:37:59][Step 2/6]    at java.lang.Thread.run(Thread.java:745)
    

    build hangs on this:

    "main" prio=10 tid=0x00007f3b4800e800 nid=0x2db1 waiting on condition [0x00007f3b4da42000]
       java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x000000041e083fe8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ExecutorCompletionService.take(ExecutorCompletionService.java:193)
        at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder.multiThreadedProjectTaskSegmentBuild(MultiThreadedBuilder.java:130)
        at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder.build(MultiThreadedBuilder.java:91)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
    
        one of PmdThreads (there are several - all doing the same)
    
        "PmdThread 7" prio=10 tid=0x00007f3a84043800 nid=0x321b runnable [0x00007f3b2ae27000]
       java.lang.Thread.State: RUNNABLE
        at java.util.TreeMap.put(TreeMap.java:560)
        at java.util.TreeSet.add(TreeSet.java:255)
        at net.sourceforge.pmd.lang.rule.stat.StatisticalRuleHelper.addDataPoint(StatisticalRuleHelper.java:47)
        at net.sourceforge.pmd.lang.java.rule.AbstractStatisticalJavaRule.addDataPoint(AbstractStatisticalJavaRule.java:19)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:66)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:247)
        at net.sourceforge.pmd.lang.java.rule.codesize.NcssMethodCountRule.visit(NcssMethodCountRule.java:26)
        at net.sourceforge.pmd.lang.java.ast.ASTMethodDeclaration.jjtAccept(ASTMethodDeclaration.java:25)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:55)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:111)
        at net.sourceforge.pmd.lang.java.ast.ASTClassOrInterfaceBodyDeclaration.jjtAccept(ASTClassOrInterfaceBodyDeclaration.java:41)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:55)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:107)
        at net.sourceforge.pmd.lang.java.ast.ASTClassOrInterfaceBody.jjtAccept(ASTClassOrInterfaceBody.java:22)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:55)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:83)
        at net.sourceforge.pmd.lang.java.ast.ASTClassOrInterfaceDeclaration.jjtAccept(ASTClassOrInterfaceDeclaration.java:26)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:55)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:223)
        at net.sourceforge.pmd.lang.java.ast.ASTTypeDeclaration.jjtAccept(ASTTypeDeclaration.java:36)
        at net.sourceforge.pmd.lang.java.rule.codesize.AbstractNcssCountRule.visit(AbstractNcssCountRule.java:55)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visit(AbstractJavaRule.java:203)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.visitAll(AbstractJavaRule.java:31)
        at net.sourceforge.pmd.lang.java.rule.AbstractJavaRule.apply(AbstractJavaRule.java:25)
        at net.sourceforge.pmd.lang.java.rule.AbstractStatisticalJavaRule.apply(AbstractStatisticalJavaRule.java:28)
        at net.sourceforge.pmd.lang.rule.AbstractDelegateRule.apply(AbstractDelegateRule.java:215)
        at net.sourceforge.pmd.RuleSet.apply(RuleSet.java:307)
        at net.sourceforge.pmd.RuleSets.apply(RuleSets.java:125)
        at net.sourceforge.pmd.SourceCodeProcessor.processSource(SourceCodeProcessor.java:146)
        at net.sourceforge.pmd.SourceCodeProcessor.processSourceCode(SourceCodeProcessor.java:76)
        at net.sourceforge.pmd.SourceCodeProcessor.processSourceCode(SourceCodeProcessor.java:43)
        at net.sourceforge.pmd.processor.PmdRunnable.call(PmdRunnable.java:79)
        at net.sourceforge.pmd.processor.PmdRunnable.call(PmdRunnable.java:25)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
    
     
  • Andreas Dangel

    Andreas Dangel - 2016-03-02

    You're right, these exceptions are occurring probably due to this bug.

    There seems no workaround here with maven-pmd-plugin, as the threads-option can't be changed (it defaults to Runtime.getRuntime().availableProcessors().

    If you really need PMD 5.4.1 right now, the only option seems to be, to run it from commandline or via ant - the PMDTask for ant also supports the "threads" option. To workaround this threading issue, you'll need to run PMD with the option -threads 0.

     
  • Andreas Dangel

    Andreas Dangel - 2016-03-13
    • status: open --> closed
    • assigned_to: Andreas Dangel
    • Milestone: New Tickets --> PMD-5.3.7
    • Priority: 3-Major --> 2-Critical
     
  • Andreas Dangel

    Andreas Dangel - 2016-03-13

    I've added a unit test (see 54ce3036) to hopefully prevent the problem in the future. I basically reverted the commit from PR #75.
    Under documentation, I added one more paragraph -> see How to write a rule

    The fix will be included in the next PMD versions (5.3.7, 5.4.2 and 5.5.0).

    @rgustav - Please create a separate bug for the performance issues you've seen. Thanks!

     
    Last edit: Andreas Dangel 2016-04-30
  • Andreas Dangel

    Andreas Dangel - 2016-06-25
    • labels: --> core
    • summary: Possible threading issue due to PR#75 --> [core] RuleSetFactory: Possible threading issue due to PR#75
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks