Menu

Thoughts and advice on PMD results usability

2015-02-11
2015-02-15
  • Boris Baldassari

    Hiho dear colleagues,

    For the last ten years, I've worked with many different companies in the industry, and I've come to a few conclusions:

    • PMD is widely used. Not only by open source projects, but also by the industry (banks, IT, insurance, service..). Even if it's not a corporate choice, developers use it a lot and it is IMHO definitely a part of the software quality landscape nowadays.
    • Many people often don't get the best of it because:
      • firstly they do not configure it to fit their needs and context, and
      • secondly (and a consequence of the first point) they have thousands of violation on rules they don't necessarily understand and thus fail to really benefit from the tool's experience.

    Considering this, I wondered if it could be possible to give them some hints on

    • their configuration, e.g. remove this rule because you won't get its benefits at this point, and
    • their quality, e.g. provide a method, like: select a few rules with a high priority and a low number of violations, so you can really acquire this practice, and then move on.

    I gave it a try and wrote a Rhtml dynamic document for that purpose. Here are some results on a few open-source sample projects:

    This is really a pre-alpha draft, so please be gentle. I would be really glad to have some feedback and advice and ways to improve it.

    PS: The code to reproduce these pages is fully available on GitHub : https://github.com/borisbaldassari/Distillery and more information on the document itself is available at http://distillery.castalia.camp/blobs/blob_pmd_analysis.html

    Thanks in advance for your enlightenment,

    --
    Boris

     

    Last edit: Boris Baldassari 2015-02-11
  • Andreas Dangel

    Andreas Dangel - 2015-02-14

    This sounds very interesting!
    I'd really like to integrate this directly into PMD, e.g. via an extra report format :)

    And your description of the best practices is definitively better than http://pmd.sourceforge.net/pmd-5.2.3/overview/bestpractices.html ...

    There are also old discussions about how to best organize all the rules in rulesets [1].
    I have some ideas around creating "specialized" rulesets - e.g. clean code rules, or OWASP related rules, or code style/formatting rules. These would be rulesets in addition to the existing one (to keep backwards compatibility) but could be easier for a project to pick for a start. E.g. a ruleset "must have rules" might be helpful - it would be a subjective selection of Prio 1 rules...

    I guess, you rely on the priority with whatever is defined in the original rulesets, right? So, reviewing these might also make sense...

    Regards,
    Andreas

     
  • Boris Baldassari

    Hi Andreas,

    Thanks for your reply.

    I'd really like to integrate this directly into PMD, 
    e.g. via an extra report format :)
    

    I could do this, but: this uses R + knitr + some R packages, which is quite a big setup for a single generation. Or it would severely increase the number of dependencies for PMD.. What do you think?

    As for the rulesets..

    I didn't know of the past discussions about the rulesets, thanks for pointing it out. The reference mechanism would definitely come handy for building new rulesets. BUt, well, I have to give it a few more thoughts before having an opinion on the "right" rulesets... The fact that the mentionned discussion didn't come up with pragmatic results is quite representative of how difficult it is to define "good" rulesets, because it is heavily dependent on the development context...

    Using priority 1 rules to define a "must have" ruleset sounds good, IMHO. Another point I would bring up is it would be good to have a categorisation of rules in quality attributes, using say the ISO9126 quality model: maintainability, reliability, reusability, etc. Many users and systems actually do it and maintain their own mapping [see next paragraph], so it would bring some standardisation if directly included in PMD. It would also really help reusing PMD results in software quality assessment processes (which is where is moving QA nowadays, as far as I can say).

    Examples of mapping between rules and quality attributes:

    • SQuORE proposes its own mapping, which can be customised by users.
    • SonarQube does it (or did it, they're removing many docs about it since they announced they would drop PMD from their default setup).
    • Kiuwan requires users to do the mapping. Kiuwan looks great, but is not free.
    • Tocea did it before they were acquired by Kiuwan (can't find a reference now, their links have vanished).

    Adding quality attributes would simply mean adding a parameter or tag in rule definition files, would not break legacy analysis processes, and would introduce a significant feature for PMD usage -- just my opinion. I could work on that, provided that someone can review the work. This would be an opportunity to review the priorities, too.

    Any thought?

     

Log in to post a comment.