Safety certification

[Daniel Marjamäki]

For your information we (Cppcheck Solutions AB) are taking Cppcheck through a safety certification.

We have documented open source Cppcheck development process and plan to get this formally approved. We have tried to make minimal requirements on open source contributors. It will be business as usual as much as possible.

Here are some key points:

In my opinion, people are good at creating tickets before working on some fixes. But maybe we can be better. If you make some functional changes then please make a trac ticket about the change first. A small refactoring that does not have a ticket will not be a disaster - my plan is to make a collect ticket for every release that collects smaller changes/refactorings..

The intention is that people will use test-driven-development. Write the test, then fix the code. I think this is most common approach for everybody. I will not try to enforce that you always follow this. We do usually check that tests are provided when PRs are shared, please continue with that. There are cases when a test is not required. For instance if the manual is changed.

Contributors will be allowed to use any OS/IDE/compiler/debugger/etc they like during development. This is not usual for certified software as far as I understand.

Release builds must be reproducable.

[Daniel Marjamäki]

I intend to create new priorities for tickets:
safety-critical, safety-major and safety-cosmetic.

Here is an explanation of these priorities:

safety-critical:

These will more or less block Cppcheck releases. I cannot claim that a Cppcheck binary is safe if it has open safety-critical issues.
In general false positives, false negatives, crashes, hang, regressions, etc are not safety-critical.

• It will be required that Cppcheck will detect the bugs in our test suite that we assert that we do find. A problem to detect one of these bugs (with that exact test code) will be safety-critical.
• If Cppcheck abort the analysis of a file but Cppcheck is silent about it and finish successfully; then that is a safety critical issue. Limited data flow for performance reasons is not critical unless checking our test code does not produce the expected output.

safety-major:

• Regression, false negative not covered by testing, if checker is safety-related
• False negatives, if checker is safety-related
• False positives, if checker is safety-related
• Fixing usability issues that are misleading users.
  safety-related : checker implements a Misra rule or is about undefined behavior or implementation defined behavior.
  We can make a list of safety-related checks.

safety-cosmetic:

• New checks
• Aborted analysis with a clear message and non-zero exit code (syntax errors, internal errors, crashes etc)
• Usability
• etc