Yes, it is the only possible way without changing cppcheck source. The better solution could be generate debug output in some structured format (json, xml), when the special CLI argument is set. I'll look at it later. I think, now the main thing is to get a working script that help to find problems in the expressions engine. And the current implementation seems to work for me.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes I think this seems like a good solution right now. When you write "Constraints" I think that is not the term I'd use.. maybe "info".
When I say "constraints" I mean conditions in a execution path. I am not professor in abstract interpretation so I might have confused the terminology..
I hope the debug output is a bit more clear now..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When you write "Constraints" I think that is not the term I'd use.. maybe "info".
The term "constraints" is equal to "equations of types" in type theory. This seems more or less similar to symbolic expressions for me, because they also set properties of the current statement in its symbolic representation. But I'm not sure either. Introductory papers [0] on abstract interpretation use the term "expressions".
We will definitely not make a mistake by name it "Information".
I hope the debug output is a bit more clear now..
Great work, it looks much better now.
Last edit: Georgiy Komarov 2020-12-13
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I would be very interested to get your input about what you would consider to be practical approaches in your real project;
I also believe that there is a lot of work needed in the abstract interpretation to make that really useful. It's tricky to interpret c++ code on a high level.
Last edit: Daniel Marjamäki 2020-12-12
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One use case I plan on trying to use cppcheck's bug hunting analysis for is the same use case I sometimes use flawfinder for. Which is trying to find memory problems when I can't use Asan, Fortify Source, or Valgrind. A lot of the code bases I have delt with were large legacy code bases. I would uses flawfinder to find every place it would find a potential buffer overflow and exam the list it gave me one by one. So I see cppcheck's bug hunting analysis helping in cases where you might not be able to use a dynamic analysis tool but you still need a tool to help you to narrow the problem down or atleast priorities where to start looking.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I would say my 2nd use case is less practical for most people, but it is a use case that I have misused the cppcheck misra addon for in the past. My code bases don't have to be Misra complaint, but occasionally I get very unrealistic deadlines for code reviews. I also find that no code base is accidently misra compliant. So I use ( or misuse ) the misra addon output to help keep my brain engaged about the code instead of for misra compliance. While being told that a variable has a naming uniqueness violation is ok. It tells my brain to pay extra attention to that variable so I don't confuse it for another. I'm hoping the output from the bug hunting analysis leads to better questions for my brain to think about the code under review. For example having the bug hunting analysis tell me that it can't prove that something isn't a buffer overflow or that a variable hasn't been ininitialized seems like really useful things for my brain to think about during the review.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I like this idea. This is a logical continuation of our work on the scripts for CVE checking.
Moreover, it should be possible to implement it now, even without support diffs in bug-hunting mode. We can just sort unrelated output in the script. This will work on GitHub infrastructure, so we don't need to care about speed and resource efficiency at all.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm hoping the output from the bug hunting analysis leads to better questions for my brain to think about the code under review. For example having the bug hunting analysis tell me that it can't prove that something isn't a buffer overflow or that a variable hasn't been ininitialized seems like really useful things for my brain to think about during the review.
+5! Our codebase strives to be MISRA compliant although not a 100% requirement (yet) Sometimes it can be tedious to deal with the MISRA restrictions in an embedded codebase that has a lot of interfacing with hardware but overall I think it results in a much more robust codebase that is easer to read and follow.
Moreover, it should be possible to implement it now, even without support diffs in bug-hunting mode. We can just sort unrelated output in the script. This will work on GitHub infrastructure, so we don't need to care about speed and resource efficiency at all.
This would be very useful. We currently run cppcheck + MISRA in our CI and any violation blocks the PR. Having a "this look suspicious" suggestion that would not block CI but alert reviewers to pay close attention would be fantastic.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It's where Jenkins is setup as a code reviewer in Gerrit. You can then have Jenkins take all the warnings from the code and mark them in the reviews. While I have never used this setup my self I always thought it was very interesting.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Nod. I wish we could use Gerrit we used it when my team was smaller and I think its revew tools were superior to GitHub but unfortunately we had to switch to only GitHub reviews due to the complexity of setting up Gerrit server that would be publicly accessible (for the 3rd parties we work with that we don't want on the VPN) yet with sufficient authentication to keep private stuff private. Been a few years now since I looked at it but switching away from GitHub would be a lot of effort at this point.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for your interest it is encouraging. :-)
I have the feeling that the analysis got slower since cppcheck-2.3. Imho, speed will not be critical for some use cases. it would probably be ok if it takes weeks to finish the analysis if that means all the bugs are found. but then there are use cases where it has to finish more quickly.. so we might need to have some configuration..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Yes, it is the only possible way without changing cppcheck source. The better solution could be generate debug output in some structured format (json, xml), when the special CLI argument is set. I'll look at it later. I think, now the main thing is to get a working script that help to find problems in the expressions engine. And the current implementation seems to work for me.
Yes I think this seems like a good solution right now. When you write "Constraints" I think that is not the term I'd use.. maybe "info".
When I say "constraints" I mean conditions in a execution path. I am not professor in abstract interpretation so I might have confused the terminology..
I hope the debug output is a bit more clear now..
The term "constraints" is equal to "equations of types" in type theory. This seems more or less similar to symbolic expressions for me, because they also set properties of the current statement in its symbolic representation. But I'm not sure either. Introductory papers [0] on abstract interpretation use the term "expressions".
We will definitely not make a mistake by name it "Information".
Great work, it looks much better now.
Last edit: Georgiy Komarov 2020-12-13
I also believe that there is a lot of work needed in the abstract interpretation to make that really useful. It's tricky to interpret c++ code on a high level.
Last edit: Daniel Marjamäki 2020-12-12
One use case I plan on trying to use cppcheck's bug hunting analysis for is the same use case I sometimes use flawfinder for. Which is trying to find memory problems when I can't use Asan, Fortify Source, or Valgrind. A lot of the code bases I have delt with were large legacy code bases. I would uses flawfinder to find every place it would find a potential buffer overflow and exam the list it gave me one by one. So I see cppcheck's bug hunting analysis helping in cases where you might not be able to use a dynamic analysis tool but you still need a tool to help you to narrow the problem down or atleast priorities where to start looking.
That is exactly the kind of use case it is designed for. Please report how it works..
I would say my 2nd use case is less practical for most people, but it is a use case that I have misused the cppcheck misra addon for in the past. My code bases don't have to be Misra complaint, but occasionally I get very unrealistic deadlines for code reviews. I also find that no code base is accidently misra compliant. So I use ( or misuse ) the misra addon output to help keep my brain engaged about the code instead of for misra compliance. While being told that a variable has a naming uniqueness violation is ok. It tells my brain to pay extra attention to that variable so I don't confuse it for another. I'm hoping the output from the bug hunting analysis leads to better questions for my brain to think about the code under review. For example having the bug hunting analysis tell me that it can't prove that something isn't a buffer overflow or that a variable hasn't been ininitialized seems like really useful things for my brain to think about during the review.
ok. interesting "misuse". :-)
I would personally like to see a github addon that show such hints during reviews.. but they do not block the merge or make it "red".
I like this idea. This is a logical continuation of our work on the scripts for CVE checking.
Moreover, it should be possible to implement it now, even without support diffs in bug-hunting mode. We can just sort unrelated output in the script. This will work on GitHub infrastructure, so we don't need to care about speed and resource efficiency at all.
+5! Our codebase strives to be MISRA compliant although not a 100% requirement (yet) Sometimes it can be tedious to deal with the MISRA restrictions in an embedded codebase that has a lot of interfacing with hardware but overall I think it results in a much more robust codebase that is easer to read and follow.
This would be very useful. We currently run cppcheck + MISRA in our CI and any violation blocks the PR. Having a "this look suspicious" suggestion that would not block CI but alert reviewers to pay close attention would be fantastic.
You can always look into the Trunk Gating with Gerrit, Jenkins & Zuul Setup https://www.youtube.com/watch?v=5wwKjdvPxa8
It's where Jenkins is setup as a code reviewer in Gerrit. You can then have Jenkins take all the warnings from the code and mark them in the reviews. While I have never used this setup my self I always thought it was very interesting.
Nod. I wish we could use Gerrit we used it when my team was smaller and I think its revew tools were superior to GitHub but unfortunately we had to switch to only GitHub reviews due to the complexity of setting up Gerrit server that would be publicly accessible (for the 3rd parties we work with that we don't want on the VPN) yet with sufficient authentication to keep private stuff private. Been a few years now since I looked at it but switching away from GitHub would be a lot of effort at this point.
Thank you for your interest it is encouraging. :-)
I have the feeling that the analysis got slower since cppcheck-2.3. Imho, speed will not be critical for some use cases. it would probably be ok if it takes weeks to finish the analysis if that means all the bugs are found. but then there are use cases where it has to finish more quickly.. so we might need to have some configuration..