Thank you!
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
git version 1.8.3.1
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
GNU Wget 1.14 built on linux-gnu.
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://www.gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Originally written by Hrvoje Niksic hniksic@xemacs.org.
Please send bug reports and questions to bug-wget@gnu.org.
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/.
Get Cppcheck..
Cloning into '/home/user/cppcheck-donate-cpu-workfolder/cppcheck'...
remote: Enumerating objects: 124737, done.
remote: Total 124737 (delta 0), reused 0 (delta 0), pack-reused 124737
Receiving objects: 100% (124737/124737), 89.65 MiB | 372.00 KiB/s, done.
Resolving deltas: 100% (98754/98754), done.
Connecting to server to get Cppcheck versions..
Failed to get cppcheck versions: [Errno 110] Connection timed out
Failed to communicate with server, retry later [user@donationstation027 tools]$ python ./donate-cpu.py
IMPORTANT
Please consider switching to Python 3.
We plan to completely drop Python 2 support
for the Donate CPU client in the near future.
Thank you!
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
git version 1.8.3.1
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
GNU Wget 1.14 built on linux-gnu.
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://www.gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Originally written by Hrvoje Niksic hniksic@xemacs.org.
Please send bug reports and questions to bug-wget@gnu.org.
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/.
Get Cppcheck..
Already on 'master'
Already up-to-date.
Connecting to server to get Cppcheck versions..
Failed to get cppcheck versions: [Errno 110] Connection timed out
Failed to communicate with server, retry later [user@donationstation027 tools]$
Last edit: Squirrel 2020-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We switched to Python 3 recently (this has several advantages).
If you have not yet installed Python 3 on the machines please do so. On newer Linux distributions it should already be installed. Some distros already stopped to install Python 2 by default.
Then the donation script can be simply called via ./donate-cpu.py.
You can also explicitly run it with Python 3 if you like: python3 ./donate-cpu.py, but that should not be necessary.
And thanks for donating to the Cppcheck project!
Last edit: versat 2020-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Regarding the connection timeouts I am not sure what caused them.
The server is working for me.
The donate script you are using is not the latest, it should be updated. But I assume that this has nothing to do with the connection issues.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I was unable to use port 8000, this has since been resolved. I am also now invoking with Python3. But, a seperate question - is it normal for cppcheck analysis to take 24+ hours of wall clock time? I put 8 servers in service and each one has at least 1500 minutes of processing time already used for a single analysis invocation (over a day of processing time each).
Last edit: Squirrel 2020-02-23
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hmm, no that is not normal.
We added a timout of one hour for a single analysis, but we detected that it could not work if several threads are used for the analysis (via the -j option).
There is a Pull Request for fixing it (https://github.com/danmar/cppcheck/pull/2510). We are not sure if it works on every platform, but maybe we should try it.
How many threads are you using? How do you invoke the donate script?
Which platform/operating system are you using? We could try to reproduce it on a similar system then and maybe fix it.
Last edit: versat 2020-02-24
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I didn't capture which pacakges it was scanning, but I can certainly do that. I also am limiting to -j8 instead of anywhere near the full capacity of the server since, when it gets into its infinite loop condition, those cores are effectively lost.
Last edit: Squirrel 2020-02-25
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The output fatal: unrecognized argument: --no-patch likely is caused by git. It seems to be an older version of git that does not support this argument. I am not sure if this is a problem, maybe it is fine.
What is strange about the first case is that gdb is started, although Cppcheck finished with -999, which means that it timed out. But it seems that it received signal 11 and that results in the donate script to assume that it crashed.
We will have to look into this.
The second case looks fine from what can be seen. But if it hangs longer than one hour in the analysis with Cppcheck 1.90 that is also an issue that should not happen.
I will try to reproduce those issues inside a virtual machine with CentOS 7.
What I have observed some times is that Cppcheck used so much memory that it slowed down the system because more and more swap was used. This can make the machine unresponsable. Thus on machines where only the analysis is run I disable the swap via sudo swapoff -a. If Cppcheck wants to use more memory than is available it simply crashes and the donate script continues accordingly. Maybe this is an option you can try.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have merged the PR 2510.
Now the donate cpu scripts should more reliably kill timed out Cppcheck analysis even when multithreading is used.
I have also set up a CentOS 7.7 system in a virtual machine and for the packages that were analysed so far it looks fine. I have disabled swap as explained before.
If you encounter a package that does not time out as expected after one hour then please post the URL of that package. Some issues in Cppcheck do only happen for some specific code.
I can see quite a lot of results that were produced via some CentOS 7.5 system. So if these are returned by your systems it looks like it is working now for most of the packages.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have also suspected swapping to be playing into things, but I haven't seen a cppcheck processes using anywhere near the memory limits of these systems. I'll keep playing around with things to help better diagnose.
Also, this package caused a cppcheck timeout/crash overnight:
ftp://ftp.de.debian.org/debian/pool/main/libr/libreoffice/libreoffice_6.4.1~rc1.orig.tar.xz
Edit: Also, is it normal to not be able to upload results on fairly regular occasion? It seems the more I bring online, the more often this happens:
The server has a limit of 1Mb for info messages (2Mb for results), so for large packages, these are rejected. I thought that message was confusing too when I first saw it, so perhaps we could improve it somehow?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I get the following when trying to donate processing power (via donate-cpu.py) from my servers:
[user@donationstation027 tools]$ python ./donate-cpu.py
IMPORTANT
Please consider switching to Python 3.
We plan to completely drop Python 2 support
for the Donate CPU client in the near future.
For further information and reporting complaints, ideas, ... see:
https://sourceforge.net/p/cppcheck/discussion/development/thread/86813a8a53/
Thank you!
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
git version 1.8.3.1
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
GNU Wget 1.14 built on linux-gnu.
+digest +https +ipv6 +iri +large-file +nls +ntlm +opie +ssl/openssl
Wgetrc:
/etc/wgetrc (system)
Locale: /usr/share/locale
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
-DLOCALEDIR="/usr/share/locale" -I. -I../lib -I../lib -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
Link: gcc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic -lssl -lcrypto
/usr/lib64/libssl.so /usr/lib64/libcrypto.so /usr/lib64/libz.so
-ldl -lz -lz -lidn -luuid -lpcre ftp-opie.o openssl.o http-ntlm.o
../lib/libgnu.a
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://www.gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Originally written by Hrvoje Niksic hniksic@xemacs.org.
Please send bug reports and questions to bug-wget@gnu.org.
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Get Cppcheck..
Cloning into '/home/user/cppcheck-donate-cpu-workfolder/cppcheck'...
remote: Enumerating objects: 124737, done.
remote: Total 124737 (delta 0), reused 0 (delta 0), pack-reused 124737
Receiving objects: 100% (124737/124737), 89.65 MiB | 372.00 KiB/s, done.
Resolving deltas: 100% (98754/98754), done.
Connecting to server to get Cppcheck versions..
Failed to get cppcheck versions: [Errno 110] Connection timed out
Failed to communicate with server, retry later
[user@donationstation027 tools]$ python ./donate-cpu.py
IMPORTANT
Please consider switching to Python 3.
We plan to completely drop Python 2 support
for the Donate CPU client in the near future.
For further information and reporting complaints, ideas, ... see:
https://sourceforge.net/p/cppcheck/discussion/development/thread/86813a8a53/
Thank you!
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
git version 1.8.3.1
GNU Make 3.82
Built for x86_64-redhat-linux-gnu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
GNU Wget 1.14 built on linux-gnu.
+digest +https +ipv6 +iri +large-file +nls +ntlm +opie +ssl/openssl
Wgetrc:
/etc/wgetrc (system)
Locale: /usr/share/locale
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
-DLOCALEDIR="/usr/share/locale" -I. -I../lib -I../lib -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic
Link: gcc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic -lssl -lcrypto
/usr/lib64/libssl.so /usr/lib64/libcrypto.so /usr/lib64/libz.so
-ldl -lz -lz -lidn -luuid -lpcre ftp-opie.o openssl.o http-ntlm.o
../lib/libgnu.a
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://www.gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Originally written by Hrvoje Niksic hniksic@xemacs.org.
Please send bug reports and questions to bug-wget@gnu.org.
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-110.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Get Cppcheck..
Already on 'master'
Already up-to-date.
Connecting to server to get Cppcheck versions..
Failed to get cppcheck versions: [Errno 110] Connection timed out
Failed to communicate with server, retry later
[user@donationstation027 tools]$
Last edit: Squirrel 2020-02-21
We switched to Python 3 recently (this has several advantages).
If you have not yet installed Python 3 on the machines please do so. On newer Linux distributions it should already be installed. Some distros already stopped to install Python 2 by default.
Then the donation script can be simply called via
./donate-cpu.py.You can also explicitly run it with Python 3 if you like:
python3 ./donate-cpu.py, but that should not be necessary.And thanks for donating to the Cppcheck project!
Last edit: versat 2020-02-21
Regarding the connection timeouts I am not sure what caused them.
The server is working for me.
The donate script you are using is not the latest, it should be updated. But I assume that this has nothing to do with the connection issues.
yeah it works for me too.. not sure why you have connection issues. Maybe your network does not allow outgoing connections on port 8000?
Apologies, I never actually posted the issue :)
I was unable to use port 8000, this has since been resolved. I am also now invoking with Python3. But, a seperate question - is it normal for cppcheck analysis to take 24+ hours of wall clock time? I put 8 servers in service and each one has at least 1500 minutes of processing time already used for a single analysis invocation (over a day of processing time each).
Last edit: Squirrel 2020-02-23
Hmm, no that is not normal.
We added a timout of one hour for a single analysis, but we detected that it could not work if several threads are used for the analysis (via the
-joption).There is a Pull Request for fixing it (https://github.com/danmar/cppcheck/pull/2510). We are not sure if it works on every platform, but maybe we should try it.
How many threads are you using? How do you invoke the donate script?
Which platform/operating system are you using? We could try to reproduce it on a similar system then and maybe fix it.
Last edit: versat 2020-02-24
CentOS 7.5 (they only support x86_64 as of 7.x), invoking with python3. There are three distinct types of failures I see after re-cloning again.
In this case, I get a crash followed by a seeminly infinite invocation:
In this case, I get a timeout followed by another seeminly infinite invocation:
I didn't capture which pacakges it was scanning, but I can certainly do that. I also am limiting to -j8 instead of anywhere near the full capacity of the server since, when it gets into its infinite loop condition, those cores are effectively lost.
Last edit: Squirrel 2020-02-25
The output
fatal: unrecognized argument: --no-patchlikely is caused by git. It seems to be an older version of git that does not support this argument. I am not sure if this is a problem, maybe it is fine.What is strange about the first case is that
gdbis started, although Cppcheck finished with-999, which means that it timed out. But it seems that it received signal 11 and that results in the donate script to assume that it crashed.We will have to look into this.
The second case looks fine from what can be seen. But if it hangs longer than one hour in the analysis with Cppcheck 1.90 that is also an issue that should not happen.
I will try to reproduce those issues inside a virtual machine with CentOS 7.
What I have observed some times is that Cppcheck used so much memory that it slowed down the system because more and more swap was used. This can make the machine unresponsable. Thus on machines where only the analysis is run I disable the swap via
sudo swapoff -a. If Cppcheck wants to use more memory than is available it simply crashes and the donate script continues accordingly. Maybe this is an option you can try.I have merged the PR 2510.
Now the donate cpu scripts should more reliably kill timed out Cppcheck analysis even when multithreading is used.
I have also set up a CentOS 7.7 system in a virtual machine and for the packages that were analysed so far it looks fine. I have disabled swap as explained before.
If you encounter a package that does not time out as expected after one hour then please post the URL of that package. Some issues in Cppcheck do only happen for some specific code.
I can see quite a lot of results that were produced via some CentOS 7.5 system. So if these are returned by your systems it looks like it is working now for most of the packages.
I have also suspected swapping to be playing into things, but I haven't seen a cppcheck processes using anywhere near the memory limits of these systems. I'll keep playing around with things to help better diagnose.
Also, this package caused a cppcheck timeout/crash overnight:
ftp://ftp.de.debian.org/debian/pool/main/libr/libreoffice/libreoffice_6.4.1~rc1.orig.tar.xz
Edit: Also, is it normal to not be able to upload results on fairly regular occasion? It seems the more I bring online, the more often this happens:
Is it possibly upset that too many results are coming from the same gateway's IP?
Last edit: Squirrel 2020-02-25
The server has a limit of 1Mb for info messages (2Mb for results), so for large packages, these are rejected. I thought that message was confusing too when I first saw it, so perhaps we could improve it somehow?