|
From: PLUG G. <plu...@gm...> - 2012-04-14 14:42:04
|
Hi, I am new to valgrind. I am trying to debug my web application (testwtdbg.wt) using valgrind. But there is a mismatch between the FIFO name valgrind generates and the FIFO vgdb tries to use. I am using a FastCGI (Apache's mod_fcgid) application developed using Wt (www.webtoolkit.eu). Valgrind is not directly invoked by me. Instead I specify the valgrind command and its options in a Wt configuration file (wt_config.xml) and Wt then prepends this command to the application before launching it using execve system call. Here are the commands issued to use valgrind and the corresponding output: *A] A.1. Command to start valgrind and web-application "testwtdbg.wt" (this command is issued, using system call execve, by Wt framework in the context of Apache webserver): /usr/local/bin/valgrind -v --trace-children=yes --vgdb=full --vgdb-error=0 /var/www/html/plug/docroot/testwtdbg.wt <<additional parameters of the application/Wt>> A.2. Log messages (Apache log messages): [root@playground build]# cat /var/log/httpd/error_log [Sat Apr 14 13:15:51 2012] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0 [Sat Apr 14 13:15:51 2012] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Sat Apr 14 13:15:51 2012] [notice] Digest: generating secret for digest authentication ... [Sat Apr 14 13:15:51 2012] [notice] Digest: done [Sat Apr 14 13:15:51 2012] [notice] Apache/2.2.22 (Unix) DAV/2 mod_fcgid/2.3.6 configured -- resuming normal operations [2012-Apr-14 13:16:35.383695] 6022 - [info] "WServer/wtfcgi: initializing relay server" [2012-Apr-14 13:16:35.387222] 6022 - [info] "config: reading Wt config file: /etc/wt/wt_config.xml (location = '/var/www/html/plug/docroot/testwtdbg.wt')" [2012-Apr-14 13:16:35.389108] 6022 - [info] "wtfcgi: reading FastCGI stream from stdin" [2012-Apr-14 13:16:35.398416] 6022 - [info] "wtfcgi: spawned dedicated process for ZHUb9FY39csyCLdY: pid=6164" ==6164== Memcheck, a memory error detector ==6164== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==6164== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==6164== Command: /var/www/html/plug/docroot/testwtdbg.wt client ZHUb9FY39csyCLdY ==6164== --6164-- Valgrind options: --6164-- -v --6164-- --trace-children=yes --6164-- --vgdb=full --6164-- --vgdb-error=0 --6164-- Contents of /proc/version: --6164-- Linux version 3.3.1-3.fc16.x86_64 (moc...@x8...) (gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) ) #1 SMP Wed Apr 4 18:08:51 UTC 2012 --6164-- Arch and hwcaps: AMD64, amd64-sse3 --6164-- Page sizes: currently 4096, max supported 4096 --6164-- Valgrind library directory: /usr/local/lib/valgrind --6164-- Reading syms from /var/www/html/plug/docroot/testwtdbg.wt (0x400000) --6164-- Reading syms from /usr/local/lib/valgrind/memcheck-amd64-linux (0x38000000) --6164-- object doesn't have a dynamic symbol table --6164-- Reading syms from /lib64/ld-2.14.90.so (0x3d26200000) --6164-- Reading suppressions file: /usr/local/lib/valgrind/default.supp ==6164== (action at startup) vgdb me ... ==6164== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-6164-by-???-on-??? ==6164== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-6164-by-???-on-??? ==6164== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-???-on-??? ==6164== ==6164== TO CONTROL THIS PROCESS USING vgdb (which you probably ==6164== don't want to do, unless you know exactly what you're doing, ==6164== or are doing some strange experiment): ==6164== /usr/local/lib/valgrind/../../bin/vgdb --pid=6164 ...command... ==6164== ==6164== TO DEBUG THIS PROCESS USING GDB: start GDB like this ==6164== /path/to/gdb /var/www/html/plug/docroot/testwtdbg.wt ==6164== and then give GDB the following command ==6164== target remote | /usr/local/lib/valgrind/../../bin/vgdb --pid=6164 ==6164== --pid is optional if only one valgrind process is running ==6164== *B] B.1. Check the FIFOs created by valgrind gdbserver: [root@playground build]# ls -la /tmp/ drwxrwxrwt. 37 root root 4096 Apr 14 14:13 . dr-xr-xr-x. 22 root root 4096 Apr 13 00:18 .. drwxrwxrwt. 2 root root 4096 Apr 13 00:30 .font-unix drwxrwxrwt. 2 root root 4096 Apr 14 11:56 .ICE-unix -rw-------. 1 root root 0 Apr 13 00:41 tmpDjwfid prw-r--r--. 1 apache apache 0 Apr 14 13:18 vgdb-pipe-from-vgdb-to-6164-by-???-on-??? -rw-r--r--. 1 apache apache 48 Apr 14 13:18 vgdb-pipe-shared-mem-vgdb-6164-by-???-on-??? prw-r--r--. 1 apache apache 0 Apr 14 13:18 vgdb-pipe-to-vgdb-from-6164-by-???-on-??? -r--r--r--. 1 root root 11 Apr 14 11:56 .X0-lock drwxrwxrwt. 2 root root 4096 Apr 14 11:56 .X11-unix drwxrwxrwt. 2 root root 4096 Apr 13 00:30 .XIM-unix *C] C.1 Start GDB to connect to the valgrind gdbserver: [root@playground build]# gdb /var/www/html/plug/docroot/testwtdbg.wt GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /var/www/html/plug/docroot/testwtdbg.wt...done. (gdb) target remote | vgdb --pid=6164 Remote debugging using | vgdb --pid=6164 syscall failed: No such file or directory error opening /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-plug-on-playground.fc16 shared memory file Remote communication error. Target disconnected.: Connection reset by peer. (gdb) *D] D.1 Check the FIFO information using vgdb: [root@playground build]# vgdb -d -d -d help --pid=6164 1334405920.781999 searching pid in directory /tmp/ format /tmp/vgdb-pipe-from-vgdb-to- 1334405920.782175 check_trial 0 1334405920.782662 trying /tmp/.X0-lock 1334405920.783015 trying /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-???-on-??? 1334405920.786167 trying /tmp/.X11-unix 1334405920.786233 trying /tmp/tmpDjwfid 1334405920.786660 trying /tmp/.font-unix 1334405920.786901 trying /tmp/.ICE-unix 1334405920.787175 trying /tmp/vgdb-pipe-from-vgdb-to-6164-by-???-on-??? 1334405920.787228 trying /tmp/vgdb-pipe-from-vgdb-to-6164-by-???-on-??? 1334405920.789632 trying /tmp/.XIM-unix 1334405920.792089 trying /tmp/vgdb-pipe-to-vgdb-from-6164-by-???-on-??? 1334405920.792142 trying /tmp/vgdb-pipe-to-vgdb-from-6164-by-???-on-??? 1334405920.792265 vgdb: using /tmp/vgdb-pipe-from-vgdb-to-6164-by-plug-on-playground.fc16 /tmp/vgdb-pipe-to-vgdb-from-6164-by-plug-on-playground.fc16 /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-plug-on-playground.fc16 syscall failed: No such file or directory error opening /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-plug-on-playground.fc16 shared memory file >From the above output it looks like the valgrind gdbserver generates FIFO names that is different than the ones expected by vgdb. How do I tell vgdb to use the correct FIFO? A postfix option (--vgdb-postfix) similar to --vgdb-prefix would have helped in this case. But as that is not available what alternate method is available for vgdb to use the correct FIFOs. Thanks and regards, ~Plug |
|
From: Philippe W. <phi...@sk...> - 2012-04-14 15:52:16
|
Thanks for the precise information you have put in your question,
it helps to see what is going on.
> ==6164== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-6164-by-???-on-???
> ==6164== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-6164-by-???-on-???
> ==6164== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-???-on-???
The above is the FIFOs and shared mem created by the Valgrind gdbserver.
They are created in the environment in which valgrind is launched.
>From the ls command below, we see that these files belongs to the user apache:
> prw-r--r--. 1 apache apache 0 Apr 14 13:18 vgdb-pipe-from-vgdb-to-6164-by-???-on-???
> -rw-r--r--. 1 apache apache 48 Apr 14 13:18 vgdb-pipe-shared-mem-vgdb-6164-by-???-on-???
> prw-r--r--. 1 apache apache 0 Apr 14 13:18 vgdb-pipe-to-vgdb-from-6164-by-???-on-???
>From the below, it looks like gdb and vgdb are launched by the user plug.
> (gdb) target remote | vgdb --pid=6164
> Remote debugging using | vgdb --pid=6164
> syscall failed: No such file or directory
> error opening /tmp/vgdb-pipe-shared-mem-vgdb-6164-by-plug-on-playground.fc16
> shared memory file
> Remote communication error. Target disconnected.: Connection reset by peer.
> (gdb)
The FIFOs and shared mem used by vgdb are based on the user name and the host name.
The logic to find the user name is:
First try the env variable LOGNAME
if not set, try the env variable USER
if still not set, then take ???.
For the host name, the variables examined are:
HOST
then HOSTNAME
then ???
You should start your gdb and vgdb in the same "context/environment" as the gdbserver
you are trying to connect to. Then both the Valgrind gdbserver and vgdb should agree
on user and host, and use the same vgdb-pipe files.
If you cannot do that for one reason or another, you might try the following
(without guarantee :) :
export LOGNAME="???"
export HOST="???"
and then launch gdb+vgdb.
If you launch your gdb/vgdb under another account, the launched vgdb will not have
rw access to the vgdb-pipe* files. To change the permissions, you will have
to one way or another let the apache user either chmod a+rw these files
or else use apache user to launch gdb and vgdb
(or maybe use root ?).
Hope this helps ...
Philippe
|
|
From: Plug G. <plu...@gm...> - 2012-04-14 23:26:48
|
On Sat, Apr 14, 2012 at 4:52 PM, Philippe Waroquiers <phi...@sk...> wrote: <snip> > The FIFOs and shared mem used by vgdb are based on the user name and the > host name. > The logic to find the user name is: > First try the env variable LOGNAME > if not set, try the env variable USER > if still not set, then take ???. > For the host name, the variables examined are: > HOST > then HOSTNAME > then ??? > You should start your gdb and vgdb in the same "context/environment" as the > gdbserver > you are trying to connect to. Then both the Valgrind gdbserver and vgdb > should agree > on user and host, and use the same vgdb-pipe files. > > If you cannot do that for one reason or another, you might try the following > (without guarantee :) : > export LOGNAME="???" > export HOST="???" > and then launch gdb+vgdb. > If you launch your gdb/vgdb under another account, the launched vgdb will > not have > rw access to the vgdb-pipe* files. To change the permissions, you will have > to one way or another let the apache user either chmod a+rw these files > or else use apache user to launch gdb and vgdb > (or maybe use root ?). > I tried starting apache webserver and gdb from root login. Did not help, because the valgrind is still launched from under apache user. So I set the LOGNAME and HOST to "???" as suggested above and started the webserver and gdb from root login. I also changed the permissions of the FIFOs using a+rw. Here is the output of the commands: *A] A.1 Valgrind (Apache log message) - Note that the error messages shown in the log start to appear after I connect to valgrind using gdb+vgdb: [plug@playground build]$ sudo tail -f /var/log/httpd/error_log [Sat Apr 14 22:31:20 2012] [notice] SELinux policy enabled; httpd running as context unconfined_u:system_r:httpd_t:s0 [Sat Apr 14 22:31:20 2012] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec) [Sat Apr 14 22:31:20 2012] [notice] Digest: generating secret for digest authentication ... [Sat Apr 14 22:31:20 2012] [notice] Digest: done [Sat Apr 14 22:31:20 2012] [notice] Apache/2.2.22 (Unix) DAV/2 mod_fcgid/2.3.6 configured -- resuming normal operations [2012-Apr-14 22:36:07.303928] 13140 - [info] "WServer/wtfcgi: initializing relay server" [2012-Apr-14 22:36:07.306978] 13140 - [info] "config: reading Wt config file: /etc/wt/wt_config.xml (location = '/var/www/html/plug/docroot/testwtdbg.wt')" [2012-Apr-14 22:36:07.308753] 13140 - [info] "wtfcgi: reading FastCGI stream from stdin" [2012-Apr-14 22:36:07.319148] 13140 - [info] "wtfcgi: spawned dedicated process for cMbPKxS2lzXyceXq: pid=13151" ==13151== Memcheck, a memory error detector ==13151== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==13151== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==13151== Command: /var/www/html/plug/docroot/testwtdbg.wt client cMbPKxS2lzXyceXq ==13151== --13151-- Valgrind options: --13151-- -v --13151-- --trace-children=yes --13151-- --vgdb=full --13151-- --vgdb-error=0 --13151-- Contents of /proc/version: --13151-- Linux version 3.3.1-3.fc16.x86_64 (moc...@x8...) (gcc version 4.6.3 20120306 (Red Hat 4.6.3-2) (GCC) ) #1 SMP Wed Apr 4 18:08:51 UTC 2012 --13151-- Arch and hwcaps: AMD64, amd64-sse3 --13151-- Page sizes: currently 4096, max supported 4096 --13151-- Valgrind library directory: /usr/local/lib/valgrind --13151-- Reading syms from /var/www/html/plug/docroot/testwtdbg.wt (0x400000) --13151-- Reading syms from /usr/local/lib/valgrind/memcheck-amd64-linux (0x38000000) --13151-- object doesn't have a dynamic symbol table --13151-- Reading syms from /lib64/ld-2.14.90.so (0x3d26200000) --13151-- Reading suppressions file: /usr/local/lib/valgrind/default.supp ==13151== (action at startup) vgdb me ... ==13151== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-13151-by-???-on-??? ==13151== embedded gdbserver: writing to /tmp/vgdb-pipe-to-vgdb-from-13151-by-???-on-??? ==13151== embedded gdbserver: shared mem /tmp/vgdb-pipe-shared-mem-vgdb-13151-by-???-on-??? ==13151== ==13151== TO CONTROL THIS PROCESS USING vgdb (which you probably ==13151== don't want to do, unless you know exactly what you're doing, ==13151== or are doing some strange experiment): ==13151== /usr/local/lib/valgrind/../../bin/vgdb --pid=13151 ...command... ==13151== ==13151== TO DEBUG THIS PROCESS USING GDB: start GDB like this ==13151== /path/to/gdb /var/www/html/plug/docroot/testwtdbg.wt ==13151== and then give GDB the following command ==13151== target remote | /usr/local/lib/valgrind/../../bin/vgdb --pid=13151 ==13151== --pid is optional if only one valgrind process is running ==13151== ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc ==13151== putpkt(write) error: no write_remote_desc *B] B.1 The read-write permissions of the FIFOs. I update the permissions after valgrind starts but before gdb+vgdb is launched. [plug@playground build]$ ls -la /tmp/vgdb* prw-rw-rw-. 1 apache apache 0 Apr 14 22:36 /tmp/vgdb-pipe-from-vgdb-to-13151-by-???-on-??? -rw-rw-rw-. 1 apache apache 48 Apr 14 22:36 /tmp/vgdb-pipe-shared-mem-vgdb-13151-by-???-on-??? prw-rw-rw-. 1 apache apache 0 Apr 14 22:36 /tmp/vgdb-pipe-to-vgdb-from-13151-by-???-on-??? *C] C.1 The output of gdb+vgdb (Note that I also tried setting the "remotetimeout" attribute of gdb to very large value as suggested in README_DEVELOPERS file. But the result is the same except that no error is displayed on gdb console and the control just waits (stalls?) after printing the message "Remote debugging using | vgdb --pid=13151" and does not return to gdb prompt. On the valgrind side though the same error, "putpkt(write) error: no write_remote_desc", is displayed as before except there is just one error rather than same error printed multiple times) [root@playground build]# gdb /var/www/html/plug/docroot/testwtdbg.wt GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /var/www/html/plug/docroot/testwtdbg.wt...done. (gdb) show env LOGNAME LOGNAME = ??? (gdb) show env HOST HOST = ??? (gdb) target remote | vgdb --pid=13151 Remote debugging using | vgdb --pid=13151 relaying data between gdb and process 13151 Ignoring packet error, continuing... warning: unrecognized item "timeout" in "qSupported" response Ignoring packet error, continuing... Ignoring packet error, continuing... Ignoring packet error, continuing... Ignoring packet error, continuing... Ignoring packet error, continuing... Ignoring packet error, continuing... Malformed response to offset query, timeout (gdb) *D] D.1 gdb, valgrind, apache and linux distro versions I am using: [plug@playground build]# gdb --version GNU gdb (GDB) Fedora (7.3.50.20110722-13.fc16) Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. [plug@playground build]# valgrind --version valgrind-3.7.0 [plug@playground build]$ uname -a Linux playground.fc16 3.3.1-3.fc16.x86_64 #1 SMP Wed Apr 4 18:08:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux Apache version is 2.2.22 and mod_fcgid is 2.3.6 *E] E.1 The output of `vgdb -d -d -d help --pid=13151` command is a continuous non-stop flow of following message: written_by_vgdb_before_sleep 1 seen_by_valgrind_before_sleep 1 Does valgrind ignore, for security reasons, the FIFOs with universal write access? I think I read something to this regard but am not able to recall where! Also, I am not able to login as user "apache" and instead have to login as root. But all this has not helped me debug using valgrind a process under apache's context. The constraint that gdb, vgdb and gdbserver all should start under the same context/environment is proving to be quite restrictive. Thanks and regards, ~Plug > Hope this helps ... > > Philippe |
|
From: Philippe W. <phi...@sk...> - 2012-04-15 07:51:04
|
On Sun, 2012-04-15 at 00:26 +0100, Plug Gulp wrote:
> I tried starting apache webserver and gdb from root login. Did not
> help, because the valgrind is still launched from under apache user.
> So I set the LOGNAME and HOST to "???" as suggested above and started
> the webserver and gdb from root login. I also changed the permissions
> of the FIFOs using a+rw. Here is the output of the commands:
Here is another solution to try :
* set the needed env var (LOGNAME, HOST)
* make a copy of the vgdb executable
* change owner of this vgdb file to be the user apache
* set the user and group id bit
(must be root or apache for that, I guess)
chmod u+s vgdb
chmod g+s vgdb
The above worked for me to then connect a root gdb+this vgdb
to a Valgrind running under my user (I replaced apache
by my own user)
3 notes:
1. once you have a "apache" vgdb, I believe root user should not be
needed anymore. You should be able to use your own user, as
the launched vgdb will "belong" to apache.
2. with the above hack, it seems some "ptrace" syscall done
by vgdb are still not authorised. So, you might have
to disable the ptrace things in vgdb using --max-invoke-ms=0
(see description and consequences in Valgrind manual).
3. IMPORTANT: of course, anybody else having access to this "apache"
vgdb can then connect to any Valgrind gdbserver belonging to
apache.
In other words, other users on the same computer have
"full" debuggability of all your apache Valgrind gdbserver
as long as this "apache" vgdb copy exists.
> > Hope this helps ...
Hope this (really) helps (now) ...
Philippe
|