|
From: Santeri P. <sa...@ss...> - 2003-08-13 07:46:07
|
When running valgrind on a program I get the following errors:
VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH): internal error:
what = 0
ld_preload_str = `(null)'
ld_library_path_str = `(null)'
vg_prel = `(null)'
sk_prel = `(null)'
coredir2 = `(null)'
VG_LIBDIR = `/l/beta/lib'
This is clearly caused by the fact that this program clears the
environmental variables on startup (to known values).
What are the environmental variables that should be preserved (not
cleared) to allow valgrind to work for the program and the child
processes (with --trace-children=yes)? (My guess: LD_PRELOAD and
LD_LIBRARY_PATH.)
The FAQ entry 12 has some information:
Q12. My program dies like this (often at exit):
VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH): internal error:
(loads of text)
A12. We're not entirely sure about this, and would appreciate
someone sending a simple test case for us to look at.
One possible cause is that your program modifies its
environment variables, possibly including zeroing them
all. Avoid this if you can.
1.9.6 contains a fix which hopefully reduces the chances
of your program bombing out like this.
Perhaps a list of not-to-be-cleared environmental variables should be
added to the FAQ?
--
sa...@ss... I have no opinions, since I cannot express any, after all.
If you think you saw an opinion, contact your optometrist.
|
|
From: Nicholas N. <nj...@ca...> - 2003-08-13 08:48:50
|
On Wed, 13 Aug 2003, Santeri Paavolainen wrote:
> VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH): internal error:
>
> This is clearly caused by the fact that this program clears the
> environmental variables on startup (to known values).
>
> What are the environmental variables that should be preserved (not
> cleared) to allow valgrind to work for the program and the child
> processes (with --trace-children=yes)? (My guess: LD_PRELOAD and
> LD_LIBRARY_PATH.)
Yes: LD_PRELOAD and LD_LIBRARY_PATH, and only with --trace-children=yes.
I updated the FAQ, it now reads:
A12. One possible cause is that your program modifies its
environment variables, possibly including zeroing them
all. Valgrind relies on the LD_PRELOAD and LD_LIBRARY_PATH
variables, so zeroing them will break things.
As of 1.9.6, Valgrind only uses these variables with
--trace-children=yes, which should reduce the potential for
problems.
N
|
|
From: Santeri P. <sa...@ss...> - 2003-08-13 09:46:46
|
Nicholas Nethercote wrote:
>Yes: LD_PRELOAD and LD_LIBRARY_PATH, and only with --trace-children=yes.
>
>
Apparently this is not completely true.. I set up the program to pass
LD_PRELOAD and LD_LIBRARY_PATH from the parent process, and this is
visible in debug output too (from the child):
debug[6918]:
LD_PRELOAD=/l/beta/lib/valgrind/vgskin_memcheck.so:/l/beta/lib/valgrind/valgrind.so:
debug[6918]: LD_LIBRARY_PATH=/l/beta/lib/valgrind:
But:
valgrind.so: Startup or configuration error:
Can't read options from env var VG_ARGS.
valgrind.so: Unable to start up properly. Giving up.
So VG_ARGS needs to be added also. I added that to the list of passed
env variables, and so:
debug[7076]: VG_ARGS=
--suppressions=/l/beta/lib/valgrind/default.supp --trace-children=yes
--num-callers=32 --logfile=/l/xxtest/xxtest.log
And now everything works fine.
--
sa...@ss... I have no opinions, since I cannot express any, after all.
If you think you saw an opinion, contact your optometrist.
|
|
From: Nicholas N. <nj...@ca...> - 2003-08-13 10:21:14
|
On Wed, 13 Aug 2003, Santeri Paavolainen wrote: > So VG_ARGS needs to be added also. Ah, yes. I've updated the FAQ in CVS. Thanks for the help. N |
|
From: Olly B. <ol...@su...> - 2003-08-13 16:51:33
|
> A12. One possible cause is that your program modifies its
> environment variables, possibly including zeroing them
> all. Valgrind relies on the LD_PRELOAD and LD_LIBRARY_PATH
> variables, so zeroing them will break things.
I'm not certain I understand the mechanisms involved completely, but
would it work for valgrind to wrapper getenv, setenv, and putenv and
keep track of what the app thinks LD_PRELOAD and LD_LIBRARY_PATH (and
indeed VG_ARGS) are set to, but when making syscalls, actually use the
values set with the paths valgrind needs prepended?
Or simpler, just wrapper setenv and putenv and force valgrind's paths
in and prevent VG_ARGS from being changed, but I worry slightly that an
app might assume it knows how long a variable is because it just set it
- something like this rather stupid contrived example:
int foo(const char * value) {
char * p = malloc(strlen(value) + 10);
if (!p) return -1;
setenv("LD_PRELOAD", value, 1);
bar();
strcpy(p, getenv("LD_PRELOAD));
strcat(p, ":/usr/lib");
setenv("LD_PRELOAD", p, 1);
baz();
free(p);
return 0;
}
Cheers,
Olly
|
|
From: Santeri P. <sa...@ss...> - 2003-08-14 07:41:33
|
Olly Betts wrote:
>Or simpler, just wrapper setenv and putenv and force valgrind's paths
>in and prevent VG_ARGS from being changed, but I worry slightly that an
>app might assume it knows how long a variable is because it just set it
>- something like this rather stupid contrived example:
>
>
To this work, you need to also monitor accesses to the `environ' global
variable. Perhaps that is actually easier, since `putenv', `clearenv'
etc. manipulate just that single global variable.
--
sa...@ss... I have no opinions, since I cannot express any, after all.
If you think you saw an opinion, contact your optometrist.
|
|
From: Nicholas N. <nj...@ca...> - 2003-08-14 07:50:19
|
On Thu, 14 Aug 2003, Santeri Paavolainen wrote: > >Or simpler, just wrapper setenv and putenv and force valgrind's paths > >in and prevent VG_ARGS from being changed, but I worry slightly that an > >app might assume it knows how long a variable is because it just set it > >- something like this rather stupid contrived example: > > > To this work, you need to also monitor accesses to the `environ' global > variable. Perhaps that is actually easier, since `putenv', `clearenv' > etc. manipulate just that single global variable. This all sounds very complicated, for a case that comes up very rarely in practice. What do you think of just improving the error message produced by VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH) so it says something like "you changed $VG_ARGS, so Valgrind can't continue", and similar for LD_PRELOAD and LD_LIBRARY_PATH? N |
|
From: Santeri P. <sa...@ss...> - 2003-08-14 08:10:17
|
Nicholas Nethercote wrote:
>This all sounds very complicated, for a case that comes up very rarely in
>practice. What do you think of just improving the error message produced
>by VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH) so it says something like "you
>changed $VG_ARGS, so Valgrind can't continue", and similar for LD_PRELOAD
>and LD_LIBRARY_PATH?
>
>
I agree, this issue is anyway already covered in the FAQ now and I just
stumbled on it because the program I was testing sanitized the
environmental variables for its children.
(I was just pointing out that putenv/clearenv/setenv are not the access
point to environmental variables, but the `environ' global variable is,
so if someone feels like patching valgrind, they should watch `environ',
not the individual functions that modify it..)
--
sa...@ss... I have no opinions, since I cannot express any, after all.
If you think you saw an opinion, contact your optometrist.
|
|
From: Crispin F. <val...@fl...> - 2003-08-14 15:25:05
|
On Thu, 2003-08-14 at 08:35, Santeri Paavolainen wrote: > Nicholas Nethercote wrote: > > >This all sounds very complicated, for a case that comes up very rarely in > >practice. What do you think of just improving the error message produced > >by VG_(mash_LD_PRELOAD_and_LD_LIBRARY_PATH) so it says something like "you > >changed $VG_ARGS, so Valgrind can't continue", and similar for LD_PRELOAD > >and LD_LIBRARY_PATH? > > > > > I agree, this issue is anyway already covered in the FAQ now and I just > stumbled on it because the program I was testing sanitized the > environmental variables for its children. > > (I was just pointing out that putenv/clearenv/setenv are not the access > point to environmental variables, but the `environ' global variable is, > so if someone feels like patching valgrind, they should watch `environ', > not the individual functions that modify it..) Monitoring the environ variable is not perfect either as you can quite happily call execve directly and pass a brand new environment in. This is presumably all the valgrind sees at the moment (AIUI all the other exec functions are libc wrappers). Wouldn't it be possible for valgrind to realise that the required environment variables aren't present and add them in again, either by mallocing some space for the new env array and the env entries, or by using a few static buffers ? Crispin |
|
From: Santeri P. <sa...@ss...> - 2003-08-15 07:25:37
|
Crispin Flowerday wrote:
>Monitoring the environ variable is not perfect either as you can quite
>happily call execve directly and pass a brand new environment in. This
>is presumably all the valgrind sees at the moment (AIUI all the other
>exec functions are libc wrappers).
>
>Wouldn't it be possible for valgrind to realise that the required
>environment variables aren't present and add them in again, either by
>mallocing some space for the new env array and the env entries, or by
>using a few static buffers ?
>
>
Good point. I noticed that Valgrind does the following steps wrt/ exec:
1. valgrind.so is hooked using LD_PRELOAD
2. Command line arguments are passed to coregrind through the VG_ARGS
environmental variable.
3. When an execve is trapped (vg_syscalls.c), clo_trace_children is
checked: if child tracing is enabled, valgrind does nothing. If child
tracing is disabled, mash_LD_PRELOAD_and_LD_LIBRARY_PATH is called to
modify LD_PRELOAD and LD_LIBRARY_PATH to remove valgrind.
To allow tracing environmetal variables, I think valgrind should instead
always remove itself from LD_PRELOAD, LD_LIBRARY_PATH and clear VG_ARGS
from the environment. When it traps the execve system call, it would
look into clo_trace_children, and if child tracing is enabled,
LD_PRELOAD, LD_LIBRARY_PATH and VG_ARGS would be modified to include the
original values (since valgrind knows what the values are, this step
would just copy the known correct values in).
--
sa...@ss... I have no opinions, since I cannot express any, after all.
If you think you saw an opinion, contact your optometrist.
|
|
From: Nicholas N. <nj...@ca...> - 2003-09-05 22:17:00
|
On Fri, 15 Aug 2003, Santeri Paavolainen wrote: > >Wouldn't it be possible for valgrind to realise that the required > >environment variables aren't present and add them in again, either by > >mallocing some space for the new env array and the env entries, or by > >using a few static buffers ? > > 1. valgrind.so is hooked using LD_PRELOAD > 2. Command line arguments are passed to coregrind through the VG_ARGS > environmental variable. > 3. When an execve is trapped (vg_syscalls.c), clo_trace_children is > checked: if child tracing is enabled, valgrind does nothing. If child > tracing is disabled, mash_LD_PRELOAD_and_LD_LIBRARY_PATH is called to > modify LD_PRELOAD and LD_LIBRARY_PATH to remove valgrind. > > To allow tracing environmetal variables, I think valgrind should instead > always remove itself from LD_PRELOAD, LD_LIBRARY_PATH and clear VG_ARGS > from the environment. When it traps the execve system call, it would > look into clo_trace_children, and if child tracing is enabled, > LD_PRELOAD, LD_LIBRARY_PATH and VG_ARGS would be modified to include the > original values (since valgrind knows what the values are, this step > would just copy the known correct values in). I've been thinking about this. It is difficult. Currently, when an execve happens, if Valgrind doesn't want to trace into the child then it removes the relevant substrings from the LD_PRELOAD and LD_LIBRARY_PATH variables that are present in the 'envp' arg to execve(). (It falls over if they are missing from 'envp'.) With your proposal, Valgrind would remove the substrings at startup (and also clear VG_ARGS). When an execve happens, if Valgrind does want to trace into the children, it would add the substrings back to LD_PRELOAD and LD_LIBRARY_PATH, and restore VG_ARGS. Here's the difficulty: 'envp' is a NULL-terminated char* array. If any of the three variables weren't present, Valgrind would have to make a new copy of 'envp', copying all the old information and adding the missing variable(s). (Unless we knew there was enough room to add the new entries require, but I can't see how we could possibly know that.) The first, smaller problem: this is a space leak -- if execve happened lots of times, this would add up. The second, bigger problem: this is a space leak -- if 'envp' was malloc'd by the client program, Valgrind will (if running Memcheck or Addrcheck as its skin) identify this as a lost heap block. Which is bad, because it's not the client's fault. Valgrind can't try to free it, because 'envp' might not be malloc'd. I can't see how using static buffers would get around these problems, either. This is a shame, because Valgrind's current handling of all this is definitely broken. Can anyone see a way out? N |