You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
1
(15) |
2
(17) |
3
(23) |
4
(13) |
5
(7) |
6
(8) |
7
(9) |
|
8
(8) |
9
(31) |
10
(31) |
11
(19) |
12
(11) |
13
(38) |
14
(14) |
|
15
(8) |
16
(11) |
17
(7) |
18
(17) |
19
(12) |
20
(12) |
21
(17) |
|
22
(19) |
23
(33) |
24
(42) |
25
(37) |
26
(23) |
27
(27) |
28
(27) |
|
29
(16) |
30
(52) |
31
(33) |
|
|
|
|
|
From: Bob F. <bfr...@si...> - 2004-08-09 23:57:57
|
On Mon, 9 Aug 2004, Tom Hughes wrote: > >> If Linux is to ever become formidable in the multi-threading world >> (which it must!) then it will need to eliminate the requirement for >> the kernel to know about each and every thread. > > Why do you think that? Because if everything has to go through a single point of control, then there is a hot-spot which becomes a bottleneck (bus contention & cache thrashing). To date, Linux has been primarily an OS for systems with one processor, or just a few processors. Think about systems with 100 or more processors. These sort of systems have already been deployed for several years in the proprietary Unix world. Sun's Ultra-IV processor is claimed to (theoretically) work in systems with over 1000 processors sharing a coherent kernel. If each processor doesn't enjoy some degree of autonomy then eventually the law of diminishing returns takes effect and the system runs slower rather than faster. Linux may run on smaller hardware today, but there is no reason it should have to stay there. Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Bryan O'S. <bo...@se...> - 2004-08-09 23:25:31
|
On Mon, 2004-08-09 at 23:59 +0100, Tom Hughes wrote: > My understanding is that this was looked at quite carefully during > the NPTL development. In fact the original plan was to go to a multi > level model but from what I've read experiment showed that the kernel > could be made to scale to vast numbers of thread/process without any > loss of performance. I think you're thinking of NGPT, IBM's parallel effort to replace LinuxThreads, which ended up being abandoned. <b |
|
From: Tom H. <th...@cy...> - 2004-08-09 23:15:26
|
In message <200...@ac...>
Julian Seward <js...@ac...> wrote:
> > Ok, maybe this is the direction we should be heading in, then. So, what's
> > preventing us from doing so? AFAIK:
> >
> > - We can't lose control of scheduling totally, because V turns atomic
> > instructions into non-atomic ones. We don't want a thread switch after
> > a shadow memory word is written, but before the word itself is written.
>
> Agreed. So the suggested solution is to maintain single-threadedness
> but instead of imposing by intercepting and simulating the pthreads
> interface, we intercept and simulate at the clone() level.
I certainly think this is a good aim as it would allow us to drop
all of vg_libpthread.c (a really good idea) and much of vg_scheduler.c
as well I suspect.
In fact we had this working once for wine support because wine
created threads using clone directly. My colleage Adam Gundy had
a solution that involved changing run_thread_for_a_while so that
if the thread to run was not the executing one then the executing
thread would send a signal to the thread to be run and then call
sigsuspend so that it blocked until such as the scheduler chose
to run it and woke it up with a signal.
We dropped that from the wine stuff after wine implemented an
alternate pthread based solution, but I got Adam to dig it out
again last Friday after Nick started this thread and am looking
at whether it can be resurrected.
> As Jeremy says it would be best to run app threads in kernel threads, iow
> allow genuinely concurrent instrumented execution. I can't say that I
> believe the problems that raises, with locking the shadow state, at
> reasonable performance overhead, are soluble -- although I would love to
> be proved wrong.
We have discussed it before, but I suspect it would be very hard.
> Yes. What is needed is to intercept the relevant functions (pth_mutex_lock
> etc), note they have happened, but still allow the native thread library to
> handle it. The downside is we'd then need at least some simulation of the
> pthreads machinery in order to detect errors, but the good thing is we can
> make it as weedy as we like, commensurate with the errors we wish to detect.
> By contrast, currently our pthreads simulation has to be accurate enough to
> actually make the program work correctly, and we can't weaken it in any way.
This is why I'm interested in the wrapper stuff that I think somebody
is working on that would hopefully allow valgrind to do some processing
on entry to/exit from a function in the client program.
I would be inclined to consider dropping all the pthread error checking
from the core and push it into a pthread validation tool.
Obviously helgrind also needs to notice lock/unlock events.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Bob F. <bfr...@si...> - 2004-08-09 23:02:11
|
On Mon, 9 Aug 2004, Julian Seward wrote: >> >> If Linux eventually wakes up and adopts a multi-level threading model >> (combination of user-level and on-demand kernel-level threads "LWP"s) >> as used by Solaris, AIX, and Digital Unix, this approach is surely >> doomed to fail. Also, if someone wants to port Valgrind to a Unix OS >> that doesn't support clone() (only Linux has clone()), it seems doomed >> to fail. > > Are you sure? So long as there is only one kernel thread, a user-level > threading library can do what it wants and V will be OK. Presumably > at some point the library asks the kernel to make a new kernel thread, > and that is the event that V must intercept and handle itself. Sure, > they may not have clone(), but they must have a way to ask the kernel > for a new kernel thread, no? No, I am not sure. Not being sure is part of the problem (at least for me). > Part of the problem is I don't have much of a clue about how a > multi-level threading system works. A short summary wouldn't > go amiss :-) Usually a multi-level threading system uses the equivalent of user-space threads by default, but then is assigned a Light Weight Process (LWP) (associated with a kernel thread) whenever a blocking call takes place. Threads can also be tied to a LWP if they need special treatment such as to be tied to a particular CPU or to be given real-time priority. In the Solaris (2.4-2.8) implementation there is a notion of "concurrency" which defines how many LWPs a process may have active at one time. A multi-threaded program starts off with two LWPs. One LWP executes normal user code while another LWP provides the thread API management. If a blocking call is about to occur and the maximum concurrency has not been hit, a signal is issued which causes the thread library to allocate a new LWP to assign for that thread. Instead of allocating and releasing kernel contexts from the kernel as required, the thread library maintains a pool of available LWPs and assigns them as needed to threads. The key issue with the multi-level system from Valgrind's point of view is that only part of the threads operation is exposed by the kernel interface. The rest is hidden in the thread library. Due to my previous post to valgrind-developers, a list member has informed me that Solaris 9 (current version) has dumped the multi-level model and has gone to a 1:1 model. Presumably this means that more of the user-level thread support has moved into the kernel. Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Tom H. <th...@cy...> - 2004-08-09 22:58:57
|
In message <Pin...@bl...>
Bob Friesenhahn <bfr...@si...> wrote:
> If Linux eventually wakes up and adopts a multi-level threading model
> (combination of user-level and on-demand kernel-level threads "LWP"s)
> as used by Solaris, AIX, and Digital Unix, this approach is surely
> doomed to fail. Also, if someone wants to port Valgrind to a Unix OS
> that doesn't support clone() (only Linux has clone()), it seems doomed
> to fail.
My understanding is that this was looked at quite carefully during
the NPTL development. In fact the original plan was to go to a multi
level model but from what I've read experiment showed that the kernel
could be made to scale to vast numbers of thread/process without any
loss of performance.
Once that was known the extra costs of a multi-level model in terms
of the complication of things like signal routing made it pointless.
> If Linux is to ever become formidable in the multi-threading world
> (which it must!) then it will need to eliminate the requirement for
> the kernel to know about each and every thread.
Why do you think that?
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Julian S. <js...@ac...> - 2004-08-09 22:28:31
|
On Monday 09 August 2004 20:19, Bob Friesenhahn wrote: > On Mon, 9 Aug 2004, Julian Seward wrote: > > At least in theory, if people then want to run different threading > > schemes (foo_threads, for example) we will then be able to at least run a > > foo-threaded program, so long as it is underlyingly based on clone(). > > If Linux eventually wakes up and adopts a multi-level threading model > (combination of user-level and on-demand kernel-level threads "LWP"s) > as used by Solaris, AIX, and Digital Unix, this approach is surely > doomed to fail. Also, if someone wants to port Valgrind to a Unix OS > that doesn't support clone() (only Linux has clone()), it seems doomed > to fail. Are you sure? So long as there is only one kernel thread, a user-level threading library can do what it wants and V will be OK. Presumably at some point the library asks the kernel to make a new kernel thread, and that is the event that V must intercept and handle itself. Sure, they may not have clone(), but they must have a way to ask the kernel for a new kernel thread, no? Part of the problem is I don't have much of a clue about how a multi-level threading system works. A short summary wouldn't go amiss :-) J |
|
From: Julian S. <js...@ac...> - 2004-08-09 22:24:54
|
> I've been thinking about this a bit w.r.t. full virtualisation. The > benefits of FV over the LD_PRELOAD approach are: > > 1. Valgrind gains control at absolute startup > 2. Valgrind can use the standard libraries itself > 3. Valgrind can run static binaries > 4. Valgrind is nicely separated from the client > > All these are good, but they don't to me seem fantastic -- none of them > were really huge problems previously. And FV has introduced its own > difficulties, mostly from the rigidity of the memory layout, which we're > still battling. Plus it results in more code. I agree that 2,3,4 are not a big deal, but 1 is. The threading stuff was getting nasty as a lack of 1 -- having the threading machinery start up in an unknown state is not good, so FV is a big win there. In the longer term 2 and 4 are useful too. For example, if we wanted to use C++ in V in a big way, 2 would be more or less mandatory and 4 would be very helpful too. > I'm not saying we shouldn't have done FV. Because, on the other hand, > one of the best things about Valgrind from a user's point of view is the > "it just works" factor. We really want to keep that as high as possible, > so that's kind of an argument against the worse-is-better approach. Re FV, and perhaps the proxyLWP stuff, Jeremy argues that we should use as much of the underlying OSs services as possible, and make V be the thinnest possible layer in between. I've never been entirely comfortable with that philosophy, although I'm not sure I know why I'm not. I'd still like to know the results of having our own software managed MMU and so decoupling the two address spaces completely -- even though, as Jeremy has accurately pointed out several times in the past, that brings its own set of difficulties. I suppose my underlying inclination is to simulate as much stuff as possible and thereby minimise the extent to which we are exposed to the vagaries of the host OS. I think that's why the thin-layer idea is a bit irksome. Perhaps it's a way of thinking about OS-level portability: either simulate a lot of stuff yourself and minimise the "passed-through" requirements, or have a seperate thin-layer arrangement for each supported OS. I lean towards the former scheme, but I don't know if it's really viable -- whereas at least Jeremy has demonstrated that his scheme works well at least for Linux. J |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 19:59:03
|
On Mon, 9 Aug 2004, Julian Seward wrote:
> * Richard Gabriel ("Patterns of Software") wrote an essay called
> "Worse is better", and I think we're in that kind of situation now.
I've been thinking about this a bit w.r.t. full virtualisation. The
benefits of FV over the LD_PRELOAD approach are:
1. Valgrind gains control at absolute startup
2. Valgrind can use the standard libraries itself
3. Valgrind can run static binaries
4. Valgrind is nicely separated from the client
All these are good, but they don't to me seem fantastic -- none of them
were really huge problems previously. And FV has introduced its own
difficulties, mostly from the rigidity of the memory layout, which we're
still battling. Plus it results in more code.
I'm not saying we shouldn't have done FV. Because, on the other hand,
one of the best things about Valgrind from a user's point of view is the
"it just works" factor. We really want to keep that as high as possible,
so that's kind of an argument against the worse-is-better approach.
So what's my point? I'm not sure.
N
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 19:45:42
|
On Mon, 9 Aug 2004, Adun R. wrote: > This may not be the time as I am still learning the subject, but since > you mentioned it as a design issue, and as this thread seems to > deal with important development steps, I wanted to present a thought of > mine - Porting valgrind to windows. > Several problems rise with this [and many will rise, as this idea is > also, in your words - 'half-baked'] , and a lot of code will have to > change, first of all our loading method (no LD_PRELOAD [except for the > user32.dll attachment dll's in the registry, but this isn't nearly > acceptable), and nothing that is like it - a solution could be to > control the client process using the Windows API and the Debugging API) > which affects the possibility for symbol "capturing" of malloc, calloc, > etc (we can change IAT pointers). Another problem is a whole different > threading model. Note that since 2.1.1, the LD_PRELOAD method is no longer used. My opinion on Windows is that it would be great, if it could be done in a way that doesn't result in Valgrind's code looking like a train crash. AFAIK, all the current developers don't know much about Windows; I sure don't. N |
|
From: Bob F. <bfr...@si...> - 2004-08-09 19:19:56
|
On Mon, 9 Aug 2004, Julian Seward wrote: > At least in theory, if people then want to run different threading schemes > (foo_threads, for example) we will then be able to at least run a foo-threaded > program, so long as it is underlyingly based on clone(). If Linux eventually wakes up and adopts a multi-level threading model (combination of user-level and on-demand kernel-level threads "LWP"s) as used by Solaris, AIX, and Digital Unix, this approach is surely doomed to fail. Also, if someone wants to port Valgrind to a Unix OS that doesn't support clone() (only Linux has clone()), it seems doomed to fail. If Linux is to ever become formidable in the multi-threading world (which it must!) then it will need to eliminate the requirement for the kernel to know about each and every thread. The latest trend is for CPUs to incorporate multiple execution units, so even systems delivered with one CPU chip will benefit performance-wise from multi-threaded programs. Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Adun R. <adu...@ml...> - 2004-08-09 19:04:31
|
On Mon, 9 Aug 2004 18:44:25 +0100, "Julian Seward" <js...@ac...> said: > So, using C++ inheritance or whatever, you could have some core set of > events, with some core state, and each OS could have its own private > state and events pertaining to it. The event queue would hold a > mixture > of the two. Valgrind's core would pull events off the queue, handle > the ones it understands directly, and call the relevant OS-specific > handler for OS-specific events it doesn't understand. This may not be the time as I am still learning the subject, but since you mentioned it as a design issue, and as this thread seems to deal with important development steps, I wanted to present a thought of mine - Porting valgrind to windows. Several problems rise with this [and many will rise, as this idea is also, in your words - 'half-baked'] , and a lot of code will have to change, first of all our loading method (no LD_PRELOAD [except for the user32.dll attachment dll's in the registry, but this isn't nearly acceptable), and nothing that is like it - a solution could be to control the client process using the Windows API and the Debugging API) which affects the possibility for symbol "capturing" of malloc, calloc, etc (we can change IAT pointers). Another problem is a whole different threading model. But first thing first - What you're thoughts on this? more importantly - how this relates to how you all see the valgrind project? Regards, Rauch Adun. -- Adun R. adu...@ml... |
|
From: Julian S. <js...@ac...> - 2004-08-09 18:43:43
|
> > Well, I think a big obvious first step is to drop all the pthreads > > support from the core, which will simplify vg_scheduler a fair bit. The > > scheduler doesn't really need to know much about a thread's state other > > than its running or blocked on some event (which is probably always > > syscall completion of some kind). > > > > This means that to do threading, all OS ports must do enough to support > > the native threads library. The downside is that this could result in > > as much complexity as dropping core ptheads support removes, and it > > would be duplicated for each OS port. But it does mean that we don't > > have to carry libpthread around, which is a large and increasing burden > > (and that's just for all the x86-linux variants). > Ok, maybe this is the direction we should be heading in, then. So, what's > preventing us from doing so? AFAIK: > > - We can't lose control of scheduling totally, because V turns atomic > instructions into non-atomic ones. We don't want a thread switch after > a shadow memory word is written, but before the word itself is written. Agreed. So the suggested solution is to maintain single-threadedness but instead of imposing by intercepting and simulating the pthreads interface, we intercept and simulate at the clone() level. As Jeremy says it would be best to run app threads in kernel threads, iow allow genuinely concurrent instrumented execution. I can't say that I believe the problems that raises, with locking the shadow state, at reasonable performance overhead, are soluble -- although I would love to be proved wrong. So for the moment, my view is we still have to impose sequential execution, but lower the level at which this is done. > - We need a way of identifying when certain interesting thread/scheduling > operations occur, such as mutex locks, thread switches, etc, to support > Helgrind, Calltree, and future tools. Yes. What is needed is to intercept the relevant functions (pth_mutex_lock etc), note they have happened, but still allow the native thread library to handle it. The downside is we'd then need at least some simulation of the pthreads machinery in order to detect errors, but the good thing is we can make it as weedy as we like, commensurate with the errors we wish to detect. By contrast, currently our pthreads simulation has to be accurate enough to actually make the program work correctly, and we can't weaken it in any way. At least in theory, if people then want to run different threading schemes (foo_threads, for example) we will then be able to at least run a foo-threaded program, so long as it is underlyingly based on clone(). J |
|
From: Julian S. <js...@ac...> - 2004-08-09 18:29:43
|
On Saturday 07 August 2004 23:24, Jeremy Fitzhardinge wrote: > One of my design goals was to use as much of the kernel machinery as > possible so we don't have to emulate it - that's what the proxyLWPs are > for, but it wasn't as successful as I'd hoped for. In what way did it fall short of what you hoped to achieve? > Well, I think a big obvious first step is to drop all the pthreads > support from the core, which will simplify vg_scheduler a fair bit. The > scheduler doesn't really need to know much about a thread's state other > than its running or blocked on some event (which is probably always > syscall completion of some kind). That sounds good to me; I'll vote in favour. When you say that "the scheduler doesn't really need to know much about a thread's state other than its running or blocked on some event" I wonder if that could be done using the discrete-event-framework I suggested in a message half an hour ago. You can think of this as the OS-specific module placing an event in the event queue, which the core understands, to say "thread X is now blocked/runnable". I guess a good question is, what is the minimal set of thread events the core needs to know about? * thread creation / destruction * changes into / out of runnable state If the core is merely scheduling threads under the direction of a seperate OS-support module, does it need to know much more than this? The OS-support module observes and handles all thread-synchronisation events, the net external effect of which is merely to change the runnability status. But hey, I'm merely parroting what you are suggesting anyway. > This means that to do threading, all OS ports must do enough to support > the native threads library. The downside is that this could result in > as much complexity as dropping core ptheads support removes, and it > would be duplicated for each OS port. But it does mean that we don't > have to carry libpthread around, which is a large and increasing burden > (and that's just for all the x86-linux variants). I agree. I would just love to get rid of libpthread. > Also, one general comment about the refactoring. It's obviously a good > idea, and you've done a really good job so far. But we have to be > careful that when things get merged together, that they're actually > semantically the same rather than just coincidentally the same. For > example, if Linux and BSD happen to have the same shaped, say, signal > delivery structure, that doesn't mean we should use the same structure > definition and delivery functions for both. Their similarity is just a > coincidence, and we don't want to introduce a false coupling between > those implementations. Sure. Of course we have to be very careful about the semantics, more so than anything else. J |
|
From: Julian S. <js...@ac...> - 2004-08-09 17:44:34
|
> I am interested in hearing ideas about how things can be improved, eg. "A
> and B have a lot in common, if we factor the commonality out it will cut
> 100 lines of code and make C easier to understand", or "I noticed that we
> no longer need D since we changed E", or "merging F and G means that we
> can convert H from a global function into a local one", or higher-level
> things.
I think cleaning up the signals/syscalls/threads stuff -- what I think of
as the "environment simulation" -- is hugely important, and I'm pleased
to see you looking into it. As you say, the processor emulation stuff is
pretty much standard compiler technology and pretty much self-contained,
with a bit of care, so the difficulties in it are manageable.
I have no silver bullet (otherwise I would have deployed it by now :)
but some comments:
* One interesting stepping stone which would help explore the design
space is to consider if/how it is possible to run the native thread
library (nptl, I guess) directly. Instead of intercepting and scheduling
at the pthread level, we intercept clone() and schedule the results
ourselves. Does that help? Does it help move towards a more modular
framework? At least it would get rid of our libpthread, which has to
be a good thing.
* More generally, I wonder if it helps to try and structure valgrind
as a discrete event simulator. Note, this idea is very half-baked
and may be utterly irrelevant.
A discrete event simulator has a state, and a time-ordered queue of
future events. The basic cycle is to take the next-to-happen event off the
queue, and do it, which changes the state and/or possibly generates new
future events which are enqueued. This is standard stuff in the
chip-simulation etc community.
So, using C++ inheritance or whatever, you could have some core set of
events, with some core state, and each OS could have its own private
state and events pertaining to it. The event queue would hold a mixture
of the two. Valgrind's core would pull events off the queue, handle
the ones it understands directly, and call the relevant OS-specific
handler for OS-specific events it doesn't understand.
The kinds of events that would be in the queue might be:
* deliver a signal
* signal delivery done (if that makes sense, I suspect not)
* mess with host signals in some way
* check to see if some event has happened by some specific time
* syscall initiation
* syscall termination
* run a valgrind thread for a while
* low-level synchronisation events between V-scheduled threads
The aim is to decouple the V core as much as possible from the
env-simulation specifics. For example, imagine we want to valgrindify
a raw ARM image (on x86 host, perhaps) which expects to run in some
low-level environment, possibly without an OS at all. Then the
events it wants to do are nothing at all to do with the Unix
syscalls/signals/threads model and so support for the Unix model
shouldn't be hardwired into the valgrind core. Ideally, supporting
that low-level environment would be done by defining a set of
events for it, and giving handlers which "do" those events. Of course
the events are opaque to the V core, but that's a good thing.
Such a simulator scheme may not be a good way to structure it, but
(1) I haven't heard any other suggestions, and (2) it feels to me like
it has a kind of generality which is useful for modularising
things. So I'd be interested to see if/how far it can be developed.
Comments welcomed.
Re-reading it, what I am suggesting is an abstraction framework
based explicitly on the notions of state and events, and around
inheritance/opacity, by which details of events specific to a
specific environment (simulation) can be localised in a module
supporting that environment, instead of being scattered across
the entire code base. Wow, that's a long sentence.
* Richard Gabriel ("Patterns of Software") wrote an essay called
"Worse is better", and I think we're in that kind of situation now.
The current syscall/signal/thread scheme has been refined sufficiently
that it works pretty well on Linux. But when I originally started on
it (esp the threads stuff) I had no idea if it would succeed. So we
should be prepared to consider ideas which look promising but for
which not all the details are worked out ahead of time. Hence the
result may be worse for a while, but I see no other way of getting
a better understanding of the design space.
J
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 15:53:15
|
On Mon, 9 Aug 2004, Josef Weidendorfer wrote: > Check out Calltree from Joerg Schilling (As that's older, I will rename my > Valgrind tool to Callgrind with the next release); it gives a static call > graph for C code, and is able to produce GraphViz format. Ah, got it, and it works! Thanks, Josef. N |
|
From: Bob F. <bfr...@si...> - 2004-08-09 15:19:24
|
On Mon, 9 Aug 2004, Nicholas Nethercote wrote: > On Sat, 7 Aug 2004, Bob Friesenhahn wrote: > >> I obtained a script from somewhere which uses gprof output to generate a >> weighted call-graph using graphviz. Bob > > Thanks for the suggestion. Unfortunately, Valgrind and gprof don't interact > very happily, ie. Valgrind tends to crash when even parts of it are compiled > with -pg. A call-graph extractor that works statically on the source code > would be better... Using gprof is pretty involved anyway. I was suggesting graphviz as a useful tool which can be used elsewhere by valgrind. The gprof-based tool was just an example. One way graphviz could be used is to provide a summary of call stack vs leaks or (unreleased memory) as an alternative to the regular valgrind memcheck output. This would help the user correlate multiple reports. The figure would show a summary call graph resulting in unreleased memory allocations. In many situations, a common bug can result in many individual reports (e.g. if a C++ class destructor fails to release internal allocations or the destructor is never called). If redundant stack information is eliminated, the resulting digraph would make it very obvious where the memory allocations were made. This would save the user a lot of time, and would be more attractive than the normal valgrind output. Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 14:11:19
|
On Sat, 7 Aug 2004, Jeremy Fitzhardinge wrote:
>> One critical factor is that all these files spend a lot of time fiddling
>> about with the ThreadState of different threads.
>
> Apart from the pthreads state, I can't see anything which doesn't need
> to be there. The structure is ~900 bytes, and 512 bytes of that is sse
> state.
My concern isn't the size in bytes, rather the number of distinct fields.
>> Another factor is that the proxyLWP stuff, while undoubtedly very clever,
>> duplicates a lot of the code in vg_signals.c and vg_scheduler.c. Well,
>> not quite duplicates, but does stuff that is annoyingly similar. Perhaps
>> this situation could be improved?
>
> Could you be a bit more specific?
Maybe "duplicates" is not quite the right word... a lot of functions are
called both from vg_proxylwp.c and vg_scheduler.c. A couple of examples:
- VG_(do_signal_routing)()
- VG_(deliver_signal)()
- VG_(is_sig_ign)()
- VG_(proxy_create)()
- VG_(proxy_set_sigmask)()
etc. I'm not passing judgment on these particular examples, they're just
ones I saw.
I guess what I was trying to say: there seem to be a number of actions
whereby vg_scheduler.c or vg_signals.c handles the non-blocking-syscall
case, and vg_proxylwp.c handles the blocking syscall case, and the two
cases are similar but not identical. do_syscall/do_pthread_syscall is one
example.
----
> One of my design goals was to use as much of the kernel machinery as
> possible so we don't have to emulate it - that's what the proxyLWPs are
> for, but it wasn't as successful as I'd hoped for. If we could work out
> a way to get all the other behaviours we want, running the app threads
> in their own kernel threads would probably simplify a lot of this
> because we could just use the kernel directly, rather than having to
> emulate it quite so much.
>
> [...]
>
> Well, I think a big obvious first step is to drop all the pthreads
> support from the core, which will simplify vg_scheduler a fair bit. The
> scheduler doesn't really need to know much about a thread's state other
> than its running or blocked on some event (which is probably always
> syscall completion of some kind).
>
> This means that to do threading, all OS ports must do enough to support
> the native threads library. The downside is that this could result in
> as much complexity as dropping core ptheads support removes, and it
> would be duplicated for each OS port. But it does mean that we don't
> have to carry libpthread around, which is a large and increasing burden
> (and that's just for all the x86-linux variants).
Are these two points (above and below the "[...]") the same?
Ok, maybe this is the direction we should be heading in, then. So, what's
preventing us from doing so? AFAIK:
- We can't lose control of scheduling totally, because V turns atomic
instructions into non-atomic ones. We don't want a thread switch after
a shadow memory word is written, but before the word itself is written.
- We need a way of identifying when certain interesting thread/scheduling
operations occur, such as mutex locks, thread switches, etc, to support
Helgrind, Calltree, and future tools.
Do we have solutions for these, and are there any other problems?
N
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 13:52:02
|
On Sat, 7 Aug 2004, Bob Friesenhahn wrote: > I obtained a script from somewhere which uses gprof output to generate a > weighted call-graph using graphviz. Bob Thanks for the suggestion. Unfortunately, Valgrind and gprof don't interact very happily, ie. Valgrind tends to crash when even parts of it are compiled with -pg. A call-graph extractor that works statically on the source code would be better... N |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 13:13:39
|
CVS commit by nethercote:
Change VG_(scheduler)() slightly to remove two more global vars from
vg_include.h.
M +6 -7 vg_include.h 1.231
M +5 -15 vg_main.c 1.193
M +30 -1 vg_scheduler.c 1.168
M +1 -4 vg_signals.c 1.80
--- valgrind/coregrind/vg_include.h #1.230:1.231
@@ -931,7 +931,7 @@ typedef
-/* The scheduler. */
-extern VgSchedReturnCode VG_(scheduler) ( Int* exit_code,
- ThreadId* last_run_thread );
+// The scheduler. 'fatal_sigNo' is only set if VgSrc_FatalSig is returned.
+extern VgSchedReturnCode VG_(scheduler)
+ ( Int* exit_code, ThreadId* last_run_thread, Int* fatal_sigNo );
extern void VG_(scheduler_init) ( void );
@@ -942,4 +942,7 @@ extern void VG_(pp_sched_status) ( void
extern void VG_(resume_scheduler) ( Int sigNo, vki_ksiginfo_t *info );
+// Longjmp, ending the scheduler, when a fatal signal occurs in the client.
+extern void VG_(scheduler_handle_fatal_signal)( Int sigNo );
+
/* The red-zone size which we put at the bottom (highest address) of
thread stacks, for paranoia reasons. This can be arbitrary, and
@@ -1286,8 +1289,4 @@ extern void VG_(start_debugger) ( Int ti
extern UInt VG_(dispatch_ctr);
-/* If we're doing the default action of a fatal signal */
-extern jmp_buf* VG_(fatal_signal_jmpbuf_ptr);
-extern Int VG_(fatal_sigNo); /* the fatal signal */
-
/* --- Counters, for informational purposes only. --- */
--- valgrind/coregrind/vg_main.c #1.192:1.193
@@ -150,10 +150,4 @@ Char** VG_(client_envp);
------------------------------------------------------------------ */
-/* jmp_buf for fatal signals; VG_(fatal_signal_jmpbuf_ptr) is NULL until
- the time is right that it can be used. */
- Int VG_(fatal_sigNo) = -1;
- jmp_buf* VG_(fatal_signal_jmpbuf_ptr) = NULL;
-static jmp_buf fatal_signal_jmpbuf;
-
/* Counts downwards in VG_(run_innerloop). */
UInt VG_(dispatch_ctr);
@@ -2748,4 +2742,5 @@ int main(int argc, char **argv)
VgSchedReturnCode src;
Int exitcode = 0;
+ Int fatal_sigNo = -1;
vki_rlimit zero = { 0, 0 };
Int padfile;
@@ -3044,12 +3039,7 @@ int main(int argc, char **argv)
VGP_PUSHCC(VgpSched);
- VG_(fatal_signal_jmpbuf_ptr) = &fatal_signal_jmpbuf;
- if (__builtin_setjmp(VG_(fatal_signal_jmpbuf_ptr)) == 0) {
- src = VG_(scheduler)( &exitcode, &last_run_tid );
- } else {
- src = VgSrc_FatalSig;
- }
- VGP_POPCC(VgpSched);
+ src = VG_(scheduler)( &exitcode, &last_run_tid, &fatal_sigNo );
+ VGP_POPCC(VgpSched);
@@ -3114,6 +3104,6 @@ int main(int argc, char **argv)
case VgSrc_FatalSig:
/* We were killed by a fatal signal, so replicate the effect */
- vg_assert(VG_(fatal_sigNo) != -1);
- VG_(kill_self)(VG_(fatal_sigNo));
+ vg_assert(fatal_sigNo != -1);
+ VG_(kill_self)(fatal_sigNo);
VG_(core_panic)("main(): signal was supposed to be fatal");
break;
--- valgrind/coregrind/vg_scheduler.c #1.167:1.168
@@ -840,4 +840,11 @@ void idle ( void )
------------------------------------------------------------------ */
+// For handling of the default action of a fatal signal.
+// jmp_buf for fatal signals; VG_(fatal_signal_jmpbuf_ptr) is NULL until
+// the time is right that it can be used.
+static jmp_buf fatal_signal_jmpbuf;
+static jmp_buf* fatal_signal_jmpbuf_ptr;
+static Int fatal_sigNo; // the fatal signal, if it happens
+
/* Run user-space threads until either
* Deadlock occurs
@@ -845,5 +852,5 @@ void idle ( void )
* The specified number of basic blocks has gone by.
*/
-VgSchedReturnCode VG_(scheduler) ( Int* exitcode, ThreadId* last_run_tid )
+VgSchedReturnCode do_scheduler ( Int* exitcode, ThreadId* last_run_tid )
{
ThreadId tid, tid_next;
@@ -1215,4 +1222,19 @@ VgSchedReturnCode VG_(scheduler) ( Int*
}
+VgSchedReturnCode VG_(scheduler) ( Int* exitcode, ThreadId* last_run_tid,
+ Int* fatal_sigNo_ptr )
+{
+ VgSchedReturnCode src;
+
+ fatal_signal_jmpbuf_ptr = &fatal_signal_jmpbuf;
+ if (__builtin_setjmp( fatal_signal_jmpbuf_ptr ) == 0) {
+ src = do_scheduler( exitcode, last_run_tid );
+ } else {
+ src = VgSrc_FatalSig;
+ *fatal_sigNo_ptr = fatal_sigNo;
+ }
+ return src;
+}
+
void VG_(need_resched) ( ThreadId prefer )
{
@@ -1250,4 +1272,11 @@ void VG_(need_resched) ( ThreadId prefer
}
+void VG_(scheduler_handle_fatal_signal) ( Int sigNo )
+{
+ if (NULL != fatal_signal_jmpbuf_ptr) {
+ fatal_sigNo = sigNo;
+ __builtin_longjmp(*fatal_signal_jmpbuf_ptr, 1);
+ }
+}
/* ---------------------------------------------------------------------
--- valgrind/coregrind/vg_signals.c #1.79:1.80
@@ -1804,8 +1804,5 @@ static void vg_default_action(const vki_
}
- if (NULL != VG_(fatal_signal_jmpbuf_ptr)) {
- VG_(fatal_sigNo) = sigNo;
- __builtin_longjmp(*VG_(fatal_signal_jmpbuf_ptr), 1);
- }
+ VG_(scheduler_handle_fatal_signal)( sigNo );
}
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 12:40:11
|
On Sun, 25 Jul 2004, Nicholas Nethercote wrote: > In making the code 64-bit clean, one big area that needs fixing is the > allocator, vg_malloc2.c. I've completely reworked it so that it's > (almost) 64-bit clean. > > The diff is at > > www.cl.cam.ac.uk/~njn25/diff.64-bit-alloc > > The new version of vg_malloc2.c is at > > www.cl.cam.ac.uk/~njn25/vg_malloc2.c > > Reading the diff is hard because so much has changed in vg_malloc2.c. > The change log is below. > > I'd be interested in hearing feedback. I'd like to see the changes go in > now, even though Opteron support is still a long way off. I think I improved > enough things while doing the change for it to be worthwhile (eg. I find it > easier to understand, there are more comments, some incorrect corner cases > are now working, magic numbers have been removed), and it would be good to > have it in earlier to flush out any problems that would affect x86 (32-bit > machines) sooner rather than later. Opinions? I'll commit this in the next day or so if no-one objects. N |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 12:22:30
|
CVS commit by nethercote:
Remove some more global variables from vg_include.h, replacing them with
(fewer) functions.
Also fixed execve() so that it works better with .in_place.
Also added a regression test for --trace-children=yes (there were none!)
A memcheck/tests/execve2.c 1.1 [no copyright]
A memcheck/tests/execve2.stderr.exp 1.1
A memcheck/tests/execve2.vgtest 1.1
M +4 -8 coregrind/vg_include.h 1.230
M +130 -55 coregrind/vg_main.c 1.192 [POSSIBLY UNSAFE: printf]
M +8 -60 coregrind/vg_syscalls.c 1.120
M +1 -0 memcheck/tests/.cvsignore 1.13
M +3 -1 memcheck/tests/Makefile.am 1.39
--- valgrind/coregrind/vg_include.h #1.229:1.230
@@ -1266,10 +1265,11 @@ extern Addr VG_(valgrind_end);
extern vki_rlimit VG_(client_rlimit_data); /* client's original rlimit data */
-/* stage1 executable file descriptor */
-extern Int VG_(vgexecfd);
-
/* client executable file descriptor */
extern Int VG_(clexecfd);
+// Help set up the child used when doing execve() with --trace-children=yes
+Char* VG_(build_child_VALGRINDCLO) ( Char* exename );
+Char* VG_(build_child_exename) ( void );
+
/* Determine if %esp adjustment must be noted */
extern Bool VG_(need_to_handle_esp_assignment) ( void );
@@ -1280,8 +1280,4 @@ extern void VG_(unimplemented) ( Char* m
__attribute__((__noreturn__));
-/* Valgrind's argc and argv */
-extern Int VG_(vg_argc);
-extern Char **VG_(vg_argv);
-
/* Something of a function looking for a home ... start up debugger. */
extern void VG_(start_debugger) ( Int tid );
--- valgrind/coregrind/vg_main.c #1.191:1.192
@@ -118,5 +118,5 @@ Bool VG_(have_ssestate);
/* stage1 (main) executable */
-Int VG_(vgexecfd) = -1;
+static Int vgexecfd = -1;
/* client executable */
@@ -127,6 +127,6 @@ const Char *VG_(libdir) = VG_LIBDIR;
/* our argc/argv */
-Int VG_(vg_argc);
-Char **VG_(vg_argv);
+static Int vg_argc;
+static Char **vg_argv;
/* PID of the main thread */
@@ -414,5 +414,5 @@ int scan_auxv(void)
case AT_UME_EXECFD:
- VG_(vgexecfd) = auxv->u.a_val;
+ vgexecfd = auxv->u.a_val;
found |= 2;
break;
@@ -568,6 +568,6 @@ static char** copy_args( char* s, char**
static void augment_command_line(Int* vg_argc_inout, char*** vg_argv_inout)
{
- int vg_argc = *vg_argc_inout;
- char** vg_argv = *vg_argv_inout;
+ int vg_argc0 = *vg_argc_inout;
+ char** vg_argv0 = *vg_argv_inout;
char* env_clo = getenv(VALGRINDOPTS);
@@ -591,9 +591,9 @@ static void augment_command_line(Int* vg
/* +2: +1 for null-termination, +1 for added '--' */
- from = vg_argv;
- vg_argv = malloc( (vg_argc + env_arg_count + f1_arg_count
+ from = vg_argv0;
+ vg_argv0 = malloc( (vg_argc0 + env_arg_count + f1_arg_count
+ f2_arg_count + 2) * sizeof(char **));
- vg_assert(vg_argv);
- to = vg_argv;
+ vg_assert(vg_argv0);
+ to = vg_argv0;
/* copy argv[0] */
@@ -621,5 +621,5 @@ static void augment_command_line(Int* vg
*to++ = "--";
- vg_argc = to - vg_argv;
+ vg_argc0 = to - vg_argv0;
/* copy rest of original command line, then NULL */
@@ -628,14 +628,16 @@ static void augment_command_line(Int* vg
}
- *vg_argc_inout = vg_argc;
- *vg_argv_inout = vg_argv;
+ *vg_argc_inout = vg_argc0;
+ *vg_argv_inout = vg_argv0;
}
+#define VG_CLO_SEP '\01'
+
static void get_command_line( int argc, char** argv,
Int* vg_argc_out, Char*** vg_argv_out,
char*** cl_argv_out )
{
- int vg_argc;
- char** vg_argv;
+ int vg_argc0;
+ char** vg_argv0;
char** cl_argv;
char* env_clo = getenv(VALGRINDCLO);
@@ -645,21 +647,24 @@ static void get_command_line( int argc,
char **cpp;
- /* OK, we're getting all our arguments from the environment - the
- entire command line belongs to the client (including argv[0]) */
- vg_argc = 1; /* argv[0] */
+ /* OK, VALGRINDCLO is set, which means we must be a child of another
+ Valgrind process using --trace-children, so we're getting all our
+ arguments from VALGRINDCLO, and the entire command line belongs to
+ the client (including argv[0]) */
+ vg_argc0 = 1; /* argv[0] */
for (cp = env_clo; *cp; cp++)
- if (*cp == '\01')
- vg_argc++;
+ if (*cp == VG_CLO_SEP)
+ vg_argc0++;
- vg_argv = malloc(sizeof(char **) * (vg_argc + 1));
- vg_assert(vg_argv);
+ vg_argv0 = malloc(sizeof(char **) * (vg_argc0 + 1));
+ vg_assert(vg_argv0);
- cpp = vg_argv;
+ cpp = vg_argv0;
*cpp++ = "valgrind"; /* nominal argv[0] */
*cpp++ = env_clo;
+ // Replace the VG_CLO_SEP args separator with '\0'
for (cp = env_clo; *cp; cp++) {
- if (*cp == '\01') {
+ if (*cp == VG_CLO_SEP) {
*cp++ = '\0'; /* chop it up in place */
*cpp++ = cp;
@@ -671,29 +676,30 @@ static void get_command_line( int argc,
} else {
/* Count the arguments on the command line. */
- vg_argv = argv;
+ vg_argv0 = argv;
- for (vg_argc = 1; vg_argc < argc; vg_argc++) {
- if (argv[vg_argc][0] != '-') /* exe name */
+ for (vg_argc0 = 1; vg_argc0 < argc; vg_argc0++) {
+ if (argv[vg_argc0][0] != '-') /* exe name */
break;
- if (VG_STREQ(argv[vg_argc], "--")) { /* dummy arg */
- vg_argc++;
+ if (VG_STREQ(argv[vg_argc0], "--")) { /* dummy arg */
+ vg_argc0++;
break;
}
}
- cl_argv = &argv[vg_argc];
+ cl_argv = &argv[vg_argc0];
/* Get extra args from VALGRIND_OPTS and .valgrindrc files.
- * Note we don't do this if getting args from VALGRINDCLO. */
- augment_command_line(&vg_argc, &vg_argv);
+ Note we don't do this if getting args from VALGRINDCLO, as
+ those extra args will already be present in VALGRINDCLO. */
+ augment_command_line(&vg_argc0, &vg_argv0);
}
if (0) {
Int i;
- for (i = 0; i < vg_argc; i++)
- printf("vg_argv[%d]=\"%s\"\n", i, vg_argv[i]);
+ for (i = 0; i < vg_argc0; i++)
+ printf("vg_argv0[%d]=\"%s\"\n", i, vg_argv0[i]);
}
- *vg_argc_out = vg_argc;
- *vg_argv_out = (Char**)vg_argv;
+ *vg_argc_out = vg_argc0;
+ *vg_argv_out = (Char**)vg_argv0;
*cl_argv_out = cl_argv;
}
@@ -1414,5 +1420,5 @@ static void load_client(char* cl_argv[],
/*====================================================================*/
-/*=== Command-line: variables, processing ===*/
+/*=== Command-line: variables, processing, etc ===*/
/*====================================================================*/
@@ -1570,23 +1576,23 @@ static void pre_process_cmd_line_options
/* parse the options we have (only the options we care about now) */
- for (i = 1; i < VG_(vg_argc); i++) {
+ for (i = 1; i < vg_argc; i++) {
- if (strcmp(VG_(vg_argv)[i], "--version") == 0) {
+ if (strcmp(vg_argv[i], "--version") == 0) {
printf("valgrind-" VERSION "\n");
exit(0);
- } else if (VG_CLO_STREQ(VG_(vg_argv)[i], "--help") ||
- VG_CLO_STREQ(VG_(vg_argv)[i], "-h")) {
+ } else if (VG_CLO_STREQ(vg_argv[i], "--help") ||
+ VG_CLO_STREQ(vg_argv[i], "-h")) {
*need_help = 1;
- } else if (VG_CLO_STREQ(VG_(vg_argv)[i], "--help-debug")) {
+ } else if (VG_CLO_STREQ(vg_argv[i], "--help-debug")) {
*need_help = 2;
- } else if (VG_CLO_STREQN(7, VG_(vg_argv)[i], "--tool=") ||
- VG_CLO_STREQN(7, VG_(vg_argv)[i], "--skin=")) {
- *tool = &VG_(vg_argv)[i][7];
+ } else if (VG_CLO_STREQN(7, vg_argv[i], "--tool=") ||
+ VG_CLO_STREQN(7, vg_argv[i], "--skin=")) {
+ *tool = &vg_argv[i][7];
- } else if (VG_CLO_STREQN(7, VG_(vg_argv)[i], "--exec=")) {
- *exec = &VG_(vg_argv)[i][7];
+ } else if (VG_CLO_STREQN(7, vg_argv[i], "--exec=")) {
+ *exec = &vg_argv[i][7];
}
}
@@ -1626,7 +1632,7 @@ static void process_cmd_line_options( UI
}
- for (i = 1; i < VG_(vg_argc); i++) {
+ for (i = 1; i < vg_argc; i++) {
- Char* arg = VG_(vg_argv)[i];
+ Char* arg = vg_argv[i];
Char* colon = arg;
@@ -1933,6 +1939,6 @@ static void process_cmd_line_options( UI
VG_(message)(Vg_UserMsg, "Startup, with flags:");
- for (i = 1; i < VG_(vg_argc); i++) {
- VG_(message)(Vg_UserMsg, " %s", VG_(vg_argv)[i]);
+ for (i = 1; i < vg_argc; i++) {
+ VG_(message)(Vg_UserMsg, " %s", vg_argv[i]);
}
@@ -1979,4 +1985,73 @@ static void process_cmd_line_options( UI
}
+// Build the string for VALGRINDCLO.
+Char* VG_(build_child_VALGRINDCLO)( Char* exename )
+{
+ /* If we're tracing the children, then we need to start it
+ with our starter+arguments, which are copied into VALGRINDCLO,
+ except the --exec= option is changed if present.
+ */
+ Int i;
+ Char *exec;
+ Char *cp;
+ Char *optvar;
+ Int optlen, execlen;
+
+ // All these allocated blocks are not free - because we're either
+ // going to exec, or panic when we fail.
+
+ // Create --exec= option: "--exec=<exename>"
+ exec = VG_(arena_malloc)(VG_AR_CORE,
+ VG_(strlen)( exename ) + 7/*--exec=*/ + 1/*\0*/);
+ vg_assert(NULL != exec);
+ VG_(sprintf)(exec, "--exec=%s", exename);
+
+ // Allocate space for optvar (may overestimate by counting --exec twice,
+ // no matter)
+ optlen = 1;
+ for (i = 0; i < vg_argc; i++)
+ optlen += VG_(strlen)(vg_argv[i]) + 1;
+ optlen += VG_(strlen)(exec)+1;
+ optvar = VG_(arena_malloc)(VG_AR_CORE, optlen);
+
+ // Copy all valgrind args except the old --exec (if present)
+ // VG_CLO_SEP is the separator.
+ cp = optvar;
+ for (i = 1; i < vg_argc; i++) {
+ Char *arg = vg_argv[i];
+
+ if (VG_(memcmp)(arg, "--exec=", 7) == 0) {
+ // don't copy existing --exec= arg
+ } else if (VG_(strcmp)(arg, "--") == 0) {
+ // stop at "--"
+ break;
+ } else {
+ // copy non "--exec" arg
+ Int len = VG_(strlen)(arg);
+ VG_(memcpy)(cp, arg, len);
+ cp += len;
+ *cp++ = VG_CLO_SEP;
+ }
+ }
+ // Add the new --exec= option
+ execlen = VG_(strlen)(exec);
+ VG_(memcpy)(cp, exec, execlen);
+ cp += execlen;
+ *cp++ = VG_CLO_SEP;
+
+ *cp = '\0';
+
+ return optvar;
+}
+
+// Build "/proc/self/fd/<execfd>".
+Char* VG_(build_child_exename)( void )
+{
+ Char* exename = VG_(arena_malloc)(VG_AR_CORE, 64);
+ vg_assert(NULL != exename);
+ VG_(sprintf)(exename, "/proc/self/fd/%d", vgexecfd);
+ return exename;
+}
+
/*====================================================================*/
@@ -2008,6 +2083,6 @@ static void setup_file_descriptors(void)
VG_(setrlimit)(VKI_RLIMIT_NOFILE, &rl);
- if (VG_(vgexecfd) != -1)
- VG_(vgexecfd) = VG_(safe_fd)( VG_(vgexecfd) );
+ if (vgexecfd != -1)
+ vgexecfd = VG_(safe_fd)( vgexecfd );
if (VG_(clexecfd) != -1)
VG_(clexecfd) = VG_(safe_fd)( VG_(clexecfd) );
@@ -2726,5 +2801,5 @@ int main(int argc, char **argv)
// p: n/a
//--------------------------------------------------------------
- get_command_line(argc, argv, &VG_(vg_argc), &VG_(vg_argv), &cl_argv);
+ get_command_line(argc, argv, &vg_argc, &vg_argv, &cl_argv);
pre_process_cmd_line_options(&need_help, &tool, &exec);
@@ -2783,5 +2858,5 @@ int main(int argc, char **argv)
if (0)
printf("entry=%x client esp=%x vg_argc=%d brkbase=%x\n",
- client_eip, esp_at_startup, VG_(vg_argc), VG_(brk_base));
+ client_eip, esp_at_startup, vg_argc, VG_(brk_base));
//==============================================================
--- valgrind/coregrind/vg_syscalls.c #1.119:1.120
@@ -1788,64 +1788,12 @@ PRE(execve)
if (VG_(clo_trace_children)) {
- /* If we're tracing the children, then we need to start it
- with our starter+arguments.
- */
- Int i;
- Char *exec = (Char *)arg1;
- Char **env = (Char **)arg3;
- Char *cp;
- Char *exename;
- Bool sawexec = False;
- Char *optvar;
- Int optlen;
-
- optlen = 1;
- for(i = 0; i < VG_(vg_argc); i++)
- optlen += VG_(strlen)(VG_(vg_argv)[i]) + 1;
-
- /* All these are leaked - we're either going to exec, or panic
- when we fail. */
- exename = VG_(arena_malloc)(VG_AR_CORE, 64);
- exec = VG_(arena_malloc)(VG_AR_CORE, VG_(strlen)(exec) + 7 /* --exec= */ + 1 /* \0 */);
-
- VG_(sprintf)(exec, "--exec=%s", (Char *)arg1);
- VG_(sprintf)(exename, "/proc/self/fd/%d", VG_(vgexecfd));
-
- optlen += VG_(strlen)(exec)+1;
-
- optvar = VG_(arena_malloc)(VG_AR_CORE, optlen);
-
- /* valgrind arguments */
- cp = optvar;
-
- for(i = 1; i < VG_(vg_argc); i++) {
- Char *arg = VG_(vg_argv)[i];
- Int len;
-
- if (VG_(memcmp)(arg, "--exec=", 7) == 0) {
- /* replace existing --exec= arg */
- sawexec = True;
- arg = exec;
- } else if (VG_(strcmp)(VG_(vg_argv)[i], "--") == 0)
- break;
-
- len = VG_(strlen)(arg);
- VG_(memcpy)(cp, arg, len);
- cp += len;
- *cp++ = '\01';
- }
-
- if (!sawexec) {
- Int execlen = VG_(strlen)(exec);
- VG_(memcpy)(cp, exec, execlen);
- cp += execlen;
- *cp++ = '\01';
- }
- *cp = '\0';
+ Char* optvar = VG_(build_child_VALGRINDCLO)( (Char*)arg1 );
- VG_(env_setenv)(&env, VALGRINDCLO, optvar);
+ // Set VALGRINDCLO and VALGRINDLIB in arg3 (the environment)
+ VG_(env_setenv)( (Char***)&arg3, VALGRINDCLO, optvar);
+ VG_(env_setenv)( (Char***)&arg3, VALGRINDLIB, VG_(libdir));
- arg1 = (UInt)exename;
- arg3 = (UInt)env;
+ // Create executable name: "/proc/self/fd/<vgexecfd>", update arg1
+ arg1 = (UInt)VG_(build_child_exename)();
}
--- valgrind/memcheck/tests/.cvsignore #1.12:1.13
@@ -14,4 +14,5 @@
errs1
execve
+execve2
exitprog
filter_leak_check_size
--- valgrind/memcheck/tests/Makefile.am #1.38:1.39
@@ -28,4 +28,5 @@
exitprog.stderr.exp exitprog.vgtest \
execve.stderr.exp execve.vgtest \
+ execve2.stderr.exp execve2.vgtest \
fpeflags.stderr.exp fpeflags.vgtest \
fprw.stderr.exp fprw.vgtest \
@@ -78,5 +79,5 @@
badaddrvalue badfree badjump badloop badrw brk buflen_check \
clientperm custom_alloc \
- doublefree error_counts errs1 exitprog execve \
+ doublefree error_counts errs1 exitprog execve execve2 \
fpeflags fprw fwrite inits inline \
malloc1 malloc2 malloc3 manuel1 manuel2 manuel3 \
@@ -105,4 +106,5 @@
errs1_SOURCES = errs1.c
execve_SOURCES = execve.c
+execve2_SOURCES = execve2.c
exitprog_SOURCES = exitprog.c
fpeflags_SOURCES = fpeflags.c
|
|
From: Oswald B. <os...@kd...> - 2004-08-09 11:38:26
|
On Mon, Aug 09, 2004 at 01:15:10PM +0200, Nicholas Nethercote wrote: > The weak attribute > +// ensures the multiple definitions are not a problem. They must be functions > +// rather than macros so that va_list can be used. > + hmm, maybe make them static inline instead of weak? dunno ... -- Hi! I'm a .signature virus! Copy me into your ~/.signature, please! -- Chaos, panic, and disorder - my work here is done. |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 11:15:22
|
CVS commit by nethercote:
Put VALGRIND_INTERNAL_PRINTF and VALGRIND_INTERNAL_PRINTF_BACKTRACE back in
vg_include.h, where they must be otherwise vg_libtpthread.c has problems.
Added a comment explaining why they must be in vg_include.h.
M +31 -3 vg_include.h 1.229
M +1 -24 vg_messages.c 1.13
--- valgrind/coregrind/vg_include.h #1.228:1.229
@@ -1076,7 +1076,35 @@ extern void VG_(send_bytes_to_logging_si
// Functions for printing from code within Valgrind, but which runs on the
-// sim'd CPU.
-int VALGRIND_INTERNAL_PRINTF ( char *format, ... );
-int VALGRIND_INTERNAL_PRINTF_BACKTRACE ( char *format, ... );
+// sim'd CPU. Defined here because needed for vg_libpthread.c,
+// vg_replace_malloc.c, plus the rest of the core. The weak attribute
+// ensures the multiple definitions are not a problem. They must be functions
+// rather than macros so that va_list can be used.
+
+__attribute__((weak))
+int
+VALGRIND_INTERNAL_PRINTF(char *format, ...)
+{
+ unsigned int _qzz_res = 0;
+ va_list vargs;
+ va_start(vargs, format);
+ VALGRIND_MAGIC_SEQUENCE(_qzz_res, 0, VG_USERREQ__INTERNAL_PRINTF,
+ (unsigned int)format, (unsigned int)vargs, 0, 0);
+ va_end(vargs);
+ return _qzz_res;
+}
+
+__attribute__((weak))
+int
+VALGRIND_INTERNAL_PRINTF_BACKTRACE(char *format, ...)
+{
+ unsigned int _qzz_res = 0;
+ va_list vargs;
+ va_start(vargs, format);
+ VALGRIND_MAGIC_SEQUENCE(_qzz_res, 0, VG_USERREQ__INTERNAL_PRINTF_BACKTRACE,
+ (unsigned int)format, (unsigned int)vargs, 0, 0);
+ va_end(vargs);
+ return _qzz_res;
+}
+
--- valgrind/coregrind/vg_messages.c #1.12:1.13
@@ -127,28 +127,5 @@ void VG_(send_bytes_to_logging_sink) ( C
}
-int VALGRIND_INTERNAL_PRINTF(char *format, ...)
-{
- unsigned int _qzz_res = 0;
- va_list vargs;
- va_start(vargs, format);
- VALGRIND_MAGIC_SEQUENCE(_qzz_res, 0, VG_USERREQ__INTERNAL_PRINTF,
- (unsigned int)format, (unsigned int)vargs, 0, 0);
- va_end(vargs);
- return _qzz_res;
-}
-
-int VALGRIND_INTERNAL_PRINTF_BACKTRACE(char *format, ...)
-{
- unsigned int _qzz_res = 0;
- va_list vargs;
- va_start(vargs, format);
- VALGRIND_MAGIC_SEQUENCE(_qzz_res, 0, VG_USERREQ__INTERNAL_PRINTF_BACKTRACE,
- (unsigned int)format, (unsigned int)vargs, 0, 0);
- va_end(vargs);
- return _qzz_res;
-}
-
-
/*--------------------------------------------------------------------*/
-/*--- end vg_messages.c ---*/
+/*--- end ---*/
/*--------------------------------------------------------------------*/
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 09:27:00
|
On Mon, 9 Aug 2004, Nicholas Nethercote wrote: > I moved the function VALGRIND_INTERNAL_PRINT() into vg_messages.c in a recent > commit, out of vg_include.h. I just found that's no good, because > vg_libpthread.c uses it, and gets linker errors when it tries. > > A better solution would be rewrite it as a macro, and put it in valgrind.h, > so that vg_libpthread.c (and vg_replace_malloc.c) can see it. > > But then, VALGRIND_PRINTF is already in valgrind.h, and it's essentially > identical to VALGRIND_INTERNAL_PRINTF. > > So: > > - Do we need both VALGRIND_PRINTF and VALGRIND_INTERNAL_PRINTF? I don't > think so. > > Also: > > - Why is VALGRIND_PRINTF wrapped in a "#ifndef NVALGRIND" conditional? > Surely it should be covered by the #ifndef NVALGRIND conditional on > VALGRIND_MAGIC_SEQUENCE? I don't think this #ifndef is necessary. Urgh, now I see there's a difference between the _PRINTF_ and _INTERNAL_PRINTF_ versions -- the first arg to VG_(vmessage)() is different, so the output formatting is different. Also, I see why they were defined as functions rather than macros -- so va_list could be used. I also guess that #ifndef was used because they are functions rather than macros. Hmm, I may well have to put the _INTERNAL_ functions back into vg_include.h, ie. undo my original change. N |
|
From: Nicholas N. <nj...@ca...> - 2004-08-09 09:14:36
|
Hi, I moved the function VALGRIND_INTERNAL_PRINT() into vg_messages.c in a recent commit, out of vg_include.h. I just found that's no good, because vg_libpthread.c uses it, and gets linker errors when it tries. A better solution would be rewrite it as a macro, and put it in valgrind.h, so that vg_libpthread.c (and vg_replace_malloc.c) can see it. But then, VALGRIND_PRINTF is already in valgrind.h, and it's essentially identical to VALGRIND_INTERNAL_PRINTF. So: - Do we need both VALGRIND_PRINTF and VALGRIND_INTERNAL_PRINTF? I don't think so. Also: - Why is VALGRIND_PRINTF wrapped in a "#ifndef NVALGRIND" conditional? Surely it should be covered by the #ifndef NVALGRIND conditional on VALGRIND_MAGIC_SEQUENCE? I don't think this #ifndef is necessary. (Nb: All this applies to VALGRIND_PRINTF_BACKTRACE too.) Thanks. N |