You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
1
(15) |
2
(17) |
3
(23) |
4
(13) |
5
(7) |
6
(8) |
7
(9) |
|
8
(8) |
9
(31) |
10
(31) |
11
(19) |
12
(11) |
13
(38) |
14
(14) |
|
15
(8) |
16
(11) |
17
(7) |
18
(17) |
19
(12) |
20
(12) |
21
(17) |
|
22
(19) |
23
(33) |
24
(42) |
25
(37) |
26
(23) |
27
(27) |
28
(27) |
|
29
(16) |
30
(52) |
31
(33) |
|
|
|
|
|
From: Bob F. <bfr...@si...> - 2004-08-10 22:37:20
|
On Tue, 10 Aug 2004, Jeremy Fitzhardinge wrote: > > Well, there's still a lot of gunk in vg_signals.c. One problem is Since Valgrind is apparently full of "gunk" and as OS support is added/extended, the level of "gunk" may increase, has it been considered to use loadable (or compile-time) driver-like modules to try to make sense of it all? If the interfaces can be defined completely, then it should be possible to support new OS versions without inadvertently breaking older versions. There may be more replicated code across modules, but each module should be simpler and easier to maintain. Thoughts? Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Jeremy F. <je...@go...> - 2004-08-10 21:30:42
|
On Mon, 2004-08-09 at 19:29 +0100, Julian Seward wrote: > On Saturday 07 August 2004 23:24, Jeremy Fitzhardinge wrote: > > > One of my design goals was to use as much of the kernel machinery as > > possible so we don't have to emulate it - that's what the proxyLWPs are > > for, but it wasn't as successful as I'd hoped for. > > In what way did it fall short of what you hoped to achieve? Well, there's still a lot of gunk in vg_signals.c. One problem is needing to support both 2.4 and 2.6 kernels, since their thread and signal models are very different, so there's almost two distinct code paths in there. If we go to "emulate clone" threading, this will get worse, since 2.4 and 2.6 clone are also quite different. There's also just a lot of careful state management to make sure the right thread has the right signal mask, and the right signal gets delivered to the right place and interrupts the right system call. The proxyLWP stuff, in 2.6, gets the kernel to do signal routing (ie, deciding which thread to hit with the signal), but the rest is still our code. J |
|
From: Jeremy F. <je...@go...> - 2004-08-10 21:04:10
|
On Mon, 2004-08-09 at 18:44 +0100, Julian Seward wrote:
> Such a simulator scheme may not be a good way to structure it, but
> (1) I haven't heard any other suggestions, and (2) it feels to me like
> it has a kind of generality which is useful for modularising
> things. So I'd be interested to see if/how far it can be developed.
> Comments welcomed.
I would argue that this is already there. The interface between
vg_scheduler and the rest of the OS-interface code is already pretty
thin, and could be cleaned up pretty easily (this is assuming you ignore
the pthreads stuff, which is a lot of the apparent complexity).
The scheduler loop is basically:
for(ever)
for(each_thread)
if runnable
run it
if (nothing ran)
idle()
And idle() is responsible for collecting async events from elsewhere.
These events are:
* timeout - not really used for anything anymore, except
pthread_cond_timewait
* signal polling - not really an event, just something which needs
to happen periodically on 2.4 kernels
* kernel event - a thread woke up from being blocked in a syscall,
either because it was interrupted by a signal, or because the
syscall completed normally
idle(), and vg_scheduler.c, doesn't do anything much with this info - it
just uses it as a hint to rescan the thread list and find something
runnable.
Most of the work (which is the part which has Nick worried, and
definitely needs cleaning up) is done by things which directly change
the thread state in response to the event. For syscalls, its just "keep
running normally"; for signals, it loads a whole new CPU context to run
the signal handler.
If we move all the CPU-specific stuff out of vg_scheduler.c (the actual
context-switching machinery) and drop all the lipthread stuff, then
vg_scheduler will essentially be just this event loop, which is nice and
simple.
Well, the other part is dealing with client requests coming out of the
running client instruction stream. This is also a pretty thin
interface. If its a client request, then do it; if its a syscall, then
just call the arch/OS-specific "do syscall" routine, saying "this thread
N has a syscall set up in its context, go do it"; vg_scheduler itself
doesn't need to know any other details.
Now one thing which particularly worries me is the number of places
which touch ThreadState->status directly, mostly because there are so
many states. Every place which changes the state needs to make sure
that their doing a valid transition in the state machine, and since I
don't think there's a formal state machine for the thread state, that's
hard to do, and hard to verify.
One of the big advantages to me of dropping the pthreads stuff is that
we can drop all those ThreadStates associated with them, which will make
the state machine much more managable. Then we can check all the places
where the thread state changes and make sure they're valid transitions.
This would naturally fix some of the bugs we currently have in the state
machine (like not being able to deal sensibly with a pthread_mutex_lock
being interrupted by a signal and running a signal handler).
J
|
|
From: Jeremy F. <je...@go...> - 2004-08-10 20:34:14
|
On Mon, 2004-08-09 at 23:24 +0100, Julian Seward wrote:
> > I'm not saying we shouldn't have done FV. Because, on the other hand,
> > one of the best things about Valgrind from a user's point of view is the
> > "it just works" factor. We really want to keep that as high as possible,
> > so that's kind of an argument against the worse-is-better approach.
>
> Re FV, and perhaps the proxyLWP stuff, Jeremy argues that we should
> use as much of the underlying OSs services as possible, and make V be
> the thinnest possible layer in between. I've never been entirely
> comfortable with that philosophy, although I'm not sure I know why
> I'm not. I'd still like to know the results of having our own software
> managed MMU and so decoupling the two address spaces completely --
> even though, as Jeremy has accurately pointed out several times in the
> past, that brings its own set of difficulties.
>
> I suppose my underlying inclination is to simulate as much stuff as
> possible and thereby minimise the extent to which we are exposed to
> the vagaries of the host OS. I think that's why the thin-layer idea
> is a bit irksome. Perhaps it's a way of thinking about OS-level
> portability: either simulate a lot of stuff yourself and minimise
> the "passed-through" requirements, or have a seperate thin-layer
> arrangement for each supported OS. I lean towards the former scheme,
> but I don't know if it's really viable -- whereas at least Jeremy
> has demonstrated that his scheme works well at least for Linux.
OK, my assumptions are:
1. No cross-system emulation, so
2. Our clients are always native apps which think they're running
on the underlying OS
I think these are important, because Valgrind is a debugging tool, and
should therefore try as hard as possible to not change the behaviour of
the target program. Ideally, it will run identically with and without
Valgrind - you'll just get more messages with.
The two models are, as I see it:
Thin:
Client application
- - - Valgrind interposer layer - - -
kernel
Thick:
Client application
- - - - Valgrind client interface - - -
Generic Valgrind innards
- - - - Valgrind kernel interface - - -
kernel
In the thin model, we have a chunk of code which takes application
requests, massage them as little as necessary, and (often) pass them
through to the kernel; and conversely, take kernel events, massage them
a little and pass them to the app. The intent is to expose the raw
kernel behaviour to the application as much as possible, because that's
what it expects to see. And in many cases, since Valgrind doesn't
particularly care about most kernel transactions (at worst, ignoring a
transaction will fail to properly instrument a syscall, rather than
causing a functional difference), it really is just passed straight
through.
In the thick model, we have a nice chunk of generic code, which doesn't
depend much on OS or CPU services (like address space translation),
because it does it all for itself. But to use this layer, we need two
interface layers: one on the top to make the generic stuff look like the
kernel to the application, and one on the bottom to make the generic
stuff look like the application to the kernel. My concern is that the
combined complexity of these two layers, in addition to the generic
code, will far exceed the complexity of the thin model. We might be
able to simplify things a bit by making the top layer fixed (eg, we
emulate Linux 2.6.7 now and forever more), but that's contradicting the
"no emulation" assumption.
Its interesting to note that this is very similar to the design space on
the CPU emulation side, with full reassembly vs copy-annotate (but I
wouldn't get carried away by the similarity - I don't think the
tradeoffs are at all similar).
J
|
|
From: Jeremy F. <je...@go...> - 2004-08-10 20:14:15
|
On Tue, 2004-08-10 at 15:00 -0500, Bob Friesenhahn wrote: > Except for popular environments like Java and .net which don't offer a > reasonable Possibly so, though I thought Java fixed this a while ago. Anyway, their runtimes are probably doing their own thread multiplexing, which makes them a distinct Valgrind target (if that even makes sense). J |
|
From: Jeremy F. <je...@go...> - 2004-08-10 20:09:40
|
On Tue, 2004-08-10 at 15:48 +1200, John Carter wrote: > Often the most effective refactoring one can do is to hop the fence and do > the things right on the other side. ie. You have the source of the Linux > kernel, it is mutable fix it there. > > (Major Hint 1 that this is effective: You are emulating something which > you wouldn't have to if you could fix the thing you're emulating.) > > Even if you destroy compatibility between the next version of Valgrind and > the current stable kernel, the gains (on both sides) can be enormous. > > In a system we have been working on we ended up deleting many hundreds of > lines of code on both sides of the fence _and_ adding functionality once > we could fix things on both sides, > > Under Linux you also have an additional weapon of loadable kernel modules. Hm, yes, but we'd like to get fewer kernel dependencies rather than more. If we fix Linux, we still can't fix all the other potential porting targets. And it would mean that porting Valgrind would require significant kernel changes. Even in Linux, it isn't enough for us to make the change to the kernel; it's only useful if it makes it into the mainline and becomes standard. The more general problem is that the "bug" I'm talking about is really ingrained in the design of Unix. I'm not sure there's any simple, acceptable (ie, will be merged into mainline) change which will solve it. In effect, we need to add a new argument (a signal mask) to *every* syscall which might block. The simple approach (just do it, add an arg to everything) is clearly impractical, but it isn't clear what you'd do instead, and how you'd implement it internally. J |
|
From: Bob F. <bfr...@si...> - 2004-08-10 20:00:26
|
On Tue, 10 Aug 2004, Jeremy Fitzhardinge wrote: > Solaris dropped the multi-level threading, and went to plain 1:1 kernel > user mapping. It got too complex, and it rarely helps with real > threaded programs - very few programs have massively more threads than > CPUs, because such programs don't scale well. Except for popular environments like Java and .net which don't offer a reasonable alternative. Bob ====================================== Bob Friesenhahn bfr...@si... http://www.simplesystems.org/users/bfriesen |
|
From: Jeremy F. <je...@go...> - 2004-08-10 19:53:07
|
On Mon, 2004-08-09 at 14:19 -0500, Bob Friesenhahn wrote: > If Linux eventually wakes up and adopts a multi-level threading model > (combination of user-level and on-demand kernel-level threads "LWP"s) > as used by Solaris, AIX, and Digital Unix, this approach is surely > doomed to fail. Solaris dropped the multi-level threading, and went to plain 1:1 kernel user mapping. It got too complex, and it rarely helps with real threaded programs - very few programs have massively more threads than CPUs, because such programs don't scale well. > Also, if someone wants to port Valgrind to a Unix OS > that doesn't support clone() (only Linux has clone()), it seems doomed > to fail. Yes, that's a problem. We'll need to emulate whatever kernel mechanism the OS threads library uses, and intercept any library-internal context switch. We would also need to handle any preemption mechanisms the kernel/threads library uses. N:M threading models are definitely going to be much more complex to handle, and we might want to do more interception at the pthreads level in those cases. > If Linux is to ever become formidable in the multi-threading world > (which it must!) then it will need to eliminate the requirement for > the kernel to know about each and every thread. In Linux's case, that's unlikely to ever happen. The N:M threading model has not been an overwhelming success. > The latest trend is for CPUs to incorporate multiple execution units, > so even systems delivered with one CPU chip will benefit > performance-wise from multi-threaded programs. Yep. J |
|
From: Julian S. <js...@ac...> - 2004-08-10 13:09:28
|
> > The syscall handing stuff is completely logically distinct from FV, and > > it's complex. But we already talked about that. > > Sure. Perhaps my point is this: worse-is-better is great for getting > things going, but it's not a long-term solution. For example, none of the > big Unixes are really worse-is-better any more (there are some > worse-is-better aspects in APIs, but implementation-wise not so much). I'm kind-of bemused by this use of worse-is-better. All I meant was that to get to a better place we may have to go through a worse intermediate place. I didn't mean that in the end we should settle for something worse than what we have already. J |
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 12:58:38
|
On Tue, 10 Aug 2004, Tom Hughes wrote:
> Actually, it seems that C99 can do it, and possibly GCC did even
> before that. The syntax is:
>
> const char** a[] = {
> [0] = (const char *[]){ "one", "two", "three", NULL },
> [1] = (const char *[]){ "four", "five", "six", NULL }
> };
>
> And "cc -std=c99 -pedantic -Wall" seems to compile that with no
> complaints as does gcc in non-pedantic c89 mode. Pedantic C89 mode
> refuses it.
Great, got it working. Thanks, Tom.
N
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 12:57:57
|
Hi, I've got the basics working for Memcheck to check whether syscall args are uninitialised. This strikes me as a good thing. Here's an example error: ==26325== Syscall param write(fd) contains uninitialised or unaddressable byte(s) ==26325== at 0x420D2473: write (in /lib/tls/libc-2.3.2.so) ==26325== by 0x42015703: __libc_start_main (in /lib/tls/libc-2.3.2.so) ==26325== by 0x8048298: ??? (start.S:81) ==26325== Address 0xFFFFFFFF is not stack'd, malloc'd or (recently) free'd I've piggybacked it onto the "Param" error kind, which explains the strange address. I think I will change Memcheck so that if the address passed into a Param error is (void*)-1, then it won't display the address part of the error. (I could also change the error message to omit the "or unaddressable" part in this case.) ---- I also propose to slightly change the current Param error output so as to distinguish between errors in the syscall arguments, and errors in the memory pointed to by syscall arguments. Eg, where we currently say: Syscall param write(buf) contains uninitialised... to mean that the memory pointed to by buf is bad, I would change this to mean that buf itself is bad. I would use Syscall param write(*buf) contains uninitialised... to indicate that the memory pointed to by buf is bad. (Nb: if buf is bad, there's a good chance that *buf is also bad.) The downside of this is that it would break some existing Param suppressions, and possibly affect tools post-processing Valgrind's output. I guess I could use something like "write(&buf)" for the new errors, but that's confusing. Maybe this non-compatibility is not a big problem? Does anyone else have any other ideas? ---- Adding in all the syscall arg names will be tedious... is there anywhere in the Linux source code where these are shown? AFAICT, that information is completely spread throughout the source tree. If not, I guess I'll be trawling through man pages, hoping that the glibc arg names match up with the kernel ones... ---- Similarly, I plan to check client request args. I'm not yet sure if I'll piggyback those errors onto the Param error kind, or create a new one. The new one would be extremely similar, though. N |
|
From: Tom H. <th...@cy...> - 2004-08-10 11:19:38
|
In message <20040810111535.GA7867@ugly.local>
Oswald Buddenhagen <os...@kd...> wrote:
> On Tue, Aug 10, 2004 at 11:52:03AM +0100, Nicholas Nethercote wrote:
>> I want to initialise a static 2-d array in C. Is this possible?
>>
>> GCC complains about this:
>>
>> const char** a[] = {
>> [0] = { "one", "two", "three", NULL },
>> [1] = { "four", "five", "six", NULL },
>> };
>>
> just leave out the "[<idx>] =" parts and everything will be fine.
No it won't, that was the first thing I tried. You still need to fix
the inner dimension or use a compound literal.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Oswald B. <os...@kd...> - 2004-08-10 11:15:39
|
On Tue, Aug 10, 2004 at 11:52:03AM +0100, Nicholas Nethercote wrote:
> I want to initialise a static 2-d array in C. Is this possible?
>
> GCC complains about this:
>
> const char** a[] = {
> [0] = { "one", "two", "three", NULL },
> [1] = { "four", "five", "six", NULL },
> };
>
just leave out the "[<idx>] =" parts and everything will be fine.
--
Hi! I'm a .signature virus! Copy me into your ~/.signature, please!
--
Chaos, panic, and disorder - my work here is done.
|
|
From: Tom H. <th...@cy...> - 2004-08-10 11:14:02
|
In message <Pin...@he...>
Nicholas Nethercote <nj...@ca...> wrote:
> I want to initialise a static 2-d array in C. Is this possible?
>
> GCC complains about this:
>
> const char** a[] = {
> [0] = { "one", "two", "three", NULL },
> [1] = { "four", "five", "six", NULL },
> };
>
> saying:
>
> b.c:7: warning: braces around scalar initializer
> b.c:7: warning: (near initialization for `a[0]')
Because a[0] is a pointer - a is an array of pointers to pointers - and
you can't initialise a pointer with an aggregate. C won't just invent
an unnamed array and then take a pointer to it.
If the inner dimension of the array is fixed then this will work:
const char* a[][4] = {
[0] = { "one", "two", "three", NULL },
[1] = { "four", "five", "six", NULL }
};
Because a is now an array of arrays of pointers so you can initialise
the [0] member with an array.
<fx: goes and reads info gcc>
Actually, it seems that C99 can do it, and possibly GCC did even
before that. The syntax is:
const char** a[] = {
[0] = (const char *[]){ "one", "two", "three", NULL },
[1] = (const char *[]){ "four", "five", "six", NULL }
};
And "cc -std=c99 -pedantic -Wall" seems to compile that with no
complaints as does gcc in non-pedantic c89 mode. Pedantic C89 mode
refuses it.
> If I break it into two pieces:
>
> const char* a1[] = { "one", "two", "three", NULL };
> const char* a2[] = { "four", "five", "six", NULL };
>
> const char** a[] = {
> [0] = a1,
> [1] = a2,
> };
>
> it works ok, but that doesn't really suit my purpose, because there's
> a macro involved and it would be much more convenient to do the former.
That is the other option.
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 10:52:10
|
Hi,
I want to initialise a static 2-d array in C. Is this possible?
GCC complains about this:
const char** a[] = {
[0] = { "one", "two", "three", NULL },
[1] = { "four", "five", "six", NULL },
};
saying:
b.c:7: warning: braces around scalar initializer
b.c:7: warning: (near initialization for `a[0]')
And it initialises a[0] to "one" and a[1] to "four".
If I break it into two pieces:
const char* a1[] = { "one", "two", "three", NULL };
const char* a2[] = { "four", "five", "six", NULL };
const char** a[] = {
[0] = a1,
[1] = a2,
};
it works ok, but that doesn't really suit my purpose, because there's a
macro involved and it would be much more convenient to do the former.
Any ideas, or is this just impossible in C? Thanks.
[In case you're wondering: I'm looking at adding uninitialised checking
of all syscall args to Memcheck, and so I want to expand the sys_info type
to include an array of arg names for all syscalls. So the sys_info
initialisation in vg_syscalls.c would look like:
SYSB_(write, MayBlock, { "fd", "buf", "count", NULL } ),
which expands to:
[__NR_write] = { MayBlock, before_write, NULL,
{ "fd", "buf", "count", NULL} }
The arg names are necessary for producing good error messages. This could
be done relatively easily non-statically, or by breaking each syscall's
SYSB_ macro into two parts, but it would be better to keep it static and
unbroken if possible.]
N
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 10:25:15
|
On Tue, 10 Aug 2004, Jeremy Fitzhardinge wrote: >> I feel that a lot of the bug fixes recently have been becoming more >> obscure. Eg. we've hit a couple of kernel bugs. Getting everything 100% >> right requires a lot more code than getting everything 98% right. Hmm. > > Yes, but they're mostly to do with the proxyLWP syscall handling rather > than FV. FV doesn't seem very complex to me (but I'm not really the > person to ask), and I threw out a ton of code dealing with the > LD_PRELOAD stuff. > > The syscall handing stuff is completely logically distinct from FV, and > it's complex. But we already talked about that. Sure. Perhaps my point is this: worse-is-better is great for getting things going, but it's not a long-term solution. For example, none of the big Unixes are really worse-is-better any more (there are some worse-is-better aspects in APIs, but implementation-wise not so much). I've strayed a long way from the original topic now, but I'm glad to see this discussion happening. N |
|
From: Jeremy F. <je...@go...> - 2004-08-10 10:11:13
|
On Tue, 2004-08-10 at 09:58 +0100, Nicholas Nethercote wrote: > The point I was trying (badly) to make here was more along the lines of > examining complexity trade-offs. Ie. LD_PRELOAD was the 90% > "worse-is-better" solution. FV is "the right thing", or at least closer > to "the right thing" than LD_PRELOAD. > > I feel that a lot of the bug fixes recently have been becoming more > obscure. Eg. we've hit a couple of kernel bugs. Getting everything 100% > right requires a lot more code than getting everything 98% right. Hmm. Yes, but they're mostly to do with the proxyLWP syscall handling rather than FV. FV doesn't seem very complex to me (but I'm not really the person to ask), and I threw out a ton of code dealing with the LD_PRELOAD stuff. The syscall handing stuff is completely logically distinct from FV, and it's complex. But we already talked about that. J |
|
From: Tom H. <th...@cy...> - 2004-08-10 09:11:21
|
In message <410...@lb...>
Iwona Sakrejda <isa...@lb...> wrote:
> I installed Valgrind 2.1.1 on our cluster and I had to rebuild it
> couple times because a guy I built it for, kept asking me to increase
> VG_N_RWLOCKS. At some point he suggested that it should be
> an environmental variable for a user to set.
Except the whole point is that it sizes the data structure statically
at compile time ;-)
I've got a patch somewhere that tried to remove the limit in the same
way that was used for semaphore but I can't remember how well it works.
> I tried to be a good citizen and following a link from
> http://valgrind.kde.org/bugs.html I went to the Bugzilla page and got
> myself an account although it looked like a KDE Bugzilla.
> So then I tried to file a suggestion, but one of the first questions
> was "Which version of KDE do you use". I don't. There was nothing
> about valgrind that I could find....
If you follow the link from http://valgrind.kde.org/bugs.html you will
there is a custom bug submission page for valgrind. The URL is:
http://bugs.kde.org/enter_valgrind_bug.cgi
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 08:58:50
|
On Mon, 9 Aug 2004, Julian Seward wrote: >> I've been thinking about this a bit w.r.t. full virtualisation. The >> benefits of FV over the LD_PRELOAD approach are: >> >> 1. Valgrind gains control at absolute startup >> 2. Valgrind can use the standard libraries itself >> 3. Valgrind can run static binaries >> 4. Valgrind is nicely separated from the client >> >> All these are good, but they don't to me seem fantastic -- none of them >> were really huge problems previously. And FV has introduced its own >> difficulties, mostly from the rigidity of the memory layout, which we're >> still battling. Plus it results in more code. > > I agree that 2,3,4 are not a big deal, but 1 is. The threading stuff > was getting nasty as a lack of 1 -- having the threading machinery > start up in an unknown state is not good, so FV is a big win there. > > In the longer term 2 and 4 are useful too. For example, if we wanted > to use C++ in V in a big way, 2 would be more or less mandatory and > 4 would be very helpful too. The point I was trying (badly) to make here was more along the lines of examining complexity trade-offs. Ie. LD_PRELOAD was the 90% "worse-is-better" solution. FV is "the right thing", or at least closer to "the right thing" than LD_PRELOAD. I feel that a lot of the bug fixes recently have been becoming more obscure. Eg. we've hit a couple of kernel bugs. Getting everything 100% right requires a lot more code than getting everything 98% right. Hmm. > I suppose my underlying inclination is to simulate as much stuff as > possible and thereby minimise the extent to which we are exposed to > the vagaries of the host OS. I think that's why the thin-layer idea > is a bit irksome. Perhaps it's a way of thinking about OS-level > portability: either simulate a lot of stuff yourself and minimise > the "passed-through" requirements, or have a seperate thin-layer > arrangement for each supported OS. I lean towards the former scheme, > but I don't know if it's really viable -- whereas at least Jeremy > has demonstrated that his scheme works well at least for Linux. Yes, all this is just talk until someone spends a large chunk of time actually implementing something. N |
|
From: Nicholas N. <nj...@ca...> - 2004-08-10 08:49:27
|
On Mon, 9 Aug 2004, Adun R. wrote: >> Note that since 2.1.1, the LD_PRELOAD method is no longer used. > > In your later post to this list you said that there is a FV method. > This is the 2.1.2 doc, and the LD_PRELOAD is still mentioned there - > http://developer.kde.org/~sewardj/docs-2.1.2/coregrind_core.html#howworks > Please refer me to another source, thank you. Yes, this is out of date. Perhaps that section should just be removed altogether, because it's just misleading. The tech docs should have a big sign on them saying they're out of date too. As for a description of how the new version works, I'm working on my dissertation now which does cover this, see the text below. Please excuse the mark-up. As always, the source code is the best reference :) >> My opinion on Windows is that it would be great, if it could be done in a >> way that doesn't result in Valgrind's code looking like a train crash. > I would like to know how this stands against Valgrind 20031012-wine; do > you think there's still a use for this idea? I have no idea, I really don't understand that stuff at all. N \subsection{Starting Up} The following ingredients are used at start-up: \begin{itemize} \item Valgrind's loader (a statically-linked executable, called \code{valgrind}); \item Valgrind's core (a dynamically-linked executable, called \stagetwo); \item the tool plug-in (a shared object); \item the client program (an ELF executable, or a script). \end{itemize} %% The first step is to get the last three parts loaded into a single process, sharing the same address space. The loader is not present in the final layout; it is required because \stagetwo must end up at a high address, an address at which the operating system would not put it. The loader is loaded and executed normally by the operating system. The loader's single task is to load \stagetwo at a high address and execute it. Then \stagetwo does a preliminary parse of the command line options to see which tool has been chosen with the \code{--tool} option. It finds the shared object for the chosen tool and uses \code{dlopen()} to load the tool plug-in and any dynamic libraries the tool plug-in uses. Then \stagetwo loads the client executable (overwriting the no-longer-needed loader in the process). If the client program is a script, the script's interpreter is loaded. Both the loader and \stagetwo judiciously use empty memory mappings along the way to ensure the different pieces end up in the desired locations. The typical resulting memory layout is as follows. \begin{itemize} \item \hex{0xc0000000}--\hex{0xffffffff}: The top 1GB is reserved for the kernel on typical x86/Linux systems. \item \hex{0xb0000000}--\hex{0xbfffffff}: The next 256MB are used for \stagetwo and the tool plug-in, any libraries (shared objects) and mappings used by them, and their stack. (They do not need a heap, as all dynamic allocations are done using mappings.) \item \hex{0x00000000}--\hex{0xafffffff}: The lower part of the bottom 2.75GB is used for the client itself. The remainder is used for shadow memory required by the tool; the amount needed, if any, depends on the tool. (Shadow memory is discussed further in Section~\ref{Crucial Features}). \end{itemize} %% The chosen boundaries are selected with Valgrind build-time options and can be changed to suit less typical systems. Small (1MB) \emph{red-zones}---areas thatshould not be touched---separate each section. Once everything is in place, ``normal'' execution begins. The core (i.e.~\stagetwo) processes any command line arguments intended for it. Tool-specific arguments are possible; the core passes any arguments it does not recognise to the tool plug-in. The core initialises itself, and tells the tool to perform any initialisation it needs. Once all this is complete, the Valgrind tool is in complete control, and everything is in place to begin translating and executing the client from its first instruction. (In an earlier implementation, the core used the \LDPRELOAD environment variable to graft itself and the tool plug-in to the client process, and then ``hijacked'' execution. This approach was simpler but had four disadvantages. First, it did not work with statically linked executables. Second, the core did not gain control over the executable until some of the dynamic linker's code had run. Third, the client and Valgrind tool were not well separated in memory, so an erroneous client could wildly overwrite the tools code or data. Fourth, the core and tool plug-ins could not use any libraries also used by the client, including \glibc, and so had to use a private implementation of common library functions.) |
|
From: Jeremy F. <je...@go...> - 2004-08-10 07:46:18
|
On Mon, 2004-08-09 at 15:11 +0100, Nicholas Nethercote wrote: > Maybe "duplicates" is not quite the right word... a lot of functions are > called both from vg_proxylwp.c and vg_scheduler.c. A couple of examples: > > - VG_(do_signal_routing)() > - VG_(deliver_signal)() > - VG_(is_sig_ign)() > - VG_(proxy_create)() > - VG_(proxy_set_sigmask)() > > etc. I'm not passing judgment on these particular examples, they're just > ones I saw. > > I guess what I was trying to say: there seem to be a number of actions > whereby vg_scheduler.c or vg_signals.c handles the non-blocking-syscall > case, and vg_proxylwp.c handles the blocking syscall case, and the two > cases are similar but not identical. do_syscall/do_pthread_syscall is one > example. > > ---- > > > One of my design goals was to use as much of the kernel machinery as > > possible so we don't have to emulate it - that's what the proxyLWPs are > > for, but it wasn't as successful as I'd hoped for. If we could work out > > a way to get all the other behaviours we want, running the app threads > > in their own kernel threads would probably simplify a lot of this > > because we could just use the kernel directly, rather than having to > > emulate it quite so much. > > > > [...] > > > > Well, I think a big obvious first step is to drop all the pthreads > > support from the core, which will simplify vg_scheduler a fair bit. The > > scheduler doesn't really need to know much about a thread's state other > > than its running or blocked on some event (which is probably always > > syscall completion of some kind). > > > > This means that to do threading, all OS ports must do enough to support > > the native threads library. The downside is that this could result in > > as much complexity as dropping core ptheads support removes, and it > > would be duplicated for each OS port. But it does mean that we don't > > have to carry libpthread around, which is a large and increasing burden > > (and that's just for all the x86-linux variants). > > Are these two points (above and below the "[...]") the same? The two points are orthogonal. The first is about internal implementation, and the second is about the interface we present. The first point is about whether we use kernel thread contexts directly for client thread contexts. Ie, do we do our own thread switching, or let the kernel do it? The second is whether we present threading to the clients in terms of pthreads, or the kernel's underlying thread abstraction (clone, in Linux's case). > > Ok, maybe this is the direction we should be heading in, then. So, what's > preventing us from doing so? AFAIK: > > - We can't lose control of scheduling totally, because V turns atomic > instructions into non-atomic ones. We don't want a thread switch after > a shadow memory word is written, but before the word itself is written. Well, the only way to do this is either by putting locks around all the critical sections, or by using lock-free algorithms. The first is expensive, and the second is v. complex in general. I've been thinking about some scheme for doing adaptive locking, where only memory accesses which end up touching memory shared between threads get locking, but most unshared memory accesses are unlocked. It's tricky, and not necessarily a performance win. > - We need a way of identifying when certain interesting thread/scheduling > operations occur, such as mutex locks, thread switches, etc, to support > Helgrind, Calltree, and future tools. I think we can do this by hooking into library function calls and observing the client behaviour. For example, if we see the client call pthread_mutex_lock, and then that thread ends up blocking in sys_futex before returning, we can deduce that it blocked in a lock. It could get fragile though. Or we could maintain a parallel model of the libpthread state, but that also seems fragility-prone. J |
|
From: Tom H. <th...@cy...> - 2004-08-10 06:10:32
|
In message <109...@se...>
Bryan O'Sullivan <bo...@se...> wrote:
> On Mon, 2004-08-09 at 23:59 +0100, Tom Hughes wrote:
>
> > My understanding is that this was looked at quite carefully during
> > the NPTL development. In fact the original plan was to go to a multi
> > level model but from what I've read experiment showed that the kernel
> > could be made to scale to vast numbers of thread/process without any
> > loss of performance.
>
> I think you're thinking of NGPT, IBM's parallel effort to replace
> LinuxThreads, which ended up being abandoned.
The paper I was thinking of is about NPTL but some of the experience
it talks about did come from NGPT I think. The paper is at:
http://people.redhat.com/drepper/nptl-design.pdf
Tom
--
Tom Hughes (th...@cy...)
Software Engineer, Cyberscience Corporation
http://www.cyberscience.com/
|
|
From: Adun R. <adu...@ml...> - 2004-08-10 04:45:05
|
> Note that since 2.1.1, the LD_PRELOAD method is no longer used. In your later post to this list you said that there is a FV method. This is the 2.1.2 doc, and the LD_PRELOAD is still mentioned there - http://developer.kde.org/~sewardj/docs-2.1.2/coregrind_core.html#howworks Please refer me to another source, thank you. > My opinion on Windows is that it would be great, if it could be done in a > way that doesn't result in Valgrind's code looking like a train crash. I would like to know how this stands against Valgrind 20031012-wine; do you think there's still a use for this idea? I believe the valgrind will look more of a debugger under Windows. For example, we can create the client process, and have it's main thread suspended. Regards, Rauch Adun. -- Adun R. adu...@ml... |
|
From: John C. <joh...@ta...> - 2004-08-10 03:48:24
|
On Sat, 7 Aug 2004, Jeremy Fitzhardinge wrote: > Yup. Valgrind is in the awkward position of having to track the > kernel's model of signals, threads and system calls. This is just all > very complex. We've made the job more complex for ourselves by > insisting that all the user code runs in a single threads, since that > introduces the extra complexity of the proxyLWP stuff. > > One of my design goals was to use as much of the kernel machinery as > possible so we don't have to emulate it - that's what the proxyLWPs are > for, but it wasn't as successful as I'd hoped for. >snip< > I would love to get rid of it, but it's basically a workaround for a bug > in the Unix API. There's just no way to atomically unblock some signals > and run a syscall. Often the most effective refactoring one can do is to hop the fence and do the things right on the other side. ie. You have the source of the Linux kernel, it is mutable fix it there. (Major Hint 1 that this is effective: You are emulating something which you wouldn't have to if you could fix the thing you're emulating.) Even if you destroy compatibility between the next version of Valgrind and the current stable kernel, the gains (on both sides) can be enormous. In a system we have been working on we ended up deleting many hundreds of lines of code on both sides of the fence _and_ adding functionality once we could fix things on both sides, Under Linux you also have an additional weapon of loadable kernel modules. John Carter Phone : (64)(3) 358 6639 Tait Electronics Fax : (64)(3) 359 4632 PO Box 1645 Christchurch Email : joh...@ta... New Zealand The universe is absolutely plastered with the dashed lines exactly one space long. |
|
From: <js...@ac...> - 2004-08-10 02:58:26
|
Nightly build on nemesis ( SuSE 9.1 ) started at 2004-08-10 03:50:01 BST Checking out source tree ... done Configuring ... done Building ... done Running regression tests ... done Last 20 lines of log.verbose follow sem: valgrind ./sem semlimit: valgrind ./semlimit sha1_test: valgrind ./sha1_test shortpush: valgrind ./shortpush shorts: valgrind ./shorts smc1: valgrind ./smc1 susphello: valgrind ./susphello syscall-restart1: valgrind ./syscall-restart1 syscall-restart2: valgrind ./syscall-restart2 system: valgrind ./system yield: valgrind ./yield -- Finished tests in none/tests ---------------------------------------- == 170 tests, 4 stderr failures, 0 stdout failures ================= corecheck/tests/as_mmap (stderr) corecheck/tests/fdleak_fcntl (stderr) memcheck/tests/writev (stderr) memcheck/tests/zeropage (stderr) make: *** [regtest] Error 1 |