You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
|
|
|
1
(9) |
2
(16) |
|
3
(9) |
4
(8) |
5
(9) |
6
(10) |
7
(14) |
8
(10) |
9
(7) |
|
10
(14) |
11
(19) |
12
(22) |
13
(18) |
14
(20) |
15
(10) |
16
(12) |
|
17
(13) |
18
(7) |
19
(12) |
20
(13) |
21
(9) |
22
(12) |
23
(6) |
|
24
(5) |
25
(5) |
26
(6) |
27
(7) |
28
(9) |
29
(13) |
30
(21) |
|
From: Nicholas N. <nj...@cs...> - 2006-09-02 22:44:09
|
On Sat, 2 Sep 2006, Bryan Meredith wrote: > I had an idea while writing this reply (6 odd hours ago) and as a > result, I have actually solved it, tested it on both boxes (in 32bit and > 64bit mode on my Athlon64 box) and am just tidying up a couple of bits > and bobs before posting Beta05! :D Nice :) You've discovered how annoying and confusing real programs can be at the binary level. Coming up with robust ways to do things in Valgrind tools is often a real challenge. The best case I can think of is function wrapping, which sound easy in theory but in practice took Julian and I about 5 abortive attempts before Julian worked out a satisfactory approach. Nick |
|
From: Bryan M. <om...@br...> - 2006-09-02 21:42:22
|
Fellow Valgrinders, please see http://www.brainmurders.eclipse.co.uk/omega.html for a complete overview of what this tool can do for you! (We use this heavily at work - feel free to give it a spin...) >From the web page: ================== Omega addresses what I perceive to be one of the few shortfalls of the excellent Valgrind Memcheck Tool - where Memcheck reports the location that a leaked block was allocated, Omega also shows where it was leaked. New in Beta5: ============= Aggregates the circular references together into common contexts, giving a repeat count and leak amount for that context and makes display optional. Broke the debugger attach :( On the positive side, it gives much more accurate leak reports in some cases, having finally closed off some edge cases with functions that return values. A reported bug with realloc() on x86 (not x86-64) was also fixed (thanks for the report). Exposed some internal sizing information for users with HUGE programs to tweak for potentially faster execution. The next patch is likely to be a release candidate unless you find me some juicy bugs to chomp on. Known Issues: ============= I have a wrapper around main() to detect when to stop tracking - if you are using threads and they don't exit before main() does, there will be problems. I haven't particularly tested this with threads yet - that's targeted for the later on. It can also affect the stack trace slightly. Requested Features? =================== I am still toying with the idea of allowing targeted tracking through the use of the suppression system - exclusively report on a specified malloc() call. show hanging pointers at a targeted call to free(). turn on more verbose output for a given memory block. Generate leak reports in some XML format to make machine parsing simple. If you use Omega and think any of these features would be useful or have requests of your own, please let me know. As ever, I would welcome your comments, bug reports and especially any news of success stories. Please share them with us on the list and copy me in so I don't miss them. thanks and happy hunting, Bryan "Brain Murders" Meredith |
|
From: Bryan M. <om...@br...> - 2006-09-02 20:49:20
|
Julian,
I had an idea while writing this reply (6 odd hours ago) and as a
result, I have actually solved it, tested it on both boxes (in 32bit and
64bit mode on my Athlon64 box) and am just tidying up a couple of bits
and bobs before posting Beta05! :D
for the record:
Problem:
The accumulator contains the final live pointer to a memory block on
return from a function but should actually die at that point as the
function does not in fact return a value (a 'C' void type function).
Eventual overwrite of the value in the accumulator produces a stack
trace far from the actual point where the leak should have been reported.
Solution:
On function exit, note that the accumulator is the only live pointer to
that memory block and store a stack trace that references the end of the
function. When the accumulator is overwritten, note that it matches the
last live reference we stored earlier then use the stack trace that was
stored earlier in the leak report(s).
If another reference is added to the block (it was a return value after
all), clear the stored information so that any future leak can be
reported normally.
In the case of nested calls, we do not overwrite the cached stack trace
because we confirm that the accumulator is still the only live reference
so leave it pointing to the original "actual leak" position.
I have left some of the comments in below as I think it useful to have
them on record, should anyone come along asking questions about stuff
like optimised code or support or more usefully, what the test cases are
looking for and why.
Bryan
Julian Seward wrote:
> I can see that some info like that would be helpful, but I'm concerned
> that it might be a case of fixing the symptoms (and/or adding a fix
> which makes sense only for eg x86 and not generally).
I agree - that was the main reason for checking before going ahead.
Whilst it would partially solve my problems, there is undetermined
impact in other areas.
>
> What I think would be helpful is to make a concise, semi-formal
> statement of the problem. I think this would help by showing how this
> relates to standard compiler concepts of liveness and reachability.
> In particular I'd like to find a solution which gives accurate results
> and which works well even in the presence of optimised code, which AIUI
> Omega currently doesn't.
The main problem with optimised code is generating any sort of useful
information to clue the user into just where things are happening.
Tracking is not the problem (although there are some issues that I would
like to clean up when the un-optimised case is correct), its the lack of
correspondence to the original source code flow. You know what it's like
trying to debug optimised code when you single step and the debugger
does anything but on the source display, leaping up and down all over
the place. Function in-lining and tail calls are also a severe problem
as it makes function scope almost impossible to determine.
>
> My initial thoughts at such a statement are:
>
> - keep track of all allocated blocks
>
> - keep track of all potential roots. Potential roots at any
> time are
>
> * the words in all currently accessible memory, plus the words
> in the registers
> BUT
> only those for which the next event is a read
Omega handles a specialised case of this in order to cope with a common
program idiom. From the web page:
----------------------------------------------------
Lets say you have something along the lines of this:
1 secret *foo = (secret *)malloc(sizeof(bar) + sizeof(secret) +
alignment_correction);
2 foo->secret_stuff = magic_key;
3 etc.
4 foo++;
5 return (bar*)foo;
Internally, Omega uses shadow blocks to track references within an
allocated block. Thus, when you increase "foo" on line 4, Omega creates
a shadow block that ties back to the main block allocated at line 1
instead of raising a leak report.
----------------------------------------------------
Without this special case handling, Omega produces meaningless results
in the presence of any custom allocation.
>
> - If some part of the address space disappears, that can be thought of
> as a write.
>
> - If a write (to a potential root) happens, and that destroys the last
> pointer to a particular block, then we can complain at that point
> of a leak.
>
> - However, that may be too late. (as per current example)
>
> What we really want to find is something along the lines of
> 'dead writes' (to registers and memory). A dead write puts a value
> into that storage location which is never read, only overwritten
> (or the storage location goes out of scope, which is equivalent).
>
> Perhaps it should be that a leak is reported at a dead write of
> the last pointer to a block. Or something. Anyway, the general
> idea is: if you can characterise the problem in terms of dataflow
> and liveness, perhaps it becomes possible to build an implementation
> which is robust to ABI changes, compiler optimisation, and across
> different platforms.
>
> The above ideas are half-baked; don't take the details too literally.
They are actually pretty much all of it...
>
> ---
>
> Another thing that would help (perhaps the same thing, really)
> is to give a very precise specification of what the tool is intended
> to do.
>
> Originally I had the impression that it was "find where the last pointer
> to block X is overwritten"; but when looked at under a magnifying glass
> it's clear this can lead to error reports which point to some location
> in the code flow which may be arbitrarily far after the place where you
> really wanted to report the error. So (and I think this is the crux
> of the problem) how do you precisely specify those program points where
> you *do* want to report an error?
>
> J
>
When a block is allocated (through malloc(), realloc() etc), a pointer
is returned to the calling program. Omega tracks this pointer and any
copies that are created and destroyed until such time that the block is
free()ed (through free() or realloc() with a bigger block size) or the
last pointer is lost, either by being overwritten or the containing
storage going out of scope or "dying".
In the Omega source are a number of small test cases that demonstrate
the classes of situation where Omega should report a useful message for
the user. They fall into 3 main categories:
Block
Overwrite
Scope
Block
-----
These tests (and functionality) deal with pointers stored in other heap
allocated blocks ie. the pointer to allocated block B is stored within
allocated block A. The lifetime of the pointer to block B (assuming it
is never overwritten) is determined by the lifetime of block A. If block
A is returned to the OS or leaks, the pointer to block B dies at this
point. If this is the last pointer to block B, then block B leaks and so
on. This functionality is quite simple as it involves death of storage
and little else. Typically, if a leak occurs it will be within the
free() function because some malloc()ed structure has pointers to other
malloc()ed structures and they weren't cleaned up properly before the
base structure was free()ed. Singly linked lists can generate long
chains of leaks if you overwrite the head pointer.
Overwrite
---------
These tests deal with the destruction of the pointer itself, rather than
the lifetime of the storage where the pointer resides. The simplest case
is overwriting a pointer (in memory or a register) with another value
(ie. NULL). If the last pointer to a block is overwritten then a leak of
that block occurs at that point. There is a specialisation of this
namely the shadow block stuff mentioned inline above.
Scope
-----
These tests deal with the death of storage (the Block section above is a
specific form of this that deals with heap allocated storage). The
non-heap storage is either stack or register based. Automatic variables
live on the stack and as such, when the function exits and the stack
pointer is moved, their storage dies. Any pointers within automatic
variables are therefore lost when the stack unwinds generating a leak
report on the last line of the function. Seeing a report here is a sure
sign that your automatic variable went out of scope holding a pointer
that you didn't stash somewhere else or free().
(**next bit written just as the brain-wave hit**)
The other part of scope, and this is the bit that is currently standing
in the way of a full blown release candidate, is the scope of registers.
The ABI for each architecture (I can only cover x86 and x86_64 here -
whilst I don't actually know x86 assembler, I have never played with
Power Architecture assembler in my life) gives a preferred use or
purpose for each of the registers and also details function entry and
exit along with parameter passing in each direction. So, as a function
returns and the stack unwinds, any register that is described in the ABI
as belonging to the called function should now be classed as dead. This
is fine and works well apart from a single fly in the ointment - the
accumulator can optionally be used to return a value of some description
to the calling function. The problem is that there is no way to easily
determine if a function is going to return a value in the accumulator or
not. The accumulator is a scratch register so could well contain a valid
pointer that should not in fact survive past the end of the function.
scope2.c in the tests directory looks like this:
----------------------------------------------------
#include <stdlib.h>
static void func1(void)
{
char *pointer = 0;
pointer = malloc(64); /* Line 7 */
return;
} /* Leak report Line 10 */
int main(int argc, char *argv[])
{
func1();
return 0;
}
----------------------------------------------------
If you debug this program in mixed source/assembler, it is clear that at
the exit of func1(), the accumulator holds the value returned by
malloc(64) and also stored in the variable "pointer". Whilst the storage
for the variable dies as the stack unwinds, we cannot easily tell from
the assembler whether func1() is returning the value in the accumulator
or not. Ideally, the leak report should occur at line 10. If we do not
invalidate the accumulator, the leak report will occur at line 15 where
main() set's it's return value, finally overwriting the accumulator.
|
|
From: Oswald B. <os...@kd...> - 2006-09-02 11:45:27
|
On Sat, Sep 02, 2006 at 12:17:34PM +0100, Julian Seward wrote: > On Saturday 02 September 2006 08:22, Oswald Buddenhagen wrote: > > On Sat, Sep 02, 2006 at 03:18:56AM +0100, Julian Seward wrote: > > > Recording program counters without a stack is of limited use, but I > > > can't see how to record a complete stack for each first-write without > > > huge space/time overheads. It might be possible to devise a highly > > > compressed representation for just the PC values, based on the idea > > > that each segment is only going to use a tiny subset of the 2^32/2^64 > > > possible PC values. > > > > i think saving traces in a tree might work. sounds a bit like callgrind. > > Can you give a bit more detail on that? I'm not sure what you mean. > each stacktrace can be represented as a PC and a pointer to the remaining stacktrace. obviously, multiple PCs can have the same remaining trace, so you get a tree if you merge them. to be able to coalesce the lists into a tree actually, each node needs to know its children. the matching of the trace with the existing tree can be done each time on a complete backtrace, which is prohibitively expensive, or gradually by tracking the call graph (and discarding unused nodes upon fn ret). i the end, each "writing incident" can be represented by the written address and a pointer to a "backtrace leaf". of course one can still compress the addresses, have specialized containers with distance pointers instead of full addresses, etc., but this obviously goes at the cost of performance. i didn't do the math, so i have no idea whether that would *actually* work - that's up to you. :) -- Hi! I'm a .signature virus! Copy me into your ~/.signature, please! -- Chaos, panic, and disorder - my work here is done. |
|
From: Bryan M. <om...@br...> - 2006-09-02 11:18:00
|
Chaps,
with respect to storing information per instruction / line of source
code, there is an idea I have been playing with to make covergrind
simpler to code and I think it could well assist in this case (and some
others) as well:
I would have a second, extended version of the DiLoc structure in the
SegInfo structure (coregrind/m_debuginfo/priv_storage.h for those that
want to follow along) that includes a void * for tools to attach data to
/ cast to an appropriate sized int and use directly. Activation would be
through a want_extended_diloc type setting so existing tools are not
affected and don't have the increased memory usage (tools use either the
standard version or the extended - not both).
The rationale is that if you want to store anything that pertains to a
line of source code, there is already an infrastructure there to manage
and index that information. For a fairly cheap modification and a couple
of access functions, it becomes possible to share that infrastructure
instead of duplicating it.
A callback is required for segment unload so that you can a) store the
data you have collected if you need to and b) free off that data if you
attached a structure of your own.
It could well be that the impact of adding a void * to the existing
structure is low enough to not cause problems, particularly when you
consider that some tools must presently duplicate this functionality in
some way.
Just an idea...
Bryan
Julian Seward wrote:
> I'm pleased to see that 'drd', Bart's data race detection tool, has
> recently progressed to the point where we can start to evaluate it.
> A robust, accurate, data race detection tool would be an excellent
> addition to our tool suite, and drd is making promising noises.
>
> There's still a way to go, however. I've been thinking about why
> drd is difficult to use, what can be done to improve it.
>
> When it reports a race, drd says, basically
>
> I found a race between two thread segments
>
> Here's the first segment: thread X, starting stack S1,
> ending stack E1
>
> Here's the second segment: thread Y, starting stack S2,
> ending stack E2
>
> Here are the data addresses ("contended addresses") involved in
> the race:
> 0xF00, 0xBAR, 0xXYZZY
>
> If you get lucky, it may identify those addresses as being global
> data symbols, or inside malloc'd blocks, but often it doesn't.
>
> That's good as far as it goes. For small programs, that kind
> of info is often enough to make sense of the race. But playing with
> drd on knode (threaded news reader in KDE 3.5.4), I find I haven't
> a clue what's going on. Ok, I'm not familar with knode (or KDE, or
> Qt at all), but nevertheless I look at what drd tells me and I still
> am completely unable to identify which parts of the sources might
> be involved in the race.
>
> What drd tells us is the data addresses involved in the race
> (is "contended addresses" a good name for them?) Problem is,
> unless you are extremely lucky, the program is tiny, or you
> are a total genius, it's nearly impossible to figure out where
> in the code these reads/writes are being done.
>
> The DIOTA papers mention the idea of a replay tool associated with
> the race detector. In a first run, the race addresses are computed
> by a drd-like tool. The program is then re-run under the control of
> a "deterministic replay tool", which somehow manages to recreate the
> previous execution, the purpose being to monitor all reads/writes so
> it can take snapshots of the thread stacks when it detects accesses
> to the race addresses. This presumably makes it simple to relate the
> racing to source code locations.
>
> It's a nice idea, but I don't think it's practical here. I don't see
> how the replay tool can work for interactive applications like web
> browsers. And I don't think users will appreciate the hassle.
>
> So here's a different impractical idea :-) When drd is recording
> races, don't just record the set of read/written addresses, but
> something else too: for each written address in the segment, also
> record the program counter of the first (or any) writer of that address.
>
> By definition a race must involve at least one of the threads writing
> the contended address(es). So this is guaranteed to produce at least
> one source code location involved in the race. Now the error report
> might look like this:
>
> [.. other stuff as before ..]
>
> Here's the data addresses involved in the race:
> 0xF00 (first write in segment 1, abcd.c:124),
> 0xBAR (first write in segment 2, snafu.cpp:678),
> 0xXYZZY (first write in segment 1, spqr.c:987
> and in segment 2, badness.c:666)
>
> Recording program counters without a stack is of limited use, but I
> can't see how to record a complete stack for each first-write without
> huge space/time overheads. It might be possible to devise a highly
> compressed representation for just the PC values, based on the idea
> that each segment is only going to use a tiny subset of the 2^32/2^64
> possible PC values.
>
>>From reading the DIOTA papers I got the impression they did a lot of
> work to make DIOTA fast and memory efficient, but I didn't see much stuff
> about what the tool was like to use in practice. Maybe I didn't look
> in the right places.
>
> J
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Valgrind-developers mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-developers
>
|
|
From: Julian S. <js...@ac...> - 2006-09-02 11:17:52
|
On Saturday 02 September 2006 08:22, Oswald Buddenhagen wrote: > On Sat, Sep 02, 2006 at 03:18:56AM +0100, Julian Seward wrote: > > Recording program counters without a stack is of limited use, but I > > can't see how to record a complete stack for each first-write without > > huge space/time overheads. It might be possible to devise a highly > > compressed representation for just the PC values, based on the idea > > that each segment is only going to use a tiny subset of the 2^32/2^64 > > possible PC values. > > i think saving traces in a tree might work. sounds a bit like callgrind. Can you give a bit more detail on that? I'm not sure what you mean. J |
|
From: <js...@ac...> - 2006-09-02 11:08:57
|
Nightly build on minnie ( SuSE 10.0, ppc32 ) started at 2006-09-02 09:00:01 BST Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 207 tests, 10 stderr failures, 6 stdout failures, 0 posttest failures == memcheck/tests/leak-cycle (stderr) memcheck/tests/leak-tree (stderr) memcheck/tests/leakotron (stdout) memcheck/tests/pointer-trace (stderr) memcheck/tests/sigaltstack (stderr) memcheck/tests/xml1 (stderr) none/tests/faultstatus (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) none/tests/ppc32/jm-fp (stdout) none/tests/ppc32/jm-fp (stderr) none/tests/ppc32/round (stdout) none/tests/ppc32/round (stderr) none/tests/ppc32/test_fx (stdout) none/tests/ppc32/test_fx (stderr) none/tests/ppc32/test_gx (stdout) |
|
From: Oswald B. <os...@kd...> - 2006-09-02 07:22:09
|
On Sat, Sep 02, 2006 at 03:18:56AM +0100, Julian Seward wrote: > Recording program counters without a stack is of limited use, but I > can't see how to record a complete stack for each first-write without > huge space/time overheads. It might be possible to devise a highly > compressed representation for just the PC values, based on the idea > that each segment is only going to use a tiny subset of the 2^32/2^64 > possible PC values. > i think saving traces in a tree might work. sounds a bit like callgrind. -- Hi! I'm a .signature virus! Copy me into your ~/.signature, please! -- Chaos, panic, and disorder - my work here is done. |
|
From: <js...@ac...> - 2006-09-02 04:05:18
|
Nightly build on phoenix ( SuSE 10.0 ) started at 2006-09-02 04:30:01 BST Checking out vex source tree ... done Building vex ... done Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 237 tests, 5 stderr failures, 1 stdout failure, 0 posttest failures == memcheck/tests/leak-tree (stderr) memcheck/tests/stack_switch (stderr) memcheck/tests/x86/scalar (stderr) memcheck/tests/x86/scalar_supp (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) |
|
From: Tom H. <th...@cy...> - 2006-09-02 03:12:29
|
Nightly build on gill ( x86_64, Fedora Core 2 ) started at 2006-09-02 03:00:02 BST Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 268 tests, 6 stderr failures, 2 stdout failures, 0 posttest failures == memcheck/tests/mempool (stderr) memcheck/tests/stack_switch (stderr) memcheck/tests/x86/scalar (stderr) memcheck/tests/x86/scalar_supp (stderr) none/tests/fdleak_fcntl (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) none/tests/tls (stdout) |
|
From: Tom H. <to...@co...> - 2006-09-02 02:45:42
|
Nightly build on dunsmere ( athlon, Fedora Core 5 ) started at 2006-09-02 03:30:06 BST Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 239 tests, 5 stderr failures, 1 stdout failure, 0 posttest failures == memcheck/tests/pointer-trace (stderr) memcheck/tests/stack_switch (stderr) memcheck/tests/x86/scalar (stderr) memcheck/tests/xml1 (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) |
|
From: Tom H. <th...@cy...> - 2006-09-02 02:25:22
|
Nightly build on dellow ( x86_64, Fedora Core 5 ) started at 2006-09-02 03:10:03 BST Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 266 tests, 4 stderr failures, 1 stdout failure, 0 posttest failures == memcheck/tests/mempool (stderr) memcheck/tests/x86/scalar (stderr) memcheck/tests/xml1 (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) |
|
From: Tom H. <th...@cy...> - 2006-09-02 02:24:29
|
Nightly build on alvis ( i686, Red Hat 7.3 ) started at 2006-09-02 03:15:02 BST Results differ from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Last 20 lines of verbose log follow echo /tmp/ccCADW43.s:4393: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:4513: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:4633: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:4753: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:4873: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:4993: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:5113: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccCADW43.s:5233: Error: no such instruction: `fisttpq -56(%ebp)' make[5]: *** [insn_sse3.o] Error 1 rm insn_mmx.c insn_sse2.c insn_fpu.c insn_mmxext.c insn_sse.c insn_sse3.c insn_cmov.c insn_basic.c make[5]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests/x86' make[4]: *** [check-am] Error 2 make[4]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests/x86' make[3]: *** [check-recursive] Error 1 make[3]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests' make[2]: *** [check-recursive] Error 1 make[2]: Leaving directory `/tmp/valgrind.27204/valgrind/none' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/tmp/valgrind.27204/valgrind' make: *** [check] Error 2 ================================================= == Results from 24 hours ago == ================================================= Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Last 20 lines of verbose log follow echo /tmp/ccWlzaIu.s:4393: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:4513: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:4633: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:4753: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:4873: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:4993: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:5113: Error: no such instruction: `fisttpq -56(%ebp)' /tmp/ccWlzaIu.s:5233: Error: no such instruction: `fisttpq -56(%ebp)' make[5]: *** [insn_sse3.o] Error 1 rm insn_mmx.c insn_sse2.c insn_fpu.c insn_mmxext.c insn_sse.c insn_sse3.c insn_cmov.c insn_basic.c make[5]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests/x86' make[4]: *** [check-am] Error 2 make[4]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests/x86' make[3]: *** [check-recursive] Error 1 make[3]: Leaving directory `/tmp/valgrind.27204/valgrind/none/tests' make[2]: *** [check-recursive] Error 1 make[2]: Leaving directory `/tmp/valgrind.27204/valgrind/none' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/tmp/valgrind.27204/valgrind' make: *** [check] Error 2 ================================================= == Difference between 24 hours ago and now == ================================================= *** old.short Sat Sep 2 03:19:45 2006 --- new.short Sat Sep 2 03:24:24 2006 *************** *** 7,16 **** Last 20 lines of verbose log follow echo ! /tmp/ccWlzaIu.s:4393: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:4513: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:4633: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:4753: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:4873: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:4993: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:5113: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccWlzaIu.s:5233: Error: no such instruction: `fisttpq -56(%ebp)' make[5]: *** [insn_sse3.o] Error 1 --- 7,16 ---- Last 20 lines of verbose log follow echo ! /tmp/ccCADW43.s:4393: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:4513: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:4633: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:4753: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:4873: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:4993: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:5113: Error: no such instruction: `fisttpq -56(%ebp)' ! /tmp/ccCADW43.s:5233: Error: no such instruction: `fisttpq -56(%ebp)' make[5]: *** [insn_sse3.o] Error 1 |
|
From: Tom H. <th...@cy...> - 2006-09-02 02:20:29
|
Nightly build on lloyd ( x86_64, Fedora Core 3 ) started at 2006-09-02 03:05:05 BST Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 266 tests, 6 stderr failures, 2 stdout failures, 0 posttest failures == memcheck/tests/leakotron (stdout) memcheck/tests/mempool (stderr) memcheck/tests/pointer-trace (stderr) memcheck/tests/stack_switch (stderr) memcheck/tests/x86/scalar (stderr) memcheck/tests/x86/scalar_supp (stderr) none/tests/mremap (stderr) none/tests/mremap2 (stdout) |
|
From: Julian S. <js...@ac...> - 2006-09-02 02:19:14
|
I'm pleased to see that 'drd', Bart's data race detection tool, has=20
recently progressed to the point where we can start to evaluate it.
A robust, accurate, data race detection tool would be an excellent=20
addition to our tool suite, and drd is making promising noises.
There's still a way to go, however. I've been thinking about why
drd is difficult to use, what can be done to improve it.
When it reports a race, drd says, basically
I found a race between two thread segments
Here's the first segment: thread X, starting stack S1,=20
ending stack E1
Here's the second segment: thread Y, starting stack S2,
ending stack E2
Here are the data addresses ("contended addresses") involved in=20
the race:
0xF00, 0xBAR, 0xXYZZY
If you get lucky, it may identify those addresses as being global
data symbols, or inside malloc'd blocks, but often it doesn't.
That's good as far as it goes. For small programs, that kind
of info is often enough to make sense of the race. But playing with=20
drd on knode (threaded news reader in KDE 3.5.4), I find I haven't
a clue what's going on. Ok, I'm not familar with knode (or KDE, or
Qt at all), but nevertheless I look at what drd tells me and I still
am completely unable to identify which parts of the sources might
be involved in the race.
What drd tells us is the data addresses involved in the race
(is "contended addresses" a good name for them?) Problem is,
unless you are extremely lucky, the program is tiny, or you=20
are a total genius, it's nearly impossible to figure out where
in the code these reads/writes are being done.
The DIOTA papers mention the idea of a replay tool associated with
the race detector. In a first run, the race addresses are computed
by a drd-like tool. The program is then re-run under the control of
a "deterministic replay tool", which somehow manages to recreate the
previous execution, the purpose being to monitor all reads/writes so
it can take snapshots of the thread stacks when it detects accesses
to the race addresses. This presumably makes it simple to relate the
racing to source code locations.
It's a nice idea, but I don't think it's practical here. I don't see
how the replay tool can work for interactive applications like web
browsers. And I don't think users will appreciate the hassle.
So here's a different impractical idea :-) When drd is recording
races, don't just record the set of read/written addresses, but
something else too: for each written address in the segment, also
record the program counter of the first (or any) writer of that address.
By definition a race must involve at least one of the threads writing
the contended address(es). So this is guaranteed to produce at least=20
one source code location involved in the race. Now the error report
might look like this:
[.. other stuff as before ..]
Here's the data addresses involved in the race:
0xF00 (first write in segment 1, abcd.c:124),=20
0xBAR (first write in segment 2, snafu.cpp:678),
0xXYZZY (first write in segment 1, spqr.c:987
and in segment 2, badness.c:666)
Recording program counters without a stack is of limited use, but I
can't see how to record a complete stack for each first-write without
huge space/time overheads. It might be possible to devise a highly=20
compressed representation for just the PC values, based on the idea=20
that each segment is only going to use a tiny subset of the 2^32/2^64
possible PC values.
=46rom reading the DIOTA papers I got the impression they did a lot of
work to make DIOTA fast and memory efficient, but I didn't see much stuff
about what the tool was like to use in practice. Maybe I didn't look
in the right places.
J
|
|
From: Julian S. <js...@ac...> - 2006-09-02 01:34:39
|
Now that 3.2.0 is out, and 3.2.1 is not far away, I've been musing about what sorts of things might go into a 3.3 line of Valgrind. Here are some suggestions for discussion. J Comments on upcoming releases ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I am hoping to roll 3.2.1 soon (in the next couple of weeks). 3.2.0 seems to have been pretty stable, but enough bugs have cropped up, a couple of them serious, to make 3.2.1 worthwhile. I've found that making major releases (3.X.0) is a major hassle. 3.1.0 and 3.2.0 each took two weeks full time by the time all the regression test fixing, packaging and testing on 99 different platforms is done. As a consequence: * I prefer not to release 3.3.0 until early next year. * We should keep 3.2.X going longer than 3.1.X and 3.0.X stayed alive, with at least a 3.2.2 release some time around November. * It would be nice to have a second person to help with some of the testing work for 3.3.0. New tools for 3.3.0 ~~~~~~~~~~~~~~~~~~~ In 3.X so far the primary emphasis has been on portability, stability and speed. Now there are some new tools under development (drd, omega, covergrind) and some old experimental ones, in various states of disrepair, which it would be nice to play with from time to time (annelid, diduce, helgrind). I've been thinking of splitting the tools into two groups: production-grade tools (memcheck, cachegrind, callgrind, massif) and experimental tools (drd, omega, covergrind, annelid, diduce, helgrind), and having both sets in the standard Valgrind tree. For production tools we would continue to ensure they have high quality and are stable, as at present. For experimental tools we would try to ensure they compile, but give no assurance beyond that. The problem with experimental tools is they need a lot of engineering effort to get them to a production status (or conclusion that the tool cannot be moved to production status for technical reasons.) Getting that effort means having users try them out and contribute feedback and patches. Putting the tools in the tree, and having them compile, even if they don't work well, makes it a lot easier for users to do that. It's also more inclusive for the tool authors. There are downsides: - more code in the tree inevitably means a higher maintenance overhead - stability of the existing code base is important, and we don't want to undermine that I'm thinking of an arrangement in which experimental tool authors have commit access to the tree, but - we make it clear it is their responsibility to keep tools compiling and working - we ask that such authors do not commit changes outside of their individual tools without first consulting with the core developers New ports for 3.3.0 ~~~~~~~~~~~~~~~~~~~ This year there has been quiet but steady work towards porting V away from Linux. There are now ports to FreeBSD, MacOSX and AIX5 in various states of progress, and it seems likely that some of those will appear in the mainline tree at some point. I am expecting that our existing porting infrastructure will continue to be refined and extended so that these ports can be accommodated without majorly intrusive changes in the majority of the code base. So far with the Linux ports to x86,amd64,ppc32,ppc64, the target-specific stuff has been isolated into a relatively few places (eg, m_syswrap, m_sigframe, m_syscall, VEX), leaving the rest of the system fairly untouched, and I am hoping this can be continued. The regression test infrastructure has proven invaluable in making V as reliable as it is. However, even with Linux it is too sensitive to changes in stack backtraces and address space layouts, and as a result causes tests to fail when really they should succeed. With new ports on the horizon this problem is about to get much worse. If anyone is motivated to overhaul this unglamorous but critical subsystem, that would be much appreciated. |