You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
1
(8) |
2
(8) |
3
(15) |
4
(14) |
5
(12) |
6
(40) |
7
(9) |
|
8
(5) |
9
(12) |
10
(9) |
11
(13) |
12
(7) |
13
(7) |
14
(19) |
|
15
(18) |
16
(13) |
17
(16) |
18
(8) |
19
(16) |
20
(16) |
21
(12) |
|
22
(21) |
23
(39) |
24
(27) |
25
(33) |
26
(41) |
27
(17) |
28
(15) |
|
From: John R.
|
[Julian] >>>> I studied the relevant loop in zlib, at deflate.c:1106, and ended up >>>> wondering if it is possible to rewrite it, without loss of performance, >>>> so that it does not need to visit uninitialised memory. [Mark] >>> I'm sure it is possible, but I am loathe to make any changes to code >>> that has been *extensively* tested over a decade and is not broken, just >>> to make a code checker happy. [Johannes] >> Indeed, but it _is_ sloppy code, from the viewpoint of a correctness >> checker. [Julian] > Being heavily involved with both Valgrind and bzip2, I sympathise with > both points of view. In the broader view, software has evolved so that developers expect that libraries will interact nicely with programmatic checkers of memory accesses. The particular code in zlib meets the old expectations of functional correctness, but not the new expectations of a longer value chain which includes smooth interaction with memcheck. Today's customers of zlib expect more from zlib than yesterday's customers did. In other similar cases, I have modified library code to initialize possible "overrun" bytes before the loop starts, even though this is not strictly necessary for functional correctness. The cost of a few instructions of static code, plus a few machine cycles per dynamic buffer, is repaid by a shorter development cycle and higher reliability in end products which use the library. The library users who highly value minimization of CPU cycles have become outnumbered by the users who highly value short time-to-market for reliable new products that use the library. For examples see http://bitwagon.com/glibc-audit/glibc-audit.html . -- |
|
From: Julian S. <js...@ac...> - 2009-02-02 18:53:16
|
> On Mon, 2 Feb 2009, Mark Adler wrote: > > On Feb 2, 2009, at 1:54 AM, Julian Seward wrote: > > >Another possibility is that it was copied to that place from somewhere > > > else, in an uninitialised state. > > > > Except that that byte happens to be exactly the right value it needs to > > be in order to part of the valid zlib stream (a 0x2c in this case). > > Any other value results in an inflate error on the data. Either an odd > > coincidence with a 1/256 probability, or far more likely, that the byte > > is in fact not uninitialized. Sloppy wording on my part. I did not mean to imply the byte was really uninitialised. I am sure that what happened is, as Johannes says, > [...] At > some stage, the "uninitializedness" of the bytes that are looked at during > longest_match() transpires to the values deduced from the length o the > longest match. Correct. And whether that happens or not depends on the details of the code that gcc generates, specifically how it interacts with Memcheck's definedness tracking machinery. That's why it appears to be distro-specific -- in fact I think it is more likely to be gcc-version-specific. > > >I studied the relevant loop in zlib, at deflate.c:1106, and ended up > > >wondering if it is possible to rewrite it, without loss of performance, > > >so that it does not need to visit uninitialised memory. > > > > I'm sure it is possible, but I am loathe to make any changes to code > > that has been *extensively* tested over a decade and is not broken, just > > to make a code checker happy. > > Indeed, but it _is_ sloppy code, from the viewpoint of a correctness > checker. Being heavily involved with both Valgrind and bzip2, I sympathise with both points of view. I see two ways out of this. 1. In the short term we can hunt around in the Ubuntu-gcc assembly code to find out what is confusing Memcheck, possibly develop a fix, and move on. 2. In the longer term, rewrite the zlib loop so it doesn't enter uninitialised memory. Given that the (maximum) trip count is known before the loop, I'm fairly sure it is possible to use standard loop transformations: unrolling, vectorization and peeling, to achieve this. I lack enthusiasm for (1). Doing bit-exact definedness tracking for the x86 condition code registers (O S Z A C and P) is expensive, and Memcheck currently has some speed-accuracy tradeoffs that work well in practice for code that doesn't use uninitialised values in this rather subtle way. I am reluctant to special-case it for zlib, as that will likely impose a performance burden on all code handled by Memcheck, and Memcheck is already undesirably slow. Also, (1) is much like playing whack-a-mole. Sure we can track down and fix this hole, and that works until some devilishly clever code generation trick in a future gcc-5.4.3 does something Memcheck doesn't expect, and we are back at square 1 again. J |
|
From: Johannes S. <Joh...@gm...> - 2009-02-02 17:58:40
|
Hi, On Mon, 2 Feb 2009, Mark Adler wrote: > On Feb 2, 2009, at 1:54 AM, Julian Seward wrote: > >Another possibility is that it was copied to that place from somewhere else, > >in an uninitialised state. > > Except that that byte happens to be exactly the right value it needs to > be in order to part of the valid zlib stream (a 0x2c in this case). > Any other value results in an inflate error on the data. Either an odd > coincidence with a 1/256 probability, or far more likely, that the byte > is in fact not uninitialized. I'm sure that the latter is the case, > since the zlib code sets that byte in the output -- it doesn't have a > mechanism to skip a byte when writing. I would conjecture that > valgrind's propagation of uninitialized values is in error. The problem is indeed the "uninitialized" thing, as far as I read it. At some stage, the "uninitializedness" of the bytes that are looked at during longest_match() transpires to the values deduced from the length o the longest match. > >I studied the relevant loop in zlib, at deflate.c:1106, and ended up > >wondering if it is possible to rewrite it, without loss of performance, > >so that it does not need to visit uninitialised memory. > > I'm sure it is possible, but I am loathe to make any changes to code > that has been *extensively* tested over a decade and is not broken, just > to make a code checker happy. Indeed, but it _is_ sloppy code, from the viewpoint of a correctness checker. I could imagine that it _might_ possible to use a tell-tale detector just for zlib, but I think that the patch I sent should be some lower-impact solution to the problem. Because a problem it is: if I add suppressions for the code that depends on deflated data, I will no longer be able to catch true errors that do depend on uninitialized memory. Ciao, Dscho |
|
From: Johannes S. <Joh...@gm...> - 2009-02-02 13:38:34
|
Hi, On Mon, 2 Feb 2009, Julian Seward wrote: > > > Can you tell me what exactly I need to do to get the byte-51 undef > > > error, including details of what distro you're using? > > > > Ubuntu Gutsy and Intrepid, both x86_64, zlib-1.2.3.3-ubuntu5, although I > > can reproduce even with zlib-1.2.3. The key is to use both -O3 and > > -DUNALIGNED_OK. > > Right. So on Ubuntu 8.04.2 I can easily reproduce it. Situation is now > much clearer. Thanks! > This is (unfortunately) another version of the FAQ#36 issue. > Exactly why the undefinedness continues to propagate when built on > Ubuntu but not on openSUSE, I don't know, despite a couple hours > of investigation. > > The error message in this case is a bit misleading: > > Uninitialised byte(s) found during client check request > at 0x400787: main (minimal.c:73) > Address 0x51de87b is 51 bytes inside a block of size 193 alloc'd > at 0x4C2694E: malloc (vg_replace_malloc.c:207) > by 0x4006A6: main (minimal.c:49) > > It's true that at the point this VALGRIND_CHECK_MEM_IS_DEFINED was done, > an uninitialised value was observed 51 bytes inside a block you malloced > yourself (so to speak). However, it is not correct to assume that > that byte is uninitialised as a result of that specific malloc. Another > possibility is that it was copied to that place from somewhere else, in an > uninitialised state. And if you add --track-origins=yes the original source > is stated: > > Uninitialised value was created by a heap allocation > at 0x4C2694E: malloc (vg_replace_malloc.c:207) > by 0x40491C: zcalloc (zutil.c:306) > by 0x402BEB: deflateInit2_ (deflate.c:289) > by 0x402D40: deflateInit_ (deflate.c:212) > by 0x40068E: main (minimal.c:47) > > which confirms it as the same source as all the other (false) uninit-value > errors that Memcheck reports in zlib-1.2.x. > > So you can safely ignore it. Can I make valgrind ignore this particular error, without having to mark _my_ code with a suppression? > See http://www.valgrind.org/docs/manual/mc-manual.html#mc-manual.value > for details of Memcheck's definedness-tracking machinery. > > I studied the relevant loop in zlib, at deflate.c:1106, and ended up > wondering if it is possible to rewrite it, without loss of performance, > so that it does not need to visit uninitialised memory. Right. Note, however, that the issue only arises with UNLIGNED_OK, which is handled at deflate.c:1138. My suspicion is that it would actually make sense to have code in zlib that _initializes_ the buffer at the end, instead of changing the loop. You could make this initialization an opt-out, so that appliances needing zlib do not need these extra-cycles. Something like this: -- snipsnap -- [PATCH] Make valgrind happy by default Valgrind does not like zlib to use uninitialized memory, but for performance reasons, zlib has to avoid the end-of-buffer check until it quite possibly overran by 8 bytes. As zlib fixes the overrun right after the loop, this is not really a mistake. To make valgrind happy, just memset() the whole buffer, so that all such accesses are no longer marked as accesses to undefined memory. As a typical window size is just 32kB, and the initialization is a one-time cost in deflateInit2_(), it does not hurt performance too much. In case that this code does hurt performance-wise, it can be switched off by defining MAKE_VALGRIND_UNHAPPY. Note: the initialization is only performed if sizeof(uInt) > 2, because zalloc() calls calloc() otherwise anyway. Note alse: the same effect could be achieved in a cleaner manner by using valgrind's VALGRIND_MAKE_MEM_DEFINED() macro, but that would make zlib's source code depend on valgrind's headers. Signed-off-by: Johannes Schindelin <joh...@gm...> --- deflate.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/deflate.c b/deflate.c index 29ce1f6..3032d8e 100644 --- a/deflate.c +++ b/deflate.c @@ -288,6 +288,11 @@ int ZEXPORT deflateInit2_(strm, level, method, windowBits, memLevel, strategy, s->prev = (Posf *) ZALLOC(strm, s->w_size, sizeof(Pos)); s->head = (Posf *) ZALLOC(strm, s->hash_size, sizeof(Pos)); +#ifndef MAKE_VALGRIND_UNHAPPY + if (sizeof(uInt) > 2) + memset(s->window, 0, s->w_size); +#endif + s->lit_bufsize = 1 << (memLevel + 6); /* 16K elements by default */ overlay = (ushf *) ZALLOC(strm, s->lit_bufsize, sizeof(ush)+2); -- 1.6.1.2.556.g3bd7 |
|
From: Julian S. <js...@ac...> - 2009-02-02 09:54:39
|
> > Can you tell me what exactly I need to do to get the byte-51 undef
> > error, including details of what distro you're using?
>
> Ubuntu Gutsy and Intrepid, both x86_64, zlib-1.2.3.3-ubuntu5, although I
> can reproduce even with zlib-1.2.3. The key is to use both -O3 and
> -DUNALIGNED_OK.
Right. So on Ubuntu 8.04.2 I can easily reproduce it. Situation is now
much clearer.
This is (unfortunately) another version of the FAQ#36 issue.
Exactly why the undefinedness continues to propagate when built on
Ubuntu but not on openSUSE, I don't know, despite a couple hours
of investigation.
The error message in this case is a bit misleading:
Uninitialised byte(s) found during client check request
at 0x400787: main (minimal.c:73)
Address 0x51de87b is 51 bytes inside a block of size 193 alloc'd
at 0x4C2694E: malloc (vg_replace_malloc.c:207)
by 0x4006A6: main (minimal.c:49)
It's true that at the point this VALGRIND_CHECK_MEM_IS_DEFINED was done,
an uninitialised value was observed 51 bytes inside a block you malloced
yourself (so to speak). However, it is not correct to assume that
that byte is uninitialised as a result of that specific malloc. Another
possibility is that it was copied to that place from somewhere else, in an
uninitialised state. And if you add --track-origins=yes the original source
is stated:
Uninitialised value was created by a heap allocation
at 0x4C2694E: malloc (vg_replace_malloc.c:207)
by 0x40491C: zcalloc (zutil.c:306)
by 0x402BEB: deflateInit2_ (deflate.c:289)
by 0x402D40: deflateInit_ (deflate.c:212)
by 0x40068E: main (minimal.c:47)
which confirms it as the same source as all the other (false) uninit-value
errors that Memcheck reports in zlib-1.2.x.
So you can safely ignore it.
See http://www.valgrind.org/docs/manual/mc-manual.html#mc-manual.value
for details of Memcheck's definedness-tracking machinery.
I studied the relevant loop in zlib, at deflate.c:1106, and ended up
wondering if it is possible to rewrite it, without loss of performance,
so that it does not need to visit uninitialised memory.
J
|
|
From: Tom H. <th...@cy...> - 2009-02-02 03:48:02
|
Nightly build on vauxhall ( x86_64, Fedora 10 ) started at 2009-02-02 03:20:07 GMT Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 486 tests, 1 stderr failure, 0 stdout failures, 0 post failures == memcheck/tests/x86-linux/scalar (stderr) |
|
From: Tom H. <th...@cy...> - 2009-02-02 03:44:14
|
Nightly build on lloyd ( x86_64, Fedora 7 ) started at 2009-02-02 03:05:05 GMT Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 477 tests, 6 stderr failures, 0 stdout failures, 0 post failures == exp-ptrcheck/tests/ccc (stderr) exp-ptrcheck/tests/preen_invars (stderr) exp-ptrcheck/tests/pth_create (stderr) exp-ptrcheck/tests/pth_specific (stderr) helgrind/tests/tc20_verifywrap (stderr) memcheck/tests/x86-linux/scalar (stderr) |
|
From: Tom H. <th...@cy...> - 2009-02-02 03:31:58
|
Nightly build on mg ( x86_64, Fedora 9 ) started at 2009-02-02 03:10:05 GMT Results unchanged from 24 hours ago Checking out valgrind source tree ... done Configuring valgrind ... done Building valgrind ... done Running regression tests ... failed Regression test results follow == 483 tests, 5 stderr failures, 2 stdout failures, 0 post failures == exp-ptrcheck/tests/ccc (stderr) exp-ptrcheck/tests/preen_invars (stderr) exp-ptrcheck/tests/pth_create (stderr) exp-ptrcheck/tests/pth_specific (stderr) memcheck/tests/linux/timerfd-syscall (stdout) memcheck/tests/x86-linux/scalar (stderr) none/tests/mremap2 (stdout) |