You can subscribe to this list here.
| 2002 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(122) |
Nov
(152) |
Dec
(69) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2003 |
Jan
(6) |
Feb
(25) |
Mar
(73) |
Apr
(82) |
May
(24) |
Jun
(25) |
Jul
(10) |
Aug
(11) |
Sep
(10) |
Oct
(54) |
Nov
(203) |
Dec
(182) |
| 2004 |
Jan
(307) |
Feb
(305) |
Mar
(430) |
Apr
(312) |
May
(187) |
Jun
(342) |
Jul
(487) |
Aug
(637) |
Sep
(336) |
Oct
(373) |
Nov
(441) |
Dec
(210) |
| 2005 |
Jan
(385) |
Feb
(480) |
Mar
(636) |
Apr
(544) |
May
(679) |
Jun
(625) |
Jul
(810) |
Aug
(838) |
Sep
(634) |
Oct
(521) |
Nov
(965) |
Dec
(543) |
| 2006 |
Jan
(494) |
Feb
(431) |
Mar
(546) |
Apr
(411) |
May
(406) |
Jun
(322) |
Jul
(256) |
Aug
(401) |
Sep
(345) |
Oct
(542) |
Nov
(308) |
Dec
(481) |
| 2007 |
Jan
(427) |
Feb
(326) |
Mar
(367) |
Apr
(255) |
May
(244) |
Jun
(204) |
Jul
(223) |
Aug
(231) |
Sep
(354) |
Oct
(374) |
Nov
(497) |
Dec
(362) |
| 2008 |
Jan
(322) |
Feb
(482) |
Mar
(658) |
Apr
(422) |
May
(476) |
Jun
(396) |
Jul
(455) |
Aug
(267) |
Sep
(280) |
Oct
(253) |
Nov
(232) |
Dec
(304) |
| 2009 |
Jan
(486) |
Feb
(470) |
Mar
(458) |
Apr
(423) |
May
(696) |
Jun
(461) |
Jul
(551) |
Aug
(575) |
Sep
(134) |
Oct
(110) |
Nov
(157) |
Dec
(102) |
| 2010 |
Jan
(226) |
Feb
(86) |
Mar
(147) |
Apr
(117) |
May
(107) |
Jun
(203) |
Jul
(193) |
Aug
(238) |
Sep
(300) |
Oct
(246) |
Nov
(23) |
Dec
(75) |
| 2011 |
Jan
(133) |
Feb
(195) |
Mar
(315) |
Apr
(200) |
May
(267) |
Jun
(293) |
Jul
(353) |
Aug
(237) |
Sep
(278) |
Oct
(611) |
Nov
(274) |
Dec
(260) |
| 2012 |
Jan
(303) |
Feb
(391) |
Mar
(417) |
Apr
(441) |
May
(488) |
Jun
(655) |
Jul
(590) |
Aug
(610) |
Sep
(526) |
Oct
(478) |
Nov
(359) |
Dec
(372) |
| 2013 |
Jan
(467) |
Feb
(226) |
Mar
(391) |
Apr
(281) |
May
(299) |
Jun
(252) |
Jul
(311) |
Aug
(352) |
Sep
(481) |
Oct
(571) |
Nov
(222) |
Dec
(231) |
| 2014 |
Jan
(185) |
Feb
(329) |
Mar
(245) |
Apr
(238) |
May
(281) |
Jun
(399) |
Jul
(382) |
Aug
(500) |
Sep
(579) |
Oct
(435) |
Nov
(487) |
Dec
(256) |
| 2015 |
Jan
(338) |
Feb
(357) |
Mar
(330) |
Apr
(294) |
May
(191) |
Jun
(108) |
Jul
(142) |
Aug
(261) |
Sep
(190) |
Oct
(54) |
Nov
(83) |
Dec
(22) |
| 2016 |
Jan
(49) |
Feb
(89) |
Mar
(33) |
Apr
(50) |
May
(27) |
Jun
(34) |
Jul
(53) |
Aug
(53) |
Sep
(98) |
Oct
(206) |
Nov
(93) |
Dec
(53) |
| 2017 |
Jan
(65) |
Feb
(82) |
Mar
(102) |
Apr
(86) |
May
(187) |
Jun
(67) |
Jul
(23) |
Aug
(93) |
Sep
(65) |
Oct
(45) |
Nov
(35) |
Dec
(17) |
| 2018 |
Jan
(26) |
Feb
(35) |
Mar
(38) |
Apr
(32) |
May
(8) |
Jun
(43) |
Jul
(27) |
Aug
(30) |
Sep
(43) |
Oct
(42) |
Nov
(38) |
Dec
(67) |
| 2019 |
Jan
(32) |
Feb
(37) |
Mar
(53) |
Apr
(64) |
May
(49) |
Jun
(18) |
Jul
(14) |
Aug
(53) |
Sep
(25) |
Oct
(30) |
Nov
(49) |
Dec
(31) |
| 2020 |
Jan
(87) |
Feb
(45) |
Mar
(37) |
Apr
(51) |
May
(99) |
Jun
(36) |
Jul
(11) |
Aug
(14) |
Sep
(20) |
Oct
(24) |
Nov
(40) |
Dec
(23) |
| 2021 |
Jan
(14) |
Feb
(53) |
Mar
(85) |
Apr
(15) |
May
(19) |
Jun
(3) |
Jul
(14) |
Aug
(1) |
Sep
(57) |
Oct
(73) |
Nov
(56) |
Dec
(22) |
| 2022 |
Jan
(3) |
Feb
(22) |
Mar
(6) |
Apr
(55) |
May
(46) |
Jun
(39) |
Jul
(15) |
Aug
(9) |
Sep
(11) |
Oct
(34) |
Nov
(20) |
Dec
(36) |
| 2023 |
Jan
(79) |
Feb
(41) |
Mar
(99) |
Apr
(169) |
May
(48) |
Jun
(16) |
Jul
(16) |
Aug
(57) |
Sep
(19) |
Oct
|
Nov
|
Dec
|
| S | M | T | W | T | F | S |
|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
1
(14) |
|
2
|
3
(6) |
4
(9) |
5
(23) |
6
(6) |
7
(10) |
8
(2) |
|
9
(1) |
10
(5) |
11
(2) |
12
(5) |
13
(2) |
14
(2) |
15
(4) |
|
16
(3) |
17
(22) |
18
(21) |
19
(15) |
20
(24) |
21
(5) |
22
(5) |
|
23
(14) |
24
(2) |
25
(2) |
26
(1) |
27
|
28
|
29
(1) |
|
30
|
31
|
|
|
|
|
|
|
From: Ivo R. <iv...@iv...> - 2016-10-15 15:36:48
|
2016-10-15 15:01 GMT+02:00 Philippe Waroquiers < phi...@sk...>: > Quadratic aspect of meta pool fixed in r16041 > Renamed macro and clarified documentation in r16042. > Philippe, Thank you very much for providing such nice explanation about implicit relation between first and second level blocks in a metapool when various flags are in effect. Now the documentation clearly describes what the implementation does. I. |
|
From: Philippe W. <phi...@sk...> - 2016-10-15 13:00:57
|
Quadratic aspect of meta pool fixed in r16041 Renamed macro and clarified documentation in r16042. Thanks Philippe On Fri, 2016-10-14 at 07:45 +0000, Ruurd Beerstra wrote: > Hi, > > > > Sorry about the delay… > > > > 1. The MEMPOOL_DELETE was, indeed, a bad comment. Has to be > VALGRIND_MEMPOOL_FREE. > The documentation (memcheck/docs/mc-manual.xml) did use the correct > name. > > 2. I see where the confusion comes from. Apologies. > In my first attempts to get valgrind to understand our custom > allocator, I saw a need to have a special attributes for a memory > pool. > - A flag to indicate that items in a pool-block are to be freed when > VALGRIND_MEMPOOL_FREE is used on such a block. > - A flag to indicate that a pool block is allocated (with > VALGRIND_MALLOCLIKE_BLOCK) which is then used as a pool to allocate > smaller VALGRIND_MALLOCLIKE_BLOCKs from. > > Without the first attribute, I got thousands of leaks reported that > were not really leaks. > Without the 2nd attribute, valgrind would crash with an assert, > complaining about overlapping blocks. > So: Without these fixes, valgrind is not usable in our environment. > My first implementation of this broke compatibility in valgrind.h by > altering the signature of VALGRIND_CREATE_MEMPOOL. > I managed to get it to work in our environment that way and we found > and fixed several issues in our large code base. Yippee! > Then I tried to get the fixes in valgrind reworked so they could > become part of the standard distribution, because our code is > multi-platform, and we continuously support new hardware and > OS-releases. I do not want to have to re-fix every future version of > valgrind. > The naming of the reworked version is, I must admit, badly chosen. The > attribute names reflect the implementation, not the functionality. > There is no point in having an AUTO_FREE pool which is not a METAPOOL. > The name of the macro already tells you that, and AUTO_FREE only makes > sense when chunks are contained in other chunks (i.e. META). > A METAPOOL that is not AUTO_FREE is more logical, I can envision a > custom allocator that. > So a VALGRIND_CREATE_META_MEMPOOL that only accepts an AUTO_FREE flag > is better. > Or a name like VALGRIND_CREATE_MEMPOOL_EXT or some other convention > that allows future expansion. > > > > So I agree with the renaming: The VALGRIND_MEMPOOL_METAPOOL flag can > go and is implied in the call. > > The documentation of that flag (which only confuses things) can be > removed. > > There is a test which tests the double-level stuff. > > The memcheck/tests/leak-autofreepool.c test creates a pool with > blocks, the allocates (with VALGRIND_MALLOCLIKE_BLOCK, like our custom > allocator does. > > In test case 5, it demonstrates that valgrind asserts and aborts if > the METAPOOL flags is not passed. > > Again, the documentation is confusing: the blocks overlap, but one is > of type MEMPOOL_ALLOC and the second of MALLOCLIKE_BLOCK type. > > > > Apologies for that: my focus was more on “getting it working here and > for us now” than on a clean, complete & spotless valgrind patch. > > I’m glad the vetting process is so thorough, thoughJ > > > > Who is going to polish up this stuff? > > I can do it, but if one of the valgrind veterans does it I’ll be > happy, too (and if I do it, it might again take a few tries before it > is deemed satisfactory). > > The change to our custom allocator will be tiny (use the new macro and > names) and as long as it works, I’m happy. > > > > Regards, > > Ruurd Beerstra > > > > > > From: Ivo Raisr [mailto:iv...@iv...] > Sent: Thursday, October 06, 2016 22:02 > To: Ruurd Beerstra <Ruu...@in...> > Subject: Fwd: [Valgrind-developers] Confusion in recent meta mempool > doc and/or usage > > > > FYI if you are not on valgrind-developers > > > > > ---------- Forwarded message ---------- > From: Philippe Waroquiers <phi...@sk...> > Date: 2016-10-06 21:59 GMT+02:00 > Subject: [Valgrind-developers] Confusion in recent meta mempool doc > and/or usage > To: valgrind-developers <val...@li...> > > > I am a little bit lost in the documentation and/or semantic > of the recently added meta mempool feature. > > First, valgrind.h tells: > ... When the VALGRIND_MEMPOOL_AUTO_FREE > is passed, a MEMPOOL_DELETE will auto-free all chunks (so not > reported as > leaks) for allocators that assume that destroying a pool destroys > all > objects in the pool. > ... > > But this operation MEMPOOL_DELETE does not exist. > This looks to be rather VALGRIND_MEMPOOL_FREE. > In the manual, the description is somewhat different. > > > Second thing confusing me: > The user manual tells for META mempool first describe the auto free > flag: > ... > This indicates that items allocated from this memory pool are > automatically > freed when VALGRIND_MEMPOOL_FREE is used on a block. > ... > > But of course, when calling VALGRIND_MEMPOOL_FREE, by definition, the > addr > given as argument is supposed to be a chunk of the pool, and is > released. > So, this flag is by itself useless, unless it is combined with > VALGRIND_MEMPOOL_METAPOOL. > Do I miss something ? > > Assuming I did not miss something, then we now have a new macro > VALGRIND_CREATE_META_MEMPOOL(pool, rzB, is_zeroed, flags) > which in its names indicates it is a meta pool. > So, it is unclear why we ask the user to specify > VALGRIND_MEMPOOL_METAPOOL > and/or allows the user to specify VALGRIND_MEMPOOL_AUTO_FREE, > as this will be useless unless the pool is a real meta pool. > (checking the code, effectively, the only thing an AUTO_FREE pool does > is to release automatically the '2nd level smaller blocks' when a > 'first level > pool bigger block' is released with VALGRIND_MEMPOOL_FREE > > > The second flag documented is also somewhat confusingly described: > ... > This indicates that memory that has been marked as being allocated > with > VALGRIND_MALLOCLIKE_BLOCK is used by a custom allocator to pass out > memory > to an application (again marked with VALGRIND_MALLOCLIKE_BLOCK). > ... > > I do not see anything in tests and/or code that would implement > something > like a double level of VALGRIND_MALLOCLIKE_BLOCK. > As I understand, we have a first level of blocks which are described > with VALGRIND_MEMPOOL_ALLOC > Then, such first level blocks can be split and allocated > using VALGRIND_MALLOCLIKE_BLOCK. > > > I think it would be better to clarify the documentation and maybe > remove this (useless for the user) VALGRIND_MEMPOOL_METAPOOL : > The underlying (internal) client request might effectively be > the same (and so have an 'internal' flag VALGRIND_MEMPOOL_METAPOOL > but it looks to me that describing this flag to the user is just > confusing. > > Philippe > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > _______________________________________________ > Valgrind-developers mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-developers > > > > > |
|
From: <sv...@va...> - 2016-10-15 12:59:21
|
Author: philippe
Date: Sat Oct 15 13:59:04 2016
New Revision: 16042
Log:
Clarify name and description/manual for meta mempool
* rename macro VALGRIND_CREATE_META_MEMPOOL
to VALGRIND_CREATE_MEMPOOL_EXT
* abort execution if a pool is marked as auto_free but is not a meta pool
+ removed test leak-autofreepool-3.vgtest, which now aborts.
* reword/clarify valgrind.h explanations for meta pool
* similarly reword/clarify the manual
Removed:
trunk/memcheck/tests/leak-autofreepool-3.stderr.exp
trunk/memcheck/tests/leak-autofreepool-3.vgtest
Modified:
trunk/include/valgrind.h
trunk/memcheck/docs/mc-manual.xml
trunk/memcheck/mc_malloc_wrappers.c
trunk/memcheck/tests/Makefile.am
trunk/memcheck/tests/leak-autofreepool.c
Modified: trunk/include/valgrind.h
==============================================================================
--- trunk/include/valgrind.h (original)
+++ trunk/include/valgrind.h Sat Oct 15 13:59:04 2016
@@ -7009,21 +7009,37 @@
VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CREATE_MEMPOOL, \
pool, rzB, is_zeroed, 0, 0)
-/* Create a memory pool with special flags. When the VALGRIND_MEMPOOL_AUTO_FREE
- is passed, a MEMPOOL_DELETE will auto-free all chunks (so not reported as
- leaks) for allocators that assume that destroying a pool destroys all
- objects in the pool. When VALGRIND_MEMPOOL_METAPOOL is passed, the custom
- allocator uses the pool blocks as superblocks to dole out MALLOC_LIKE blocks.
- The resulting behaviour would normally be classified as overlapping blocks,
- and cause assert-errors in valgrind.
- These 2 MEMPOOL flags can be OR-ed together into the "flags" argument.
+/* Create a memory pool with some flags specifying extended behaviour.
When flags is zero, the behaviour is identical to VALGRIND_CREATE_MEMPOOL.
+
+ The flag VALGRIND_MEMPOOL_METAPOOL specifies that the pieces of memory
+ associated with the pool using VALGRIND_MEMPOOL_ALLOC will be used
+ by the application as superblocks to dole out MALLOC_LIKE blocks using
+ VALGRIND_MALLOCLIKE_BLOCK. In other words, a meta pool is a "2 levels"
+ pool : first level is the blocks described by VALGRIND_MEMPOOL_ALLOC.
+ The second level blocks are described using VALGRIND_MALLOCLIKE_BLOCK.
+ Note that the association between the pool and the second level blocks
+ is implicit : second level blocks will be located inside first level
+ blocks. It is necessary to use the VALGRIND_MEMPOOL_METAPOOL flag
+ for such 2 levels pools, as otherwise valgrind will detect overlapping
+ memory blocks, and will abort execution (e.g. during leak search).
+
+ Such a meta pool can also be marked as an 'auto free' pool using the flag
+ VALGRIND_MEMPOOL_AUTO_FREE, which must be OR-ed together with the
+ VALGRIND_MEMPOOL_METAPOOL. For an 'auto free' pool, VALGRIND_MEMPOOL_FREE
+ will automatically free the second level blocks that are contained
+ inside the first level block freed with VALGRIND_MEMPOOL_FREE.
+ In other words, calling VALGRIND_MEMPOOL_FREE will cause implicit calls
+ to VALGRIND_FREELIKE_BLOCK for all the second level blocks included
+ in the first level block.
+ Note: it is an error to use the VALGRIND_MEMPOOL_AUTO_FREE flag
+ without the VALGRIND_MEMPOOL_METAPOOL flag.
*/
#define VALGRIND_MEMPOOL_AUTO_FREE 1
#define VALGRIND_MEMPOOL_METAPOOL 2
-#define VALGRIND_CREATE_META_MEMPOOL(pool, rzB, is_zeroed, flags) \
- VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CREATE_MEMPOOL, \
- pool, rzB, is_zeroed, flags, 0)
+#define VALGRIND_CREATE_MEMPOOL_EXT(pool, rzB, is_zeroed, flags) \
+ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CREATE_MEMPOOL, \
+ pool, rzB, is_zeroed, flags, 0)
/* Destroy a memory pool. */
#define VALGRIND_DESTROY_MEMPOOL(pool) \
Modified: trunk/memcheck/docs/mc-manual.xml
==============================================================================
--- trunk/memcheck/docs/mc-manual.xml (original)
+++ trunk/memcheck/docs/mc-manual.xml Sat Oct 15 13:59:04 2016
@@ -2320,34 +2320,53 @@
</listitem>
<listitem>
+ <!-- Note: the below is mostly a copy of valgrind.h. Keep in sync! -->
<para>
- <varname>VALGRIND_CREATE_META_MEMPOOL(pool, rzB, is_zeroed, flags)</varname>:
- This does the same as <varname>VALGRIND_CREATE_MEMPOOL</varname>,
- but allows you to specify two seldom-used options for custom
- allocators (or-ed together) in the <varname>flags</varname> argument:</para>
+ <varname>VALGRIND_CREATE_MEMPOOL_EXT(pool, rzB, is_zeroed, flags)</varname>:
+ Create a memory pool with some flags (that can
+ be OR-ed together) specifying extended behaviour. When flags is
+ zero, the behaviour is identical to
+ <varname>VALGRIND_CREATE_MEMPOOL</varname>.</para>
<itemizedlist>
<listitem>
- <para>
- <varname>VALGRIND_MEMPOOL_AUTO_FREE</varname>.
- This indicates that items allocated from this
- memory pool are automatically freed when
- <varname>VALGRIND_MEMPOOL_FREE</varname>
- is used on a block. This allows a custom allocator to delete
- (part of) a memory pool without explicitly deleting all allocated
- items. Without this option, such an action will report all items
- in the pool as memory leaks.
+ <para> The flag <varname>VALGRIND_MEMPOOL_METAPOOL</varname>
+ specifies that the pieces of memory associated with the pool
+ using <varname>VALGRIND_MEMPOOL_ALLOC</varname> will be used
+ by the application as superblocks to dole out MALLOC_LIKE
+ blocks using <varname>VALGRIND_MALLOCLIKE_BLOCK</varname>.
+ In other words, a meta pool is a "2 levels" pool : first
+ level is the blocks described
+ by <varname>VALGRIND_MEMPOOL_ALLOC</varname>. The second
+ level blocks are described
+ using <varname>VALGRIND_MALLOCLIKE_BLOCK</varname>. Note
+ that the association between the pool and the second level
+ blocks is implicit : second level blocks will be located
+ inside first level blocks. It is necessary to use
+ the <varname>VALGRIND_MEMPOOL_METAPOOL</varname> flag for
+ such 2 levels pools, as otherwise valgrind will detect
+ overlapping memory blocks, and will abort execution
+ (e.g. during leak search).
</para>
</listitem>
<listitem>
<para>
- <varname>VALGRIND_MEMPOOL_METAPOOL</varname>.
- This indicates that memory that has been
- marked as being allocated with
- <varname>VALGRIND_MALLOCLIKE_BLOCK</varname> is used
- by a custom allocator to pass out memory to an application (again
- marked with <varname>VALGRIND_MALLOCLIKE_BLOCK</varname>).
- Without this option, such overlapping memory blocks may trigger
- a fatal error message in memcheck.
+ <varname>VALGRIND_MEMPOOL_AUTO_FREE</varname>. Such a meta
+ pool can also be marked as an 'auto free' pool using the
+ flag <varname>VALGRIND_MEMPOOL_AUTO_FREE</varname>, which
+ must be OR-ed together with
+ the <varname>VALGRIND_MEMPOOL_METAPOOL</varname>. For an
+ 'auto free' pool, <varname>VALGRIND_MEMPOOL_FREE</varname>
+ will automatically free the second level blocks that are
+ contained inside the first level block freed
+ with <varname>VALGRIND_MEMPOOL_FREE</varname>. In other
+ words, calling <varname>VALGRIND_MEMPOOL_FREE</varname> will
+ cause implicit calls
+ to <varname>VALGRIND_FREELIKE_BLOCK</varname> for all the
+ second level blocks included in the first level block.
+ Note: it is an error to use
+ the <varname>VALGRIND_MEMPOOL_AUTO_FREE</varname> flag
+ without the
+ <varname>VALGRIND_MEMPOOL_METAPOOL</varname> flag.
</para>
</listitem>
</itemizedlist>
Modified: trunk/memcheck/mc_malloc_wrappers.c
==============================================================================
--- trunk/memcheck/mc_malloc_wrappers.c (original)
+++ trunk/memcheck/mc_malloc_wrappers.c Sat Oct 15 13:59:04 2016
@@ -711,12 +711,17 @@
{
MC_Mempool* mp;
- if (VG_(clo_verbosity) > 2) {
+ if (VG_(clo_verbosity) > 2 || (auto_free && !metapool)) {
VG_(message)(Vg_UserMsg,
- "create_mempool(0x%lx, rzB=%u, zeroed=%d, autofree=%d, metapool=%d)\n",
- pool, rzB, is_zeroed, auto_free, metapool);
+ "create_mempool(0x%lx, rzB=%u, zeroed=%d,"
+ " autofree=%d, metapool=%d)\n",
+ pool, rzB, is_zeroed,
+ auto_free, metapool);
VG_(get_and_pp_StackTrace)
(VG_(get_running_tid)(), MEMPOOL_DEBUG_STACKTRACE_DEPTH);
+ if (auto_free && !metapool)
+ VG_(tool_panic)("Inappropriate use of mempool:"
+ " an auto free pool must be a meta pool. Aborting\n");
}
mp = VG_(HT_lookup)(MC_(mempool_list), (UWord)pool);
Modified: trunk/memcheck/tests/Makefile.am
==============================================================================
--- trunk/memcheck/tests/Makefile.am (original)
+++ trunk/memcheck/tests/Makefile.am Sat Oct 15 13:59:04 2016
@@ -153,13 +153,11 @@
leak-pool-0.vgtest leak-pool-0.stderr.exp \
leak-pool-1.vgtest leak-pool-1.stderr.exp \
leak-pool-2.vgtest leak-pool-2.stderr.exp \
- leak-pool-3.vgtest leak-pool-3.stderr.exp \
leak-pool-4.vgtest leak-pool-4.stderr.exp \
leak-pool-5.vgtest leak-pool-5.stderr.exp \
leak-autofreepool-0.vgtest leak-autofreepool-0.stderr.exp \
leak-autofreepool-1.vgtest leak-autofreepool-1.stderr.exp \
leak-autofreepool-2.vgtest leak-autofreepool-2.stderr.exp \
- leak-autofreepool-3.vgtest leak-autofreepool-3.stderr.exp \
leak-autofreepool-4.vgtest leak-autofreepool-4.stderr.exp \
leak-autofreepool-5.vgtest leak-autofreepool-5.stderr.exp \
leak-autofreepool-6.vgtest leak-autofreepool-6.stderr.exp \
Removed: trunk/memcheck/tests/leak-autofreepool-3.stderr.exp
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool-3.stderr.exp (original)
+++ trunk/memcheck/tests/leak-autofreepool-3.stderr.exp (removed)
@@ -1,10 +0,0 @@
-
-
-HEAP SUMMARY:
- in use at exit: ... bytes in ... blocks
- total heap usage: ... allocs, ... frees, ... bytes allocated
-
-All heap blocks were freed -- no leaks are possible
-
-For counts of detected and suppressed errors, rerun with: -v
-ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Removed: trunk/memcheck/tests/leak-autofreepool-3.vgtest
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool-3.vgtest (original)
+++ trunk/memcheck/tests/leak-autofreepool-3.vgtest (removed)
@@ -1,4 +0,0 @@
-prog: leak-autofreepool
-vgopts: --leak-check=full --show-possibly-lost=no --track-origins=yes
-args: 3
-stderr_filter: filter_allocs
Modified: trunk/memcheck/tests/leak-autofreepool.c
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool.c (original)
+++ trunk/memcheck/tests/leak-autofreepool.c Sat Oct 15 13:59:04 2016
@@ -7,7 +7,7 @@
#include "../memcheck.h"
-// Test VALGRIND_CREATE_META_MEMPOOL features, the VALGRIND_MEMPOOL_METAPOOL and
+// Test VALGRIND_CREATE_MEMPOOL_EXT features, the VALGRIND_MEMPOOL_METAPOOL and
// VALGRIND_MEMPOOL_AUTO_FREE flags.
// Also show that without these, having a custom allocator that:
// - Allocates a MEMPOOL
@@ -64,7 +64,7 @@
void create_meta_pool (void)
{
- VALGRIND_CREATE_META_MEMPOOL(MetaPool, 0, 0, MetaPoolFlags);
+ VALGRIND_CREATE_MEMPOOL_EXT(MetaPool, 0, 0, MetaPoolFlags);
VALGRIND_MEMPOOL_ALLOC(MetaPool, MetaBlock, POOL_BLOCK_SIZE);
MetaPool->buf = (uint8_t *) MetaBlock;
@@ -124,7 +124,7 @@
static void set_flags ( int n )
{
switch (n) {
- // Case 0: No special flags. VALGRIND_CREATE_META_MEMPOOL is same as
+ // Case 0: No special flags. VALGRIND_CREATE_MEMPOOL_EXT is same as
// VALGRIND_CREATE_MEMPOOL.
// When mempools are destroyed, the METAPOOL leaks because auto-free is
// missing. Must show 2*N (20) leaks.
@@ -148,13 +148,15 @@
// Same as before, but now the MALLOCLIKE blocks are auto-freed.
// Must show 0 leaks.
case 2:
- MetaPoolFlags = VALGRIND_MEMPOOL_AUTO_FREE | VALGRIND_MEMPOOL_METAPOOL;
+ MetaPoolFlags = VALGRIND_MEMPOOL_METAPOOL | VALGRIND_MEMPOOL_AUTO_FREE;
CleanupBeforeExit = 1;
break;
- case 3:
- // Just auto-free, with cleanup. The cleanup removes the overlapping
- // blocks, so this is the same as case 2: No leaks, no problems.
+ case 3: // Note: this is incorrect behaviour, and aborts valgrind.
+ // (so it is not exercised during regression testing).
+ // Just auto-free, not marked with meta pool flag.
+ // This is an error, and will cause valgrind to abort when the pool
+ // is created.
MetaPoolFlags = VALGRIND_MEMPOOL_AUTO_FREE;
CleanupBeforeExit = 1;
break;
@@ -185,7 +187,7 @@
// already done above) is by allocating lots of other chunks that are
// NOT part of the pool so the MC_Alloc lists contain other stuff.
// That will make the iterator find stuff AND skip stuff.
- MetaPoolFlags = VALGRIND_MEMPOOL_AUTO_FREE | VALGRIND_MEMPOOL_METAPOOL;
+ MetaPoolFlags = VALGRIND_MEMPOOL_METAPOOL | VALGRIND_MEMPOOL_AUTO_FREE;
CleanupBeforeExit = 1;
GenerateNoise = 1;
break;
@@ -314,9 +316,9 @@
pool_block_size = nr_elts * sizeof(struct cell) + sizeof(uint8_t) + 1;
// Create perf meta pool
- VALGRIND_CREATE_META_MEMPOOL
+ VALGRIND_CREATE_MEMPOOL_EXT
(&perf_meta_pool, 0, 0,
- VALGRIND_MEMPOOL_AUTO_FREE | VALGRIND_MEMPOOL_METAPOOL);
+ VALGRIND_MEMPOOL_METAPOOL | VALGRIND_MEMPOOL_AUTO_FREE);
perf_meta_block = malloc(pool_block_size);
VALGRIND_MEMPOOL_ALLOC(&perf_meta_pool, perf_meta_block,
|
|
From: <sv...@va...> - 2016-10-15 09:30:47
|
Author: philippe
Date: Sat Oct 15 10:30:39 2016
New Revision: 16041
Log:
fix 369468 Remove quadratic metapool alg. using VG_(HT_remove_at_Iter)(VgHashTable *table)
Based on a patch from Ruurd Beerstra
but reworked VG_(HT_remove_at_Iter) so that
the function is implemented without touching the rest of m_hashtable.c
to ensure no performance impact on other hash table usages.
Testing with
for f in 1 2 3 4 5 6 7 8 9; do echo $f; time ./vg-in-place -q ./memcheck/tests/leak-autofreepool 2 $(expr $f \* 100000); done|&grep user
With the patch :
user 0m0.524s
user 0m0.660s
user 0m0.784s
user 0m0.916s
user 0m1.064s
user 0m1.192s
user 0m1.316s
user 0m1.496s
user 0m1.632s
Without the patch, the same gives:
user 0m4.464s
user 0m16.776s
user 0m24.472s
user 1m5.544s
user 1m21.168s
user 1m40.500s
user 1m54.884s
user 4m58.308s
user 5m34.060s
Added:
trunk/memcheck/tests/leak-autofreepool-6.stderr.exp
trunk/memcheck/tests/leak-autofreepool-6.vgtest
Modified:
trunk/NEWS
trunk/coregrind/m_hashtable.c
trunk/include/pub_tool_hashtable.h
trunk/memcheck/mc_malloc_wrappers.c
trunk/memcheck/tests/Makefile.am
trunk/memcheck/tests/leak-autofreepool.c
Modified: trunk/NEWS
==============================================================================
--- trunk/NEWS (original)
+++ trunk/NEWS Sat Oct 15 10:30:39 2016
@@ -179,6 +179,7 @@
369000 AMD64 fma4 instructions unsupported.
361253 [s390x] ex_clone.c:42: undefined reference to `pthread_create'
369169 ppc64 fails jm_int_isa_2_07 test
+369175 jm_vec_isa_2_07 test crashes on ppc64
369209 valgrind loops and eats up all memory if cwd doesn't exist.
369356 pre_mem_read_sockaddr syscall wrapper can crash with bad sockaddr
369359 msghdr_foreachfield can crash when handling bad iovec
@@ -190,12 +191,14 @@
369441 bad lvec argument crashes process_vm_readv/writev syscall wrappers
369446 valgrind crashes on unknown fcntl command
369439 S390x: Unhandled insns RISBLG/RISBHG and LDE/LDER
-369175 jm_vec_isa_2_07 test crashes on ppc64
+369468 Remove quadratic metapool alg. using VG_(HT_remove_at_Iter)
+ (VgHashTable *table)
370265 ISA 3.0 HW cap stuff needs updating
n-i-bz Fix incorrect (or infinite loop) unwind on RHEL7 x86 and amd64
n-i-bz massif --pages-as-heap=yes does not report peak caused by mmap+munmap
-n-i-bz false positive leaks due to aspacemgr merging non heap segments with heap segments.
+n-i-bz false positive leaks due to aspacemgr merging non heap segments
+ with heap segments.
n-i-bz Fix ppoll_alarm exclusion on OS X
n-i-bz Document brk segment limitation, reference manual in limit reached msg.
n-i-bz Fix clobber list in none/tests/amd64/xacq_xrel.c [valgrind r15737]
Modified: trunk/coregrind/m_hashtable.c
==============================================================================
--- trunk/coregrind/m_hashtable.c (original)
+++ trunk/coregrind/m_hashtable.c Sat Oct 15 10:30:39 2016
@@ -365,7 +365,9 @@
vg_assert(table);
/* See long comment on HT_Next prototype in pub_tool_hashtable.h.
In short if this fails, it means the caller tried to modify the
- table whilst iterating over it, which is a bug. */
+ table whilst iterating over it, which is a bug.
+ One exception: HT_remove_at_Iter can remove the current entry and
+ leave the iterator in a valid state for HT_Next. */
vg_assert(table->iterOK);
if (table->iterNode && table->iterNode->next) {
@@ -383,6 +385,37 @@
return NULL;
}
+void VG_(HT_remove_at_Iter)(VgHashTable *table)
+{
+ vg_assert(table);
+ vg_assert(table->iterOK);
+ vg_assert(table->iterNode);
+
+ const UInt curChain = table->iterChain - 1; // chain of iterNode.
+
+
+ if (table->chains[curChain] == table->iterNode) {
+ /* iterNode is the first of its chain -> remove it from the chain. */
+ table->chains[curChain] = table->iterNode->next;
+ /* Setup the iterator to visit first node of curChain: */
+ table->iterNode = NULL;
+ table->iterChain = curChain;
+ } else {
+ /* iterNode is somewhere inside curChain chain */
+ VgHashNode* prev = table->chains[curChain];
+
+ while (prev->next != table->iterNode)
+ prev = prev->next;
+ /* Remove iterNode from the chain. */
+ prev->next = table->iterNode->next;
+ /* Setup the iterator to visit prev->next, which is the node
+ that was after the deleted node. */
+ table->iterNode = prev;
+ }
+
+ table->n_elements--;
+}
+
void VG_(HT_destruct)(VgHashTable *table, void(*freenode_fn)(void*))
{
UInt i;
Modified: trunk/include/pub_tool_hashtable.h
==============================================================================
--- trunk/include/pub_tool_hashtable.h (original)
+++ trunk/include/pub_tool_hashtable.h Sat Oct 15 10:30:39 2016
@@ -121,6 +121,14 @@
assurance. */
extern void* VG_(HT_Next) ( VgHashTable *table );
+/* Remove the element pointed to by the iterator and leave the iterator
+ in a state where VG_(HT_Next) will return the element just after the removed
+ node.
+ This allows removing elements from the table whilst iterating over it.
+ Note that removing an entry does not resize the hash table, making this
+ safe. */
+extern void VG_(HT_remove_at_Iter)( VgHashTable *table );
+
/* Destroy a table and deallocates the memory used by the nodes using
freenode_fn.*/
extern void VG_(HT_destruct) ( VgHashTable *table, void(*freenode_fn)(void*) );
Modified: trunk/memcheck/mc_malloc_wrappers.c
==============================================================================
--- trunk/memcheck/mc_malloc_wrappers.c (original)
+++ trunk/memcheck/mc_malloc_wrappers.c Sat Oct 15 10:30:39 2016
@@ -681,7 +681,6 @@
{
MC_Chunk *mc;
ThreadId tid;
- Bool found;
tl_assert(mp->auto_free);
@@ -693,23 +692,18 @@
tid = VG_(get_running_tid)();
- do {
- found = False;
+ VG_(HT_ResetIter)(MC_(malloc_list));
+ while ( (mc = VG_(HT_Next)(MC_(malloc_list))) ) {
+ if (mc->data >= StartAddr && mc->data + mc->szB <= EndAddr) {
+ if (VG_(clo_verbosity) > 2) {
+ VG_(message)(Vg_UserMsg, "Auto-free of 0x%lx size=%lu\n",
+ mc->data, (mc->szB + 0UL));
+ }
- VG_(HT_ResetIter)(MC_(malloc_list));
- while (!found && (mc = VG_(HT_Next)(MC_(malloc_list))) ) {
- if (mc->data >= StartAddr && mc->data + mc->szB <= EndAddr) {
- if (VG_(clo_verbosity) > 2) {
- VG_(message)(Vg_UserMsg, "Auto-free of 0x%lx size=%lu\n",
- mc->data, (mc->szB + 0UL));
- }
-
- mc = VG_(HT_remove) ( MC_(malloc_list), (UWord) mc->data);
- die_and_free_mem(tid, mc, mp->rzB);
- found = True;
- }
+ VG_(HT_remove_at_Iter)(MC_(malloc_list));
+ die_and_free_mem(tid, mc, mp->rzB);
}
- } while (found);
+ }
}
void MC_(create_mempool)(Addr pool, UInt rzB, Bool is_zeroed,
Modified: trunk/memcheck/tests/Makefile.am
==============================================================================
--- trunk/memcheck/tests/Makefile.am (original)
+++ trunk/memcheck/tests/Makefile.am Sat Oct 15 10:30:39 2016
@@ -162,6 +162,7 @@
leak-autofreepool-3.vgtest leak-autofreepool-3.stderr.exp \
leak-autofreepool-4.vgtest leak-autofreepool-4.stderr.exp \
leak-autofreepool-5.vgtest leak-autofreepool-5.stderr.exp \
+ leak-autofreepool-6.vgtest leak-autofreepool-6.stderr.exp \
leak-tree.vgtest leak-tree.stderr.exp \
leak-segv-jmp.vgtest leak-segv-jmp.stderr.exp \
lks.vgtest lks.stdout.exp lks.supp lks.stderr.exp \
Added: trunk/memcheck/tests/leak-autofreepool-6.stderr.exp
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool-6.stderr.exp (added)
+++ trunk/memcheck/tests/leak-autofreepool-6.stderr.exp Sat Oct 15 10:30:39 2016
@@ -0,0 +1,10 @@
+
+
+HEAP SUMMARY:
+ in use at exit: ... bytes in ... blocks
+ total heap usage: ... allocs, ... frees, ... bytes allocated
+
+All heap blocks were freed -- no leaks are possible
+
+For counts of detected and suppressed errors, rerun with: -v
+ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Added: trunk/memcheck/tests/leak-autofreepool-6.vgtest
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool-6.vgtest (added)
+++ trunk/memcheck/tests/leak-autofreepool-6.vgtest Sat Oct 15 10:30:39 2016
@@ -0,0 +1,4 @@
+prog: leak-autofreepool
+vgopts: --leak-check=full --show-possibly-lost=no --track-origins=yes
+args: 6
+stderr_filter: filter_overlaperror
Modified: trunk/memcheck/tests/leak-autofreepool.c
==============================================================================
--- trunk/memcheck/tests/leak-autofreepool.c (original)
+++ trunk/memcheck/tests/leak-autofreepool.c Sat Oct 15 10:30:39 2016
@@ -43,16 +43,22 @@
#define N 10
#define POOL_BLOCK_SIZE 4096
+#define NOISE_SIZE 256
+
// For easy testing, the plain mempool uses N allocations, the
// metapool 2 * N (so 10 reported leaks are from the plain pool, 20 must be
-// from the metapool.
+// from the metapool).
static int MetaPoolFlags = 0;
static int CleanupBeforeExit = 0;
+static int GenerateNoise = 0;
+static int NoiseCounter = 0;
static struct cell *cells_plain[2 * N];
static struct cell *cells_meta[2 * N];
+static unsigned char *noise[3 * N];
+
static char PlainBlock[POOL_BLOCK_SIZE];
static char MetaBlock[POOL_BLOCK_SIZE];
@@ -106,7 +112,7 @@
void *a = p->buf + p->used;
assert(p->used + n < p->allocated);
- // And this is custom allocator that knows what it is allocating from a pool.
+ // And this is custom allocator that knows that it is allocating from a pool.
VALGRIND_MEMPOOL_ALLOC(p, a, n);
p->used += n;
@@ -172,11 +178,49 @@
CleanupBeforeExit = 0;
break;
+ case 6:
+ // Test the VG_(HT_remove_at_Iter)() function, which removes a chunk
+ // from a hashlist without the need to reset the iterator. The pool
+ // is auto_freed, and the best test for the function (besides the ones
+ // already done above) is by allocating lots of other chunks that are
+ // NOT part of the pool so the MC_Alloc lists contain other stuff.
+ // That will make the iterator find stuff AND skip stuff.
+ MetaPoolFlags = VALGRIND_MEMPOOL_AUTO_FREE | VALGRIND_MEMPOOL_METAPOOL;
+ CleanupBeforeExit = 1;
+ GenerateNoise = 1;
+ break;
+
default:
assert(0);
}
}
+static void GenerateNoisyBit (void)
+{
+ // In case the HT_remove_at_Iter messes up the administration, the wrong
+ // blocks may be deleted from the list, making access to these noise-blocks
+ // invalid. So fill 256-byte blocks with easily tested contents.
+
+ noise[NoiseCounter] = malloc(NOISE_SIZE);
+ assert(noise[NoiseCounter] != NULL);
+ memset(noise[NoiseCounter],(unsigned char) (NoiseCounter % 256), NOISE_SIZE);
+ NoiseCounter++;
+}
+
+static void CheckNoiseContents (void)
+{
+ int i;
+
+ for (i = 0; i < NoiseCounter; i++) {
+ unsigned char Check = (unsigned char) ( i % 256);
+ int j;
+
+ for (j = 0; j < NOISE_SIZE; j++) {
+ assert(noise[i][j] == Check);
+ }
+ }
+}
+
int main( int argc, char** argv )
{
int arg;
@@ -195,11 +239,17 @@
// N plain allocs
for (i = 0; i < N; ++i) {
cells_plain[i] = allocate_plain_style(PlainPool,sizeof(struct cell));
+
+ if (GenerateNoise)
+ GenerateNoisyBit();
}
// 2*N meta allocs
for (i = 0; i < 2 * N; ++i) {
cells_meta[i] = allocate_meta_style(MetaPool,sizeof(struct cell));
+
+ if (GenerateNoise)
+ GenerateNoisyBit();
}
// Leak the memory from the pools by losing the pointers.
@@ -211,18 +261,43 @@
cells_meta[i] = NULL;
}
+ if (GenerateNoise)
+ CheckNoiseContents();
+
// This must free MALLOCLIKE allocations from the pool when
// VALGRIND_MEMPOOL_AUTO_FREE
// is set for the pool and report leaks when not.
+
if (CleanupBeforeExit) {
VALGRIND_MEMPOOL_FREE(MetaPool, MetaBlock);
- VALGRIND_DESTROY_MEMPOOL(MetaPool);
+
+ if (GenerateNoise)
+ CheckNoiseContents();
+
+ VALGRIND_DESTROY_MEMPOOL(MetaPool);
+
+ if (GenerateNoise)
+ CheckNoiseContents();
+
}
// Cleanup.
VALGRIND_DESTROY_MEMPOOL(PlainPool);
- // Perf test
+ if (GenerateNoise)
+ CheckNoiseContents();
+
+ // Try to trigger an error in the bookkeeping by freeing the noise bits.
+ // Valgrind should report no leaks, and zero memory in use. If the
+ // new HT_remove_at_Iter function would corrupt the bookkeeping in any
+ // way, this should bring it out!
+ if (GenerateNoise) {
+ for (i = 0; i < NoiseCounter; i++)
+ free(noise[i]);
+ }
+
+
+ // Perf test
if (argc == 3) {
struct pool perf_plain_pool;
void *perf_plain_block;
|