Menu
▾
▴
valgrind-users
|
From: Bastien C. <ba...@ch...> - 2003-04-09 13:45:07
|
Hello there,
I have serious troubles with possible memory leaks in programs heavily
using the STL. I am not able to tell whether it is a problem with the
compiler, the STL or wrong valgrind output, so I'll start here before
filing in a gcc bug report.
One of my programs grew larger and larger without I knew why, so took
valgrind to look and started building testcases to find out what was
happening. I have a SuSE 8.1 distribution, that's kernel 2.4.x,
glibc2.2 and gcc version 3.2
Consider this test case example:
-------------------------------------------------------
#include <vector>
#include <ext/hash_map>
using namespace __gnu_cxx;
void f1()
{
int n=3D42;
vector<int> v;
for(int i=3D0; i<1000000; i++) v.push_back(n);
}
int main(){
f1();
return 0;
}
----------------------------------------------------------
g++ -g -o test test.C=20
and then=20
valgrind --leak-resolution=3Dhigh --num-callers=3D20 --show-reachable=3Dy=
es --leak-check=3Dyes ./test
I will get the following summary:
----------------------------------------------------------
=3D=3D16880=3D=3D LEAK SUMMARY:
=3D=3D16880=3D=3D definitely lost: 16 bytes in 1 blocks.
=3D=3D16880=3D=3D possibly lost: 0 bytes in 0 blocks.
=3D=3D16880=3D=3D still reachable: 6912 bytes in 4 blocks.
----------------------------------------------------------
The number which troubles me ist the bytes that are still
reachable. Here's, the detail:
----------------------------------------------------------
=3D=3D19169=3D=3D 6848 bytes in 3 blocks are still reachable in loss reco=
rd 3 of 3
=3D=3D19169=3D=3D at 0x4015DE3B: __builtin_new (vg_clientfuncs.c:129)
=3D=3D19169=3D=3D by 0x4015DE76: operator new(unsigned) (vg_clientfunc=
s.c:142)
=3D=3D19169=3D=3D by 0x40278E00: std::__default_alloc_template<true, 0=
>::_S_chunk_alloc(unsigned, int&) (in /usr/lib/libstdc++.so.5.0.0)
=3D=3D19169=3D=3D by 0x40278D1C: std::__default_alloc_template<true, 0=
>::_S_refill(unsigned) (in /usr/lib/libstdc++.so.5.0.0)
=3D=3D19169=3D=3D by 0x402788EF: std::__default_alloc_template<true, 0=
>::allocate(unsigned) (in /usr/lib/libstdc++.so.5.0.0)
=3D=3D19169=3D=3D by 0x8049008: std::__simple_alloc<int, std::__defaul=
t_alloc_template<true, 0> >::allocate(unsigned) (/usr/include/g++/bits/st=
l_alloc.h:224)
=3D=3D19169=3D=3D by 0x8048D7E: std::_Vector_alloc_base<int, std::allo=
cator<int>, true>::_M_allocate(unsigned) (/usr/include/g++/bits/stl_vecto=
r.h:121)
=3D=3D19169=3D=3D by 0x8048A45: std::vector<int, std::allocator<int> >=
::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::=
allocator<int> > >, int const&) (/usr/include/g++/bits/stl_vector.h:898)
=3D=3D19169=3D=3D by 0x804884C: std::vector<int, std::allocator<int> >=
::push_back(int const&) (/usr/include/g++/bits/stl_vector.h:496)
=3D=3D19169=3D=3D by 0x80486A1: f1() (test2.C:10)
=3D=3D19169=3D=3D by 0x80487A2: main (test2.C:21)
=3D=3D19169=3D=3D by 0x403094A1: __libc_start_main (in /lib/libc.so.6)
=3D=3D19169=3D=3D by 0x8048580: (within /home/bach/work/assembly/htga/=
src/progs/test)
----------------------------------------------------------
Regarding the program above, I sincerely do think that there should be
no leak at all, even not in "reachable" parts.
Now, a few bytes don't hurt. Unfortunately, when I let run my real
program, here's what I get (for really small data sets):
----------------------------------------------------------
=3D=3D698=3D=3D LEAK SUMMARY:
=3D=3D698=3D=3D definitely lost: 24825 bytes in 3492 blocks.
=3D=3D698=3D=3D possibly lost: 1398 bytes in 3 blocks.
=3D=3D698=3D=3D still reachable: 1125492 bytes in 65 blocks.
----------------------------------------------------------
(please note that I don't care about the that the definitely and
possibly lost numbers, these I can trace back to real oversights in my
code.)
The "still reachable" 1M number is about 40 times greater than the
other two numbers added together and I have the distinct impression
that the memory is really eaten away somewhere:=20
1) all valgrind detail messages are more or less similar to the one of
the test case above, all have something to do with containers
2) putting a "while(1)" loop at a distinctive point in my program
where everything should have been more or less freed after some
heavy computation (using about any existing STL container type
that exists with dozens of different classes) gives me remaining
memory footprints of >1G (yes, that's gigabyte).
Now my question: any idea where to start searching? is valgrind at
fault (which I don't think, but one never knows)? the gnu STL? the gnu
g++ compiler?
Any suggestion welcome.=20
Regards,
Bastien
--=20
-- The universe has its own cure for stupidity. --
-- Unfortunately, it doesn't always apply it. --
|
|
From: David E. <da...@2g...> - 2003-04-09 17:37:55
|
On Wed, 2003-04-09 at 15:45, Bastien Chevreux wrote:
> Hello there,
>
> I have serious troubles with possible memory leaks in programs heavily
> using the STL. I am not able to tell whether it is a problem with the
> compiler, the STL or wrong valgrind output, so I'll start here before
> filing in a gcc bug report.
>
> One of my programs grew larger and larger without I knew why, so took
> valgrind to look and started building testcases to find out what was
> happening. I have a SuSE 8.1 distribution, that's kernel 2.4.x,
> glibc2.2 and gcc version 3.2
>
> Consider this test case example:
>
> -------------------------------------------------------
> #include <vector>
> #include <ext/hash_map>
> using namespace __gnu_cxx;
>
> void f1()
> {
> int n=42;
> vector<int> v;
> for(int i=0; i<1000000; i++) v.push_back(n);
> }
>
> int main(){
> f1();
> return 0;
> }
> ----------------------------------------------------------
>
> g++ -g -o test test.C
>
> and then
>
> valgrind --leak-resolution=high --num-callers=20 --show-reachable=yes --leak-check=yes ./test
>
> I will get the following summary:
>
> ----------------------------------------------------------
> ==16880== LEAK SUMMARY:
> ==16880== definitely lost: 16 bytes in 1 blocks.
> ==16880== possibly lost: 0 bytes in 0 blocks.
> ==16880== still reachable: 6912 bytes in 4 blocks.
> ----------------------------------------------------------
>
> The number which troubles me ist the bytes that are still
> reachable. Here's, the detail:
>
> ----------------------------------------------------------
> ==19169== 6848 bytes in 3 blocks are still reachable in loss record 3 of 3
> ==19169== at 0x4015DE3B: __builtin_new (vg_clientfuncs.c:129)
> ==19169== by 0x4015DE76: operator new(unsigned) (vg_clientfuncs.c:142)
> ==19169== by 0x40278E00: std::__default_alloc_template<true, 0>::_S_chunk_alloc(unsigned, int&) (in /usr/lib/libstdc++.so.5.0.0)
> ==19169== by 0x40278D1C: std::__default_alloc_template<true, 0>::_S_refill(unsigned) (in /usr/lib/libstdc++.so.5.0.0)
> ==19169== by 0x402788EF: std::__default_alloc_template<true, 0>::allocate(unsigned) (in /usr/lib/libstdc++.so.5.0.0)
> ==19169== by 0x8049008: std::__simple_alloc<int, std::__default_alloc_template<true, 0> >::allocate(unsigned) (/usr/include/g++/bits/stl_alloc.h:224)
> ==19169== by 0x8048D7E: std::_Vector_alloc_base<int, std::allocator<int>, true>::_M_allocate(unsigned) (/usr/include/g++/bits/stl_vector.h:121)
> ==19169== by 0x8048A45: std::vector<int, std::allocator<int> >::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, std::allocator<int> > >, int const&) (/usr/include/g++/bits/stl_vector.h:898)
> ==19169== by 0x804884C: std::vector<int, std::allocator<int> >::push_back(int const&) (/usr/include/g++/bits/stl_vector.h:496)
> ==19169== by 0x80486A1: f1() (test2.C:10)
> ==19169== by 0x80487A2: main (test2.C:21)
> ==19169== by 0x403094A1: __libc_start_main (in /lib/libc.so.6)
> ==19169== by 0x8048580: (within /home/bach/work/assembly/htga/src/progs/test)
> ----------------------------------------------------------
>
> Regarding the program above, I sincerely do think that there should be
> no leak at all, even not in "reachable" parts.
>
> Now, a few bytes don't hurt. Unfortunately, when I let run my real
> program, here's what I get (for really small data sets):
>
> ----------------------------------------------------------
> ==698== LEAK SUMMARY:
> ==698== definitely lost: 24825 bytes in 3492 blocks.
> ==698== possibly lost: 1398 bytes in 3 blocks.
> ==698== still reachable: 1125492 bytes in 65 blocks.
> ----------------------------------------------------------
>
> (please note that I don't care about the that the definitely and
> possibly lost numbers, these I can trace back to real oversights in my
> code.)
>
> The "still reachable" 1M number is about 40 times greater than the
> other two numbers added together and I have the distinct impression
> that the memory is really eaten away somewhere:
> 1) all valgrind detail messages are more or less similar to the one of
> the test case above, all have something to do with containers
> 2) putting a "while(1)" loop at a distinctive point in my program
> where everything should have been more or less freed after some
> heavy computation (using about any existing STL container type
> that exists with dozens of different classes) gives me remaining
> memory footprints of >1G (yes, that's gigabyte).
>
> Now my question: any idea where to start searching? is valgrind at
> fault (which I don't think, but one never knows)? the gnu STL? the gnu
> g++ compiler?
>
> Any suggestion welcome.
My guess is that the STL allocator keeps the memory around. What happens
if you call f1() several times in main()?
--
-\- David Eriksson -/- www.2GooD.nu
"I personally refuse to use inferior tools because of ideology."
- Linus Torvalds
|
|
From: Bastien C. <ba...@ch...> - 2003-04-09 22:10:20
|
On Wednesday 09 April 2003 19:37, you wrote:
> My guess is that the STL allocator keeps the memory around. What happen=
s
> if you call f1() several times in main()?
No change, at least for this small example. But I continued to play a bit=
with=20
containers my program uses and came out with this gem:
------------------------------------------------
#include<iostream>
#include<deque>
#include<set>
using namespace std;
void f1(int ic)
{
set<char> n;
for(char c=3D'a'; c < 'z'; c++) n.insert(c);
deque<set<char> > v;
for(int i=3D0; i<ic; i++) v.push_back(n);
}
int main(){
f1(1000);
cout << "The memory footprint ..." << endl;
f1(20000);
cout << "... should be ..." << endl;
f1(50000);
cout << "... near zero exactly now! (it isn't *sigh*)" << endl;
//while(1);
return 0;
}
------------------------------------------------
Everyone's invited to let this run on their system ...
------------------------------------------------
=3D=3D13669=3D=3D LEAK SUMMARY:
=3D=3D13669=3D=3D definitely lost: 16 bytes in 1 blocks.
=3D=3D13669=3D=3D possibly lost: 0 bytes in 0 blocks.
=3D=3D13669=3D=3D still reachable: 32568232 bytes in 127 blocks.
=3D=3D13669=3D=3D suppressed: 0 bytes in 0 blocks.
------------------------------------------------
=2E.. and play with it: the numbers get lower when stopping after f1(2000=
) or=20
f1(20000)). Best thing is, when one uncomments the while(1); statement th=
e=20
memory footprint is around 80M (where it shouldn't be much greater than t=
he=20
size of the executable).
I digged the news a bit and found this:=20
http://groups.google.com/groups?hl=3Den&lr=3D&ie=3DUTF-8&oe=3Dutf-8&frame=
=3Dright&th=3D432bcc216e83d78f&seekm=3Dslrnaa3v4d.lt.mixtim_nospam%40taco=
=2Emixtim.ispwest.com#link3
Here's one interesting part:
> The default allocator for many c++ standard libraries (such as the one =
that
> ships with gcc) never actually "frees" memory. It just adds the memory =
back
> to a pool for later use. So, if you allocate a map that contains 80 MB
> worth of data and then destroy the map, your application still has that=
80
> MB allocated and will until your program exits.
On the other hand, after the program exited, valgrind should not find any=
=20
leaks (the STL pool should have been freed, right?). Any ideas?
So the STL is to 'blame'. The description of the pool behaviour should go=
into=20
the FAQ of valgrind, though, I'm sure other people tripped (are tripping,=
=20
will trip) over that too.
And now for something completely related (but going off topic): is there =
any=20
way to "flush" that pool?=20
Regards,
Bastien
PS: Did I already thank the valgrind author? No? I longed for a tool like=
that=20
for Linux since I first worked with purify on a SUN. Thanks a lot.
--=20
-- The universe has its own cure for stupidity. --
-- Unfortunately, it doesn't always apply it. --
|
|
From: Geoff A. <gal...@nc...> - 2003-04-10 02:27:59
|
This appears to be the same libstdc++ "memory leak" I mentioned in the = "still reachable" memory from g++'s std::string thread, which I started = on 31 March 2003. You can check the Valgrind-users archive at = http://sourceforge.net/mailarchive/forum.php?thread_id=3D1908802&forum_id= =3D32038. Note that the thread continues into April, so you'll need to = search both the March and April archives. In one of my later postings, = I give suppression files for suppressing these libstdc++ "still = reachable" messages for both gcc 2.95.2 and 3.2. Note that this is working as intended. There are a number of discussions = on this in the libstdc++ mailing list. For example, see = http://gcc.gnu.org/ml/libstdc++/2002-10/msg00038.html, = http://gcc.gnu.org/ml/libstdc++/2002-08/msg00105.html, and = http://gcc.gnu.org/ml/libstdc++/2002-10/msg00044.html. It's not = considered a bug, but rather part of a memory cache optimization. From = http://gcc.gnu.org/ml/libstdc++/2002-08/msg00105.html: Both libstdc++-v2 and libstdc++-v3 cache memory internally to greatly = aid performance yet, as far as we know, no pointers are ever lost as in = foo(). I believe there are compile options for disabling the libstdc++ memory = cache optimization if you don't like the behavior. But if you do, = expect significant performance degradation on some platforms. Geoff Alexander ----- Original Message -----=20 From: "Bastien Chevreux" <ba...@ch...> To: "valgrind users" <val...@li...> Sent: Wednesday, April 09, 2003 9:45 AM Subject: [Valgrind-users] "Reachable" memory, where's the bug (g++, STL, = valgrind)? Hello there, I have serious troubles with possible memory leaks in programs heavily using the STL. I am not able to tell whether it is a problem with the compiler, the STL or wrong valgrind output, so I'll start here before filing in a gcc bug report. One of my programs grew larger and larger without I knew why, so took valgrind to look and started building testcases to find out what was happening. I have a SuSE 8.1 distribution, that's kernel 2.4.x, glibc2.2 and gcc version 3.2 Consider this test case example: ------------------------------------------------------- #include <vector> #include <ext/hash_map> using namespace __gnu_cxx; void f1() { int n=3D42; vector<int> v; for(int i=3D0; i<1000000; i++) v.push_back(n); } int main(){ f1(); return 0; } ---------------------------------------------------------- g++ -g -o test test.C=20 and then=20 valgrind --leak-resolution=3Dhigh --num-callers=3D20 = --show-reachable=3Dyes --leak-check=3Dyes ./test I will get the following summary: ---------------------------------------------------------- =3D=3D16880=3D=3D LEAK SUMMARY: =3D=3D16880=3D=3D definitely lost: 16 bytes in 1 blocks. =3D=3D16880=3D=3D possibly lost: 0 bytes in 0 blocks. =3D=3D16880=3D=3D still reachable: 6912 bytes in 4 blocks. ---------------------------------------------------------- The number which troubles me ist the bytes that are still reachable. Here's, the detail: ---------------------------------------------------------- =3D=3D19169=3D=3D 6848 bytes in 3 blocks are still reachable in loss = record 3 of 3 =3D=3D19169=3D=3D at 0x4015DE3B: __builtin_new (vg_clientfuncs.c:129) =3D=3D19169=3D=3D by 0x4015DE76: operator new(unsigned) = (vg_clientfuncs.c:142) =3D=3D19169=3D=3D by 0x40278E00: std::__default_alloc_template<true, = 0>::_S_chunk_alloc(unsigned, int&) (in /usr/lib/libstdc++.so.5.0.0) =3D=3D19169=3D=3D by 0x40278D1C: std::__default_alloc_template<true, = 0>::_S_refill(unsigned) (in /usr/lib/libstdc++.so.5.0.0) =3D=3D19169=3D=3D by 0x402788EF: std::__default_alloc_template<true, = 0>::allocate(unsigned) (in /usr/lib/libstdc++.so.5.0.0) =3D=3D19169=3D=3D by 0x8049008: std::__simple_alloc<int, = std::__default_alloc_template<true, 0> >::allocate(unsigned) = (/usr/include/g++/bits/stl_alloc.h:224) =3D=3D19169=3D=3D by 0x8048D7E: std::_Vector_alloc_base<int, = std::allocator<int>, true>::_M_allocate(unsigned) = (/usr/include/g++/bits/stl_vector.h:121) =3D=3D19169=3D=3D by 0x8048A45: std::vector<int, std::allocator<int> = >::_M_insert_aux(__gnu_cxx::__normal_iterator<int*, std::vector<int, = std::allocator<int> > >, int const&) = (/usr/include/g++/bits/stl_vector.h:898) =3D=3D19169=3D=3D by 0x804884C: std::vector<int, std::allocator<int> = >::push_back(int const&) (/usr/include/g++/bits/stl_vector.h:496) =3D=3D19169=3D=3D by 0x80486A1: f1() (test2.C:10) =3D=3D19169=3D=3D by 0x80487A2: main (test2.C:21) =3D=3D19169=3D=3D by 0x403094A1: __libc_start_main (in = /lib/libc.so.6) =3D=3D19169=3D=3D by 0x8048580: (within = /home/bach/work/assembly/htga/src/progs/test) ---------------------------------------------------------- Regarding the program above, I sincerely do think that there should be no leak at all, even not in "reachable" parts. Now, a few bytes don't hurt. Unfortunately, when I let run my real program, here's what I get (for really small data sets): ---------------------------------------------------------- =3D=3D698=3D=3D LEAK SUMMARY: =3D=3D698=3D=3D definitely lost: 24825 bytes in 3492 blocks. =3D=3D698=3D=3D possibly lost: 1398 bytes in 3 blocks. =3D=3D698=3D=3D still reachable: 1125492 bytes in 65 blocks. ---------------------------------------------------------- (please note that I don't care about the that the definitely and possibly lost numbers, these I can trace back to real oversights in my code.) The "still reachable" 1M number is about 40 times greater than the other two numbers added together and I have the distinct impression that the memory is really eaten away somewhere:=20 1) all valgrind detail messages are more or less similar to the one of the test case above, all have something to do with containers 2) putting a "while(1)" loop at a distinctive point in my program where everything should have been more or less freed after some heavy computation (using about any existing STL container type that exists with dozens of different classes) gives me remaining memory footprints of >1G (yes, that's gigabyte). Now my question: any idea where to start searching? is valgrind at fault (which I don't think, but one never knows)? the gnu STL? the gnu g++ compiler? Any suggestion welcome.=20 Regards, Bastien --=20 -- The universe has its own cure for stupidity. -- -- Unfortunately, it doesn't always apply it. -- ------------------------------------------------------- This SF.net email is sponsored by: Etnus, makers of TotalView, The = debugger=20 for complex code. Debugging C/C++ programs can leave you feeling lost = and=20 disoriented. TotalView can help you find your way. Available on major = UNIX=20 and Linux platforms. Try it free. www.etnus.com _______________________________________________ Valgrind-users mailing list Val...@li... https://lists.sourceforge.net/lists/listinfo/valgrind-users |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X