|
From: Mark S. <ms...@cl...> - 2005-05-23 17:05:06
|
Thanks a lot for taking a look at the problem! I should indeed be
checking the result of malloc, I'll make sure to do so in my actual
program. In my case, my machine started swapping heavily before malloc()
started failing.
The actual program was reading 4kb blocks from a unix file descriptor
and appending the blocks to an in-memory buffer. After noticing problems
with that program under valgrind, I played around a bit to distill it
into the test program I sent.
I really should be re-allocating the buffer exponentially rather than
linearly, but haven't made that change yet. Once I do that, I shouldn't
have any problems as the total number of allocations will ~12 rather
than 2000.
Thanks again for the help.
--Mark
-----Original Message-----
From: Nicholas Nethercote [mailto:nj...@cs...]=20
Sent: Saturday, May 21, 2005 12:07 PM
To: Mark Stemm
Cc: val...@li...
Subject: RE: [Valgrind-users] Problem with realloc() + valgrind 2.4.0
(incl. sample program)
On Thu, 19 May 2005, Mark Stemm wrote:
> That version behaves properly, but this modification (walking each=20
> buffer after it's been allocated) does not:
>
> #include <stdlib.h>
> #include <string.h>
>
> int main(int argc, char **argv)
> {
> int len =3D 0;
> char *data =3D NULL;
> int do_copy =3D 1;
> for(int i=3D0; i<2000; i++) {
> len +=3D 4096;
> char *data =3D (char *) malloc(len*sizeof(char));
> for (int j=3D0; j<len; j++) {
> data[j] =3D j;
> }
> free(data);
> }
> return 0;
> }
>
> Some nonscientific analysis (i.e. watching "top" while valgrind is
> running) indicates that the memory usage is initially stable, then=20
> quickly climbs as the buffer gets larger.
I just looked at this. Turns out that under Valgrind, on my machine
(and=20
presumably yours) a lot of those allocations are failing -- you should
be=20
checking the result of malloc.
I think the problem is this: the program allocates a 4KB block, then=20
frees it. Valgrind puts this block on its free list. The program then=20
allocates an 8KB block. There's no block on the free list which is 8KB
or=20
greater, so Valgrind allocates new memory. And so on. So Valgrind's
free=20
list ends up holding blocks of size 4KB, 8KB, 12KB, 16KB, etc. Huge=20
amounts of memory quickly get heldon the free list, but none of the
blocks=20
are ever big enough to satisfy the requests.
Basically it's a pathological case for Valgrind's allocator. Mark, did=20
you come up with your original program (the realloc one) based on=20
behaviour you saw with a real application, or were you just
experimenting?
N
|