|
From: Jason E. <ja...@ca...> - 2008-03-21 23:28:47
|
Firefox recently started embedding a custom memory allocator (jemalloc),
but Firefox developers still want to be able to use valgrind, so I added
the necessary VALGRIND_{MALLOC,FREE}LIKE_BLOCK() macro calls to make
this work (sans redzones). This all went well except for peculiar
errors that I suspect are indicative of a problem within valgrind.
Early on during jemalloc initialization, a low level data structure is
allocated internally. This data structure happens to start 64 bytes
past the beginning of a page, and it is 3432 bytes (on amd64). I see
many errors of the following nature, both 'invalid read' and 'invalid
write' errors. These errors are always within the last 64 bytes of the
allocated region (even when I turn off a compile-time flag, which
shrinks the data structure by a few hundred bytes).
==16589== Invalid read of size 8
==16589== at 0x5CF3DF3: arena_malloc (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x5CF640F: malloc (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x64F5AC5: JS_ArenaAllocate (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x6544511: NewRENode (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x654B6F9: ParseQuantifier (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x654C333: ParseRegExp (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x654C851: js_NewRegExp (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x654CF59: js_NewRegExpObject (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x653E87C: PrimaryExpr (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x653F6C1: MemberExpr (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x653FCFF: UnaryExpr (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== by 0x653FDD0: MulExpr (in
/home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
==16589== Address 0x421ED80 is 3,392 bytes inside a block of size 3,432
alloc'd
==16589== at 0x5CF1744: base_alloc (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x5CF4D9A: arenas_extend (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x5CF5348: malloc_init (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x5CF62DD: calloc (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x6FD954A: _dlerror_run (dlerror.c:142)
==16589== by 0x6FD8EF0: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==16589== by 0x6DB269F: _PR_InitZones (in
/home/jasone/mozilla/mozilla/obj_opt/nsprpub/pr/src/libnspr4.so)
==16589== by 0x6DB7501: _PR_ImplicitInitialization (in
/home/jasone/mozilla/mozilla/obj_opt/nsprpub/pr/src/libnspr4.so)
==16589== by 0x6DAD834: PR_NewLogModule (in
/home/jasone/mozilla/mozilla/obj_opt/nsprpub/pr/src/libnspr4.so)
==16589== by 0x5C3C665:
__static_initialization_and_destruction_0(int, int) (in
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x5CF6585: (within
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
==16589== by 0x546EADA: (within
/home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
I tried increasing the alignment of internal allocations in order to
avoid this problem. If this large object is 64 or 128 bytes past the
beginning of a page boundary, the problem persists. If it is any power
of two in [256..4096] past a page boundary, the problem goes away.
Does anyone familiar with valgrind's internals have some idea of what
the problem is? My guess is that objects of this size are usually more
strictly aligned, and that I'm hitting an edge case that other
allocators happen to avoid.
Thanks,
Jason Evans
|
|
From: Nicholas N. <nj...@cs...> - 2008-03-22 00:38:08
|
On Fri, 21 Mar 2008, Jason Evans wrote:
> Firefox recently started embedding a custom memory allocator (jemalloc),
> but Firefox developers still want to be able to use valgrind, so I added
> the necessary VALGRIND_{MALLOC,FREE}LIKE_BLOCK() macro calls to make
> this work (sans redzones). This all went well except for peculiar
> errors that I suspect are indicative of a problem within valgrind.
>
> Early on during jemalloc initialization, a low level data structure is
> allocated internally. This data structure happens to start 64 bytes
> past the beginning of a page, and it is 3432 bytes (on amd64). I see
> many errors of the following nature, both 'invalid read' and 'invalid
> write' errors. These errors are always within the last 64 bytes of the
> allocated region (even when I turn off a compile-time flag, which
> shrinks the data structure by a few hundred bytes).
>
> ==16589== Invalid read of size 8
> ==16589== at 0x5CF3DF3: arena_malloc (in
> /home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
> ==16589== by 0x5CF640F: malloc (in
> /home/jasone/mozilla/mozilla/obj_opt/toolkit/library/libxul.so)
> ==16589== by 0x64F5AC5: JS_ArenaAllocate (in
> /home/jasone/mozilla/mozilla/obj_opt/js/src/libmozjs.so)
>
> [...]
>
> I tried increasing the alignment of internal allocations in order to
> avoid this problem. If this large object is 64 or 128 bytes past the
> beginning of a page boundary, the problem persists. If it is any power
> of two in [256..4096] past a page boundary, the problem goes away.
>
> Does anyone familiar with valgrind's internals have some idea of what
> the problem is? My guess is that objects of this size are usually more
> strictly aligned, and that I'm hitting an edge case that other
> allocators happen to avoid.
I can't think of any problems off the top of my head. Valgrind's allocator
doesn't consider pages in any way, as far as I know, so I don't see why the
page boundary location would have an effect.
It could be a Valgrind bug. Can you make a small test case, eg. by using
jemalloc in a small C program? The {MALLOC,FREE}_LIKE_BLOCK stuff has
always been a little flaky.
Nick
|
|
From: Christoph B. <bar...@or...> - 2008-03-22 10:10:30
|
Am Samstag, 22. März 2008 schrieb Jason Evans: > I tried increasing the alignment of internal allocations in order to > avoid this problem. If this large object is 64 or 128 bytes past the > beginning of a page boundary, the problem persists. If it is any power > of two in [256..4096] past a page boundary, the problem goes away. > > Does anyone familiar with valgrind's internals have some idea of what > the problem is? My guess is that objects of this size are usually more > strictly aligned, and that I'm hitting an edge case that other > allocators happen to avoid. Do you have the traceback with debug information and the code that produces the erorr? You could check right after allocation from the system that the whole range is accessible with valgrinds macros. Christoph |
|
From: Jason E. <ja...@ca...> - 2008-03-22 15:47:17
|
Christoph Bartoschek wrote: > Do you have the traceback with debug information and the code that produces > the erorr? You could check right after allocation from the system that the > whole range is accessible with valgrinds macros. Thank you for your suggestion; it helped me find my mistake. Valgrind did have the whole range marked as accessible, and writes/reads worked fine to begin with. The problem was that in another place I was calling MALLOC_FREELIKE_BLOCK(ptr, size), but the second argument is redzone size, not allocation size. This deallocated object was close enough to the large object for the erroneous redzone to overlap. Whoops. Thank you everyone for your help. Jason |
|
From: Bart V. A. <bar...@gm...> - 2008-03-22 16:23:48
|
On Sat, Mar 22, 2008 at 4:47 PM, Jason Evans <ja...@ca...> wrote: > Christoph Bartoschek wrote: > > Do you have the traceback with debug information and the code that produces > > the erorr? You could check right after allocation from the system that the > > whole range is accessible with valgrinds macros. > > Thank you for your suggestion; it helped me find my mistake. Valgrind > did have the whole range marked as accessible, and writes/reads worked > fine to begin with. The problem was that in another place I was calling > MALLOC_FREELIKE_BLOCK(ptr, size), but the second argument is redzone > size, not allocation size. This deallocated object was close enough to > the large object for the erroneous redzone to overlap. Whoops. Hello Jason, Will the Valgrind client requests be included in the official Firefox source code distribution ? Firefox is an interesting application to test Valgrind tools, and having these client requests in the official source distribution would help us (the Valgrind developers). Bart. |
|
From: Jason E. <ja...@ca...> - 2008-03-22 16:56:27
|
Bart Van Assche wrote: > Will the Valgrind client requests be included in the official Firefox > source code distribution ? Firefox is an interesting application to > test Valgrind tools, and having these client requests in the official > source distribution would help us (the Valgrind developers). Yes, I think the valgrind client request code will be included in the Firefox source, but it will be necessary to build a custom Firefox binary with the --with-valgrind configure option specified, since the calls will not be enabled by default. The feature addition is being tracked here: https://bugzilla.mozilla.org/show_bug.cgi?id=424040 Jason |
|
From: Bart V. A. <bar...@gm...> - 2008-03-24 10:48:22
Attachments:
jemalloc-support-for-valgrind.patch
|
On Sat, Mar 22, 2008 at 12:06 AM, Jason Evans <ja...@ca...> wrote:
> Firefox recently started embedding a custom memory allocator (jemalloc),
> but Firefox developers still want to be able to use valgrind, so I added
> the necessary VALGRIND_{MALLOC,FREE}LIKE_BLOCK() macro calls to make
> this work (sans redzones).
Hello Jason,
Is jemalloc always loaded as a shared library in Firefox (and not
linked statically) ? In that case there is an alternative to inserting
client requests in the Firefox source code, namely applying the
attached patch to the Valgrind source code. This patch makes Valgrind
recognize libjemalloc's memory allocation functions.
Bart.
|
|
From: Julian S. <js...@ac...> - 2008-03-24 11:57:34
|
On Monday 24 March 2008 11:48, Bart Van Assche wrote:
> On Sat, Mar 22, 2008 at 12:06 AM, Jason Evans <ja...@ca...> wrote:
> > Firefox recently started embedding a custom memory allocator (jemalloc),
> > but Firefox developers still want to be able to use valgrind, so I added
> > the necessary VALGRIND_{MALLOC,FREE}LIKE_BLOCK() macro calls to make
> > this work (sans redzones).
>
> Hello Jason,
>
> Is jemalloc always loaded as a shared library in Firefox (and not
> linked statically) ? In that case there is an alternative to inserting
> client requests in the Firefox source code, namely applying the
> attached patch to the Valgrind source code. This patch makes Valgrind
> recognize libjemalloc's memory allocation functions.
Does jemalloc have semantics beyond standard malloc/free?
I had the impression that it allows multiple independent allocation
pools.
J
|
|
From: Jason E. <ja...@ca...> - 2008-03-29 15:01:10
|
Julian Seward wrote:
> On Monday 24 March 2008 11:48, Bart Van Assche wrote:
>> On Sat, Mar 22, 2008 at 12:06 AM, Jason Evans <ja...@ca...> wrote:
>>> Firefox recently started embedding a custom memory allocator (jemalloc),
>>> but Firefox developers still want to be able to use valgrind, so I added
>>> the necessary VALGRIND_{MALLOC,FREE}LIKE_BLOCK() macro calls to make
>>> this work (sans redzones).
>> Hello Jason,
>>
>> Is jemalloc always loaded as a shared library in Firefox (and not
>> linked statically) ? In that case there is an alternative to inserting
>> client requests in the Firefox source code, namely applying the
>> attached patch to the Valgrind source code. This patch makes Valgrind
>> recognize libjemalloc's memory allocation functions.
Loading jemalloc as a separate shared library impacts startup speed, so
I think the current goal is to statically link it into firefox-bin.
Right now jemalloc is linked into libxul (a big shared library where
lots of mozilla functionality resides), but that causes some bad
interactions with dlopen.
One problem with the VALGRIND_*_BLOCK() macros is that there's no way to
support in-place realloc. If valgrind were patched to recognize
jemalloc, would in-place realloc cause valgrind any problems?
> Does jemalloc have semantics beyond standard malloc/free?
> I had the impression that it allows multiple independent allocation
> pools.
jemalloc implements malloc/calloc/posix_memalign/realloc/free. It does
use multiple arenas internally for multi-threaded applications, but the
arena API is not currently exposed.
Thanks,
Jason
|