|
From: Julian S. <js...@ac...> - 2005-09-18 12:25:16
|
Starting to consider symbol loading on the aspacem branch.
I looked at docs/internals/segments-seginfos.txt, and studied the
associated code a bit, and I think I see what's going on.
What I can't figure out is whether such a complex approach is
really needed. I realise that the question "should we load=20
symbols for the mmap that just happened" can't be answered=20
exactly, because there is no surefire way to distinguish an
mmap done by ld.so(dlopen) vs some random mmap of an object file
which just happens to look indistinguishable. However clever
we are, a sufficiently devious program can always fool us into
not loading debuginfo when we should, or vice versa. =20
I don't feel there's much point in trying to correctly handle
anything other than the bog-standard nothing-weird case.
It would be useful to enumerate the cases which happen in
practice. segments-seginfos.txt contains a couple of comments
in support of the reference counting scheme:
> Similarly, if a Segment gets split (by mprotect, for example), the two
> pieces will still be associated with the same SegInfo.
> If a new mmap appears in the address range of an existing SegInfo
Do either of these happen in practice?
> For example, if the process mmaps the
> first page of an ELF file (the one containing the header), a SegInfo
> will be created for that ELF file's mappings, which will include memory
> which will be later mmaped by the client's ELF loader.
I wasn't clear what is supposed to happen here.
=2D-----------
=46rom peering at various /proc/*/maps files, the following scheme
sounds plausible:
Load symbols following an mmap if:
map is to a file
map has r-x permissions
file has a valid ELF header
possibly: mapping is > 1 page (catches the case of mapping first
page just to examine the header)
If the client wants to subsequently chop up the mapping, or change its
permissions, we ignore that. I have never seen any evidence in proc/*/maps
that ld.so does such things.
=46ollowing an munmap, dump symbols for any previously loaded range
which intersects the unmapped range. Again, this screws up if half an
executable segment is unmapped, and again I'm not sure this happens in
practice.
=46ollowing an mprotect, dump symbols for any previously loaded range
which intersects the protected range AND the new protection does not
include 'x' access.
=2D-----------
Any ideas if something this simple would work (before I find out
the hard way? :-) IIRC 1.0.X had an even more moronic scheme and I
don't remember people complaining about it, although equally it
could have had problems with corner cases / unusual uses=20
that I wasn't aware of.
J
|
|
From: Nicholas N. <nj...@cs...> - 2005-09-18 13:00:31
|
On Sun, 18 Sep 2005, Julian Seward wrote: > Any ideas if something this simple would work (before I find out > the hard way? :-) IIRC 1.0.X had an even more moronic scheme and I > don't remember people complaining about it, although equally it > could have had problems with corner cases / unusual uses > that I wasn't aware of. I don't know how important the corner cases are. I suggest you implement your scheme, and make it abort when it hits a corner case; you'll soon find out how common they are :) Nick |
Julian Seward wrote: >>Similarly, if a Segment gets split (by mprotect, for example), the two >>pieces will still be associated with the same SegInfo. This happens when a debugger inserts a breakpoint, or when ld-linux relocates a module that has DT_TEXTREL, or when a co-resident monitor rewrites some instructions. On x86, a shared lib with relocations to .text "works" just fine. The modified pages are no longer sharable, but the instruction stream is functional. It's even rather common, when a builder forgets to use -fpic for one or more files. It can be done on purpose when the modularity is more important than the page sharing. Non-pic code is faster, too: register %ebx is not dedicated to _GLOBAL_OFFSET_TABLE_ addressing, and global variables can be accessed by [relocated] inline 32-bit offset rather than by address fetched from the GOT. >>If a new mmap appears in the address range of an existing SegInfo On x86_64 the static linker ld inserts a 1MB "hole" between .text and .data. This is on advice from the hardware performance mavens, because various caching+prefetching hardware can look ahead that far. Currently ld-linux leaves this as PROT_NONE, but anybody else is free to override that assignment. > From peering at various /proc/*/maps files, the following scheme > sounds plausible: > > Load symbols following an mmap if: > > map is to a file > map has r-x permissions > file has a valid ELF header > possibly: mapping is > 1 page (catches the case of mapping first > page just to examine the header) > > If the client wants to subsequently chop up the mapping, or change its > permissions, we ignore that. I have never seen any evidence in proc/*/maps > that ld.so does such things. glibc-2.3.5 ld-linux does. It finds the minimum interval of pages which covers the p_memsz of all PT_LOAD, mmap()s that much from the file [even if this maps beyond EOF of the file], then munmap()s [or mprotect(,,PROT_NONE)] everything that is not covered by the first PT_LOAD, then mmap(,,,MAP_FIXED,,) each remaining PT_LOAD. This is done to overcome the possibility that a kernel which randomizes the placement of mmap(0, ...) might place the first PT_LOAD so that subsequent PT_LOAD [must maintain relative addressing to other PT_LOAD from the same file] would evict something else. Needless to say, ld-linux assumes that it is the only actor (well, dlopen() does try for mutual exclusion) and that any "holes" between PT_LOAD from the same module are ignorable as far as allocation is concerned. Also, there is nothing to stop a file from having PT_LOAD that overlap, or appear in non-ascending order, etc. The results might depend on order of processing, but always it has been by order of appearance in the file. [Probably this is a good way to trigger "bugs" in ld-linux and/or the kernel.] Some algorithms and data structures internal to glibc-2.3.5 assume that modules do not overlap. In particular, ld-linux sometimes searches for __builtin_return_address_(0) in a set of intervals in order to determine which shared lib called ld-linux. This matters for dlsym(), dlmopen(), etc., and assumes that the intervals are a disjoint cover of any "legal" callers. ld-linux tries to hide all of this from the prying eyes of anyone else [the internal version of struct link_map contains much more than specified in <link.h>]. Some of this is good because it changes very frequently, but some parts are bad because in the past ld-linux has been slow to provide needed services [such as dl_iterate_phdr()] and even antagonistic towards anybody else trying for peaceful co-existence without the blessing of ld-linux. -- John Reiser, jreiser@BitWagon.com |
|
From: Julian S. <js...@ac...> - 2005-09-18 23:15:23
|
On Sunday 18 September 2005 14:00, Nicholas Nethercote wrote: > On Sun, 18 Sep 2005, Julian Seward wrote: > > Any ideas if something this simple would work (before I find out > > the hard way? :-) IIRC 1.0.X had an even more moronic scheme and I > > don't remember people complaining about it, although equally it > > could have had problems with corner cases / unusual uses > > that I wasn't aware of. > > I don't know how important the corner cases are. I suggest you implement > your scheme, and make it abort when it hits a corner case; you'll soon > find out how common they are :) The simplistic version seems to be working now, and all tools except memcheck run. memcheck is a problem, though. It dies like this at startup: valgrind: m_mallocfree.c:434 (ensure_mm_init): Assertion 'client_redzone_szB == VG_(tdict).tool_client_redzone_szB' failed. I know why, but I don't know how to fix it. It's another circular dependency. The new startup sequence in main() looks more or less like this (yay!): - start debug logger - start address space manager (very early, as desired) - start m_mallocfree (do a test malloc/free) Starting mallocfree early is convenient for all later phases. It then goes on to use m_ume to load the client, builds its stack, loads initial debug info, etc. Around this point it also initialises the tool. Memcheck's initialisation code sets VG_(tdict).tool_client_redzone_szB to some new non-default value, and that causes m_mallocfree to assert, because it specifically checks that the client redzone size doesn't change after the first allocation (fair enough). One solution might be for the core to have a way to ask a tool its client redzone size very early on, way before actually initialising the tool, and then passing that value into m_mallocfree when starting it up. [It might be good to give m_mallocfree an explicit initialisation method rather than having it self-initialising as at present]. This would require a change to the core/tool interface, though. What do you reckon? J |
|
From: Nicholas N. <nj...@cs...> - 2005-09-19 04:49:23
|
On Mon, 19 Sep 2005, Julian Seward wrote: > memcheck is a problem, though. It dies like this at startup: > > valgrind: m_mallocfree.c:434 (ensure_mm_init): > Assertion 'client_redzone_szB == VG_(tdict).tool_client_redzone_szB' failed. > > I know why, but I don't know how to fix it. It's another circular > dependency. The new startup sequence in main() looks more or less > like this (yay!): > > - start debug logger > - start address space manager (very early, as desired) > - start m_mallocfree (do a test malloc/free) > > Starting mallocfree early is convenient for all later phases. > > It then goes on to use m_ume to load the client, builds its stack, > loads initial debug info, etc. Around this point it also initialises > the tool. Memcheck's initialisation code sets > VG_(tdict).tool_client_redzone_szB to some new non-default value, > and that causes m_mallocfree to assert, because it specifically checks > that the client redzone size doesn't change after the first allocation > (fair enough). > > One solution might be for the core to have a way to ask a tool its > client redzone size very early on, way before actually initialising > the tool, and then passing that value into m_mallocfree when > starting it up. [It might be good to give m_mallocfree an explicit > initialisation method rather than having it self-initialising as > at present]. This would require a change to the core/tool interface, > though. What do you reckon? Until recently the tools specified their redzone size with a variable (which overrode the default value thanks to weak symbol magic) for exactly this reason. Then things changed and I made it part of the VG_(needs_malloc_replacement)() call, which is neater because it puts all the things a tool needs to initialise for malloc replacement in a single call. So we could require a malloc-replacing tool to provide a function that returns the redzone size, which the core calls, which would be separate from VG_(needs_malloc_replacement)(). It's easy, but a shame because it's less neat than having a single function. Another possibility is to call the tools VG_(tdict).pre_clo_init function before starting m_mallocfree.c. That would preclude it from allocating memory, though, which is probably less neat than having two functions. Another possibility is to allow the individual arenas in m_mallocfree.c to be initialized not all at the same time, but on demand. So ensure_mm_init(void) would become ensure_mm_init(Arena aid). If the tool tried to set the redzone size with VG_(needs_malloc_replacement)() after having allocated some memory, m_mallocfree could abort saying "you can't do that". I like that solution the best. If you like, just get some hacky workaround working so you can move onto more important things, and I'll implement the nice solution later. Nick |
|
From: Julian S. <js...@ac...> - 2005-09-19 09:37:28
|
> So we could require a malloc-replacing tool to provide a function that > returns the redzone size, which the core calls, which would be separate > from VG_(needs_malloc_replacement)(). It's easy, but a shame because it's > less neat than having a single function. Yes. Simple and moderately grubby. > Another possibility is to call the tools VG_(tdict).pre_clo_init function > before starting m_mallocfree.c. That would preclude it from allocating > memory, though, which is probably less neat than having two functions. Agreed. Significantly less neat, imo. > Another possibility is to allow the individual arenas in m_mallocfree.c to > be initialized not all at the same time, but on demand. So > ensure_mm_init(void) would become ensure_mm_init(Arena aid). If the tool > tried to set the redzone size with VG_(needs_malloc_replacement)() after > having allocated some memory, m_mallocfree could abort saying "you can't > do that". I like that solution the best. That's an interesting idea. > If you like, just get some hacky workaround working so you can move onto > more important things, and I'll implement the nice solution later. I just set the default size to 16 in m_mallocfree, which allowed me to make progress with memcheck last night. I now have it running all the way up to dying in the leak checker. Overall, this new aspacem is turning out to me a mammoth task, but I now expect to have something tryable-outable in the next couple of days. J |