|
From: Doug R. <df...@nl...> - 2003-12-27 14:40:37
|
Since I've had a bit of festive spare time available, I thought I would attempt to port valgrind 2.1.0 to FreeBSD-5.x. After a bit of a struggle, two falls and a submission, I have most of it working well enough to pass most of the tests (with the notable exception of the pthread stuff which I haven't even tried to port yet). Is there any interest here in merging support for FreeBSD? Most of the syscall hairy bits should also help to get NetBSD and OpenBSD going too. I have ended up with a bit of an ifdef tangle in a few places but that might be best managed by simply forking a few of the source files (vg_kerneliface.h, vg_syscalls.c and similar). My patch is currently about 130k (as a unified diff against 2.1.0. Anyway, whether you want the port or not, I'd like to thank the valgrind developers for an excellent tool which I've used to find lots of ugly problems in my own programs on Linux (and now BSD as well). |
|
From: Dirk M. <dm...@gm...> - 2003-12-27 17:04:04
|
On Saturday 27 December 2003 15:40, Doug Rabson wrote: > Is there any interest here in merging support for FreeBSD? Absolutely yes. > Most of the > syscall hairy bits should also help to get NetBSD and OpenBSD going too. > I have ended up with a bit of an ifdef tangle in a few places but that > might be best managed by simply forking a few of the source files > (vg_kerneliface.h, vg_syscalls.c and similar). My patch is currently > about 130k (as a unified diff against 2.1.0. Yes. ideally we should start with a directory structure and move the linux specific files in a subdir, and fork them for other operating systems or architectures. which files did you have to modify most? vg_syscalls and kerneliface, anything else? Dirk |
|
From: Doug R. <df...@nl...> - 2003-12-27 17:54:00
|
On Sat, 2003-12-27 at 17:02, Dirk Mueller wrote: > On Saturday 27 December 2003 15:40, Doug Rabson wrote: > > > Is there any interest here in merging support for FreeBSD? > > Absolutely yes. > > > Most of the > > syscall hairy bits should also help to get NetBSD and OpenBSD going too. > > I have ended up with a bit of an ifdef tangle in a few places but that > > might be best managed by simply forking a few of the source files > > (vg_kerneliface.h, vg_syscalls.c and similar). My patch is currently > > about 130k (as a unified diff against 2.1.0. > > Yes. ideally we should start with a directory structure and move the linux > specific files in a subdir, and fork them for other operating systems or > architectures. > > which files did you have to modify most? vg_syscalls and kerneliface, anything > else? Most of the really serious hacking was in vg_mylibc, vg_proxylwp, vg_signals, vg_syscalls and vg_kerneliface. Most of the rest of the changes are simple things (like working around the lack of ksa_restorer). |
|
From: Robert W. <rj...@du...> - 2003-12-28 11:31:37
|
> Most of the really serious hacking was in vg_mylibc, vg_proxylwp, > vg_signals, vg_syscalls and vg_kerneliface. Most of the rest of the > changes are simple things (like working around the lack of > ksa_restorer). vg_mylibc should start going away real soon now. FV pretty much took care of that. Regards, Robert. --=20 Robert Walsh Amalgamated Durables, Inc. - "We don't make the things you buy." Email: rj...@du... |
|
From: Doug R. <df...@nl...> - 2003-12-27 21:38:48
|
On Sat, 2003-12-27 at 20:59, Robert Walsh wrote: > > Most of the really serious hacking was in vg_mylibc, vg_proxylwp, > > vg_signals, vg_syscalls and vg_kerneliface. Most of the rest of the > > changes are simple things (like working around the lack of > > ksa_restorer). > > vg_mylibc should start going away real soon now. FV pretty much took > care of that. Yeah, I figured that out from reading the mailing list archives and cvs logs. I wish I had started hacking post-FV though - I'll have to merge forwards a little :-( |
|
From: Julian S. <js...@ac...> - 2003-12-28 13:15:19
|
On Saturday 27 December 2003 17:02, Dirk Mueller wrote: > > Most of the > > syscall hairy bits should also help to get NetBSD and OpenBSD going too. > > I have ended up with a bit of an ifdef tangle in a few places but that > > might be best managed by simply forking a few of the source files > > (vg_kerneliface.h, vg_syscalls.c and similar). My patch is currently > > about 130k (as a unified diff against 2.1.0. > > Yes. ideally we should start with a directory structure and move the linux > specific files in a subdir, and fork them for other operating systems or > architectures. I agree with Dirk. Ports are of course a good thing, but we need structure. Can you make a proposal for how to adjust the directory structure so as to accommodate these x86 unixes as cleanly/modularly as possible? Then we can think about adding the FreeBSD stuff to the cvs. Solaris-x86 is another potential target, btw. Thanks, J |
|
From: Doug R. <df...@nl...> - 2003-12-28 14:43:25
|
On Sun, 2003-12-28 at 14:32, Julian Seward wrote: > On Saturday 27 December 2003 17:02, Dirk Mueller wrote: > > > > Most of the > > > syscall hairy bits should also help to get NetBSD and OpenBSD going too. > > > I have ended up with a bit of an ifdef tangle in a few places but that > > > might be best managed by simply forking a few of the source files > > > (vg_kerneliface.h, vg_syscalls.c and similar). My patch is currently > > > about 130k (as a unified diff against 2.1.0. > > > > Yes. ideally we should start with a directory structure and move the linux > > specific files in a subdir, and fork them for other operating systems or > > architectures. > > I agree with Dirk. Ports are of course a good thing, but we need > structure. Can you make a proposal for how to adjust the directory > structure so as to accommodate these x86 unixes as cleanly/modularly > as possible? Then we can think about adding the FreeBSD stuff to > the cvs. Solaris-x86 is another potential target, btw. It seems to me that the simplest approach would be to move the really os-dependant files into a subdir of coregrind (e.g. coregrind/linux). Files which seem to be in this category are vg_libpthread*.c, vg_procselfmaps.c, vg_proxylwp.c, vg_signals.c (maybe), vg_syscall.S, vg_syscalls.c, vg_unistd.h, vg_unsafe.h. Similarly in include, vg_kerneliface.h would move to include/linux/vg_kerneliface.h. > Thanks, > > J > > > ------------------------------------------------------- > This SF.net email is sponsored by: IBM Linux Tutorials. > Become an expert in LINUX or just sharpen your skills. Sign up for IBM's > Free Linux Tutorials. Learn everything from the bash shell to sys admin. > Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click > _______________________________________________ > Valgrind-developers mailing list > Val...@li... > https://lists.sourceforge.net/lists/listinfo/valgrind-developers |
|
From: Jeremy F. <je...@go...> - 2003-12-28 21:52:19
|
On Sun, 2003-12-28 at 06:43, Doug Rabson wrote: > It seems to me that the simplest approach would be to move the really > os-dependant files into a subdir of coregrind (e.g. coregrind/linux). > Files which seem to be in this category are vg_libpthread*.c, > vg_procselfmaps.c, vg_proxylwp.c, vg_signals.c (maybe), vg_syscall.S, > vg_syscalls.c, vg_unistd.h, vg_unsafe.h. Similarly in include, > vg_kerneliface.h would move to include/linux/vg_kerneliface.h. Yes, but I think we should take the time to break it down further. Some of those files are just Linux-dependent (like most of proxylwp.c, most of vg_syscalls.c, procselfmaps.c, etc), and some are both linux-x86 dependent (*.S, vg_kerneliface.c, and so on). Is FreeBSD x86 only, or has it been ported to other architectures (I know the other BSDs are widely ported). Do you know if all the BSDs have, for example, common syscall numbers across architectures? I'd be really interested if you can update the port for the FV world. I'd like to know where the new non-portable bits are. For example, how does FreeBSD lay out the user address space, and how much is reserved for the kernel? J |
|
From: Doug R. <df...@nl...> - 2003-12-29 09:30:40
|
On Sun, 2003-12-28 at 21:52, Jeremy Fitzhardinge wrote: > On Sun, 2003-12-28 at 06:43, Doug Rabson wrote: > > It seems to me that the simplest approach would be to move the really > > os-dependant files into a subdir of coregrind (e.g. coregrind/linux). > > Files which seem to be in this category are vg_libpthread*.c, > > vg_procselfmaps.c, vg_proxylwp.c, vg_signals.c (maybe), vg_syscall.S, > > vg_syscalls.c, vg_unistd.h, vg_unsafe.h. Similarly in include, > > vg_kerneliface.h would move to include/linux/vg_kerneliface.h. > > Yes, but I think we should take the time to break it down further. Some > of those files are just Linux-dependent (like most of proxylwp.c, most > of vg_syscalls.c, procselfmaps.c, etc), and some are both linux-x86 > dependent (*.S, vg_kerneliface.c, and so on). Right. I noticed that you have already created an x86 subdirectory - perhaps os subdirectories off that would be useful. > > Is FreeBSD x86 only, or has it been ported to other architectures (I > know the other BSDs are widely ported). Do you know if all the BSDs > have, for example, common syscall numbers across architectures? FreeBSD supports x86, amd64, sparc64, ia64, alpha and powerpc right now (with varying degrees of completenes). FreeBSD uses common syscall numbers for all architectures but it does have a different syscall ABI from e.g. NetBSD. > > I'd be really interested if you can update the port for the FV world. I'm going to do that next now that I have the pthreads stuff mostly working. > I'd like to know where the new non-portable bits are. For example, how > does FreeBSD lay out the user address space, and how much is reserved > for the kernel? Thats configurable but most people don't change the default configuration which is 0-3G user and 3-4G kernel. For ia32 emulation on amd64 and ia64 systems, the ia32 userland gets the full 0-4G address space. Large memory (>4G) systems are often configured with just 2G for userland but we probably don't need to worry about them. |
|
From: Johan R. <jry...@ni...> - 2003-12-29 11:06:30
|
Doug Rabson <df...@nl...> wrote: : Since I've had a bit of festive spare time available, I thought I would : attempt to port valgrind 2.1.0 to FreeBSD-5.x. Are there any plans what so ever to port V to other architectures than IA-32? It isn't a small task, I know that, but it would be great to have V running for example SPARC binaries. It would most probably also increase the user base of V. -- Johan Rydberg, Free Software Developer, Sweden http://rtmk.sf.net | http://www.nongnu.org/guss/ |
|
From: Doug R. <df...@nl...> - 2004-01-02 11:57:10
|
On Mon, 2003-12-29 at 09:30, Doug Rabson wrote: > > > > I'd be really interested if you can update the port for the FV world. > > I'm going to do that next now that I have the pthreads stuff mostly > working. I've updated the port to work with the post FV valgrind. You can get a patch against today's CVS from http://people.freebsd.org/~dfr/valgrind-20040102-dfr.diff. The only really dodgy bits in this patch are in stage1.c where I had to stub out the stack alignment bits. The code couldn't code when the alignment offset wasn't exactly zero because it assumed that the new aux entries would fit exactly into the gap. I also had problems moving the brk() up past the end of stage2 so I punted on that and just overrode brk() and sbrk() instead. I had problems with vg_signals.c for the async signal handlers. Currently FreeBSD has a bug (which will be fixed RSN) with sigaltstack that means that all threads share the same stack setting. This meant that async signals (intended for the proxylwp) were being delivered on the signal stack instead of the proxylwp stack. I just changed the code to not set SA_ONSTACK for those signals. Attaching GDB doesn't work very well with this port since we have no equivalent of /proc/self/fd. I guess the code could be changed to remember the exec filename and pass that instead of /proc/self/fd/$d. This also affected the implementation of VG_(resolve_filename) which I worked around by using the file descriptor tracking code to lookup the filename. I haven't tested this patch with a Linux box - my RH9 machine is several miles away and switched off for the holidays. I've tried to keep things decently conditional but there are almost certainly one or two nits. |
|
From: Doug R. <df...@nl...> - 2004-01-03 12:26:25
|
On Fri, 2004-01-02 at 11:56, Doug Rabson wrote: > On Mon, 2003-12-29 at 09:30, Doug Rabson wrote: > > > > > > I'd be really interested if you can update the port for the FV world. > > > > I'm going to do that next now that I have the pthreads stuff mostly > > working. > > I've updated the port to work with the post FV valgrind. You can get a > patch against today's CVS from > http://people.freebsd.org/~dfr/valgrind-20040102-dfr.diff. I've just uploaded another patch, http://people.freebsd.org/~dfr/valgrind-20040103-dfr.diff which fixes a few minor problems and implements a few more syscalls. |
|
From: Nicholas N. <nj...@ca...> - 2004-01-03 13:23:13
|
On Sat, 27 Dec 2003, Doug Rabson wrote: > Since I've had a bit of festive spare time available, I thought I would > attempt to port valgrind 2.1.0 to FreeBSD-5.x. After a bit of a > struggle, two falls and a submission, I have most of it working well > enough to pass most of the tests (with the notable exception of the > pthread stuff which I haven't even tried to port yet). Well done! A few people have tried this, but it sounds like you've got a lot further than anyone else. Would you be able to write some kind of summary describing the changes you had to make? That would be very useful for people to get a grip on what you've done. N |
|
From: Doug R. <df...@nl...> - 2004-01-03 13:39:18
|
On Sat, 2004-01-03 at 13:23, Nicholas Nethercote wrote: > On Sat, 27 Dec 2003, Doug Rabson wrote: > > > Since I've had a bit of festive spare time available, I thought I would > > attempt to port valgrind 2.1.0 to FreeBSD-5.x. After a bit of a > > struggle, two falls and a submission, I have most of it working well > > enough to pass most of the tests (with the notable exception of the > > pthread stuff which I haven't even tried to port yet). > > Well done! A few people have tried this, but it sounds like you've got a > lot further than anyone else. Would you be able to write some kind of > summary describing the changes you had to make? That would be very useful > for people to get a grip on what you've done. I summarised most of the changes in a later message and you can get a patch against today's cvs at http://people.freebsd.org/~dfr/valgrind-20040103-dfr.diff. The most fiddly part was getting syscalls to work. FreeBSD (and probably all 4.4BSD derived systems) has a quite different syscall ABI with arguments on the stack and error-returns signalled with the carry flag. Add to that the fact that some (not all) syscalls return an extra 32bits in %edx and things get a bit tricky in vg_syscalls.c. Still, it helped that I've been hacking the FreeBSD kernel for about eight years and understand it about as well as anyone :-) |
|
From: Nicholas N. <nj...@ca...> - 2004-01-03 18:06:08
|
On Sat, 3 Jan 2004, Doug Rabson wrote: > > Well done! A few people have tried this, but it sounds like you've got a > > lot further than anyone else. Would you be able to write some kind of > > summary describing the changes you had to make? That would be very useful > > for people to get a grip on what you've done. > > I summarised most of the changes in a later message and you can get a > patch against today's cvs at > http://people.freebsd.org/~dfr/valgrind-20040103-dfr.diff. Er, which message? I only see a couple of brief descriptions... I was thinking more along the lines of a page or two of summary -- more work for you, I realise, but much easier for everyone else to understand than a 130KB diff... > The most fiddly part was getting syscalls to work. FreeBSD (and probably > all 4.4BSD derived systems) has a quite different syscall ABI with > arguments on the stack and error-returns signalled with the carry flag. > Add to that the fact that some (not all) syscalls return an extra 32bits > in %edx and things get a bit tricky in vg_syscalls.c. That's exactly the sort of thing I'd like to see in a summary (I don't think you mentioned this previously)... what are the details of "things get a bit tricky"? N |
|
From: Doug R. <df...@nl...> - 2004-01-03 18:49:03
|
On Sat, 2004-01-03 at 18:06, Nicholas Nethercote wrote: > On Sat, 3 Jan 2004, Doug Rabson wrote: > > > > Well done! A few people have tried this, but it sounds like you've got a > > > lot further than anyone else. Would you be able to write some kind of > > > summary describing the changes you had to make? That would be very useful > > > for people to get a grip on what you've done. > > > > I summarised most of the changes in a later message and you can get a > > patch against today's cvs at > > http://people.freebsd.org/~dfr/valgrind-20040103-dfr.diff. > > Er, which message? I only see a couple of brief descriptions... I was > thinking more along the lines of a page or two of summary -- more work for > you, I realise, but much easier for everyone else to understand than a > 130KB diff... Its earlier in this very thread. I'll just quote the relavent part: The only really dodgy bits in this patch are in stage1.c where I had to stub out the stack alignment bits. The code couldn't code when the alignment offset wasn't exactly zero because it assumed that the new aux entries would fit exactly into the gap. I also had problems moving the brk() up past the end of stage2 so I punted on that and just overrode brk() and sbrk() instead. I had problems with vg_signals.c for the async signal handlers. Currently FreeBSD has a bug (which will be fixed RSN) with sigaltstack that means that all threads share the same stack setting. This meant that async signals (intended for the proxylwp) were being delivered on the signal stack instead of the proxylwp stack. I just changed the code to not set SA_ONSTACK for those signals. Attaching GDB doesn't work very well with this port since we have no equivalent of /proc/self/fd. I guess the code could be changed to remember the exec filename and pass that instead of /proc/self/fd/$d. This also affected the implementation of VG_(resolve_filename) which I worked around by using the file descriptor tracking code to lookup the filename. > > > The most fiddly part was getting syscalls to work. FreeBSD (and probably > > all 4.4BSD derived systems) has a quite different syscall ABI with > > arguments on the stack and error-returns signalled with the carry flag. > > Add to that the fact that some (not all) syscalls return an extra 32bits > > in %edx and things get a bit tricky in vg_syscalls.c. > > That's exactly the sort of thing I'd like to see in a summary (I don't > think you mentioned this previously)... what are the details of "things > get a bit tricky"? Well the part which executes the system call has to be changed to account for the extra return values (eax, edx and eflags instead of just eax). It also needs to be able to preserve the value of edx for those syscalls which don't change its value. All the bits of code through the system which assume the linux-style error return value of -errno need to be tweaked to account for the BSD-style error return which has errno in eax and eflags.C set. Most of that can be hidden by a macro so: res = -VKI_EINVAL; changes to #define seterror(e) do {tst->m_eax = e; tst->m_eflags |= EFlagC;} while(0) ... seterror(VKI_EINVAL); |
|
From: Jeremy F. <je...@go...> - 2004-01-03 22:30:49
|
On Fri, 2004-01-02 at 03:56, Doug Rabson wrote: > I've updated the port to work with the post FV valgrind. You can get a > patch against today's CVS from > http://people.freebsd.org/~dfr/valgrind-20040102-dfr.diff. That's great! I'll have a look at your latest patch in detail later today. > The only really dodgy bits in this patch are in stage1.c where I had to > stub out the stack alignment bits. The code couldn't code when the ^^^^ cope? > alignment offset wasn't exactly zero because it assumed that the new aux > entries would fit exactly into the gap. Do you mean the stuff in fix_auxv? > I also had problems moving the > brk() up past the end of stage2 so I punted on that and just overrode > brk() and sbrk() instead. Is that because FreeBSD doesn't overcommit by default? The reason for trying to get the brk base into the right place for stage2 is so that stage2 can use libraries which want to use brk (eg, libc malloc). If you override brk in stage2, will that definitely override all possible ways the brk() syscall can be called? Because it could get pretty disastrous if brk(2) accidentally got called. Also, in FreeBSD, is sbrk a syscall or just a library function? Also, what does FreeBSD's RLIMIT_DATA govern? In Linux it is effectively useless, because it only governs the brk syscall (and a.out executable BSS size). This is useless because glibc will always fall back to using mmap() if brk fails, so if brk fails because of RLIMIT_DATA, it will still keep allocating memory. I was thinking of taking advantage of this by setting RLIMIT_DATA to 0 so that I can be sure that brk(2) will always fail and not cause problems. > I had problems with vg_signals.c for the async signal handlers. > Currently FreeBSD has a bug (which will be fixed RSN) with sigaltstack > that means that all threads share the same stack setting. This meant > that async signals (intended for the proxylwp) were being delivered on > the signal stack instead of the proxylwp stack. I just changed the code > to not set SA_ONSTACK for those signals. That sounds OK. > Attaching GDB doesn't work very well with this port since we have no > equivalent of /proc/self/fd. I guess the code could be changed to > remember the exec filename and pass that instead of /proc/self/fd/$d. Yes, I was planning something like that anyway, cause all the /proc stuff is pretty non-portable. That said, the GDB attach stuff is broken anyway, because it's pretty hard to get right (it's very hard for a program to attach gdb to itself, and set its own state up into something which gdb wants to examine). In particular, the state of %ebp gets trashed as part of the exec of gdb, so gdb doesn't see a sane backtrace at the moment. > This also affected the implementation of VG_(resolve_filename) which I > worked around by using the file descriptor tracking code to lookup the > filename. For just the cases where the fd was obviously created out of a name? > I haven't tested this patch with a Linux box - my RH9 machine is several > miles away and switched off for the holidays. I've tried to keep things > decently conditional but there are almost certainly one or two nits. I think the best thing to do is use this as a basis for factoring all the OS-specific code into OS-specific files. J |
|
From: Doug R. <df...@nl...> - 2004-01-04 10:43:10
|
On Sat, 2004-01-03 at 22:30, Jeremy Fitzhardinge wrote: > On Fri, 2004-01-02 at 03:56, Doug Rabson wrote: > > I've updated the port to work with the post FV valgrind. You can get a > > patch against today's CVS from > > http://people.freebsd.org/~dfr/valgrind-20040102-dfr.diff. > > That's great! I'll have a look at your latest patch in detail later > today. > > > The only really dodgy bits in this patch are in stage1.c where I had to > > stub out the stack alignment bits. The code couldn't code when the > ^^^^ cope? yes :-) > > alignment offset wasn't exactly zero because it assumed that the new aux > > entries would fit exactly into the gap. > > Do you mean the stuff in fix_auxv? Thats right. My FreeBSD system started valgrind with a stack that wasn't aligned to 16 bytes and this confused fix_auxv enough to corrupt the auxv entries given to stage2. > > > I also had problems moving the > > brk() up past the end of stage2 so I punted on that and just overrode > > brk() and sbrk() instead. > > Is that because FreeBSD doesn't overcommit by default? The reason for > trying to get the brk base into the right place for stage2 is so that > stage2 can use libraries which want to use brk (eg, libc malloc). If > you override brk in stage2, will that definitely override all possible > ways the brk() syscall can be called? Because it could get pretty > disastrous if brk(2) accidentally got called. Also, in FreeBSD, is sbrk > a syscall or just a library function? The main problem (I think) is that the default data size limit for FreeBSD is only 0.5G. We do overcommit by default but moving the brk this way generates valid mappings (lazy allocated zero-filled mappings but still valid) for the entire ~3G address space and this is greater than the limit. Overriding brk, sbrk (and even malloc itself) is perfectly safe in FreeBSD and it should override all ways that the syscall can be called apart from a pathological use of syscall(3). I made sure that stage2 was able to call malloc safely with these changes although there was a problem with non-trivial use of malloc because stage2's copy of libc.so ended up being loaded very close to the end of stage2 which didn't leave much spare to expand the 'brk'. The implementation of brk and sbrk in FreeBSD is a hybrid of library and syscall. The current and minimum break values are handled in user code and new break locations are given to the kernel via the syscall. Its very simple and easy to understand if you read the code (src/lib/libc/i386/sys/brk.S and src/lib/libc/i386/sys/sbrk.S in any FreeBSD source tree). > > Also, what does FreeBSD's RLIMIT_DATA govern? In Linux it is > effectively useless, because it only governs the brk syscall (and a.out > executable BSS size). This is useless because glibc will always fall > back to using mmap() if brk fails, so if brk fails because of > RLIMIT_DATA, it will still keep allocating memory. I was thinking of > taking advantage of this by setting RLIMIT_DATA to 0 so that I can be > sure that brk(2) will always fail and not cause problems. I think that RLIMIT_DATA in FreeBSD governs the virtual size of the brk-managed region. The libc implementation will not fall back to mmap if sbrk (or brk) fails. > > > I had problems with vg_signals.c for the async signal handlers. > > Currently FreeBSD has a bug (which will be fixed RSN) with sigaltstack > > that means that all threads share the same stack setting. This meant > > that async signals (intended for the proxylwp) were being delivered on > > the signal stack instead of the proxylwp stack. I just changed the code > > to not set SA_ONSTACK for those signals. > > That sounds OK. > > > Attaching GDB doesn't work very well with this port since we have no > > equivalent of /proc/self/fd. I guess the code could be changed to > > remember the exec filename and pass that instead of /proc/self/fd/$d. > > Yes, I was planning something like that anyway, cause all the /proc > stuff is pretty non-portable. That said, the GDB attach stuff is broken > anyway, because it's pretty hard to get right (it's very hard for a > program to attach gdb to itself, and set its own state up into something > which gdb wants to examine). In particular, the state of %ebp gets > trashed as part of the exec of gdb, so gdb doesn't see a sane backtrace > at the moment. > > > This also affected the implementation of VG_(resolve_filename) which I > > worked around by using the file descriptor tracking code to lookup the > > filename. > > For just the cases where the fd was obviously created out of a name? Thats right. I also made it unconditional for dup-like calls where the original file descriptor might have had a name. > > > I haven't tested this patch with a Linux box - my RH9 machine is several > > miles away and switched off for the holidays. I've tried to keep things > > decently conditional but there are almost certainly one or two nits. > > I think the best thing to do is use this as a basis for factoring all > the OS-specific code into OS-specific files. Right - I think that process has already been started. |
|
From: Jeremy F. <je...@go...> - 2004-01-04 22:05:37
|
On Sun, 2004-01-04 at 02:42, Doug Rabson wrote: > Thats right. My FreeBSD system started valgrind with a stack that wasn't > aligned to 16 bytes and this confused fix_auxv enough to corrupt the > auxv entries given to stage2. Hm, I wonder if that code is ever needed... > The main problem (I think) is that the default data size limit for > FreeBSD is only 0.5G. We do overcommit by default but moving the brk > this way generates valid mappings (lazy allocated zero-filled mappings > but still valid) for the entire ~3G address space and this is greater > than the limit. Yep. Then they all get unmapped again immediately afterwards. Would it help to make a loop which increases brk by a bit, unmaps the pages, etc until brk is at the right place? > Overriding brk, sbrk (and even malloc itself) is perfectly safe in > FreeBSD and it should override all ways that the syscall can be called > apart from a pathological use of syscall(3). I made sure that stage2 was > able to call malloc safely with these changes although there was a > problem with non-trivial use of malloc because stage2's copy of libc.so > ended up being loaded very close to the end of stage2 which didn't leave > much spare to expand the 'brk'. Hm, yes, the kernel gets to place it, so I guess it could do that. Can we convince it to place it below stage2 (which is what happens on Linux)? > I think that RLIMIT_DATA in FreeBSD governs the virtual size of the > brk-managed region. The libc implementation will not fall back to mmap > if sbrk (or brk) fails. OK. And I presume brk/sbrk are not allowed to grow the brk segment beyond the next mapping up. > > I think the best thing to do is use this as a basis for factoring all > > the OS-specific code into OS-specific files. > > Right - I think that process has already been started. Hm. After looking at the patch in a bit more detail, the changes are smaller than I would have guessed. I'm pleased to see that almost all of vg_syscalls.c is common. I definitely don't want to duplicate common stuff, nor do I want to have lots of ifdefs, so the division will need to be more fine-grained than I first thought. I didn't really look at the differences between vg_libpthread.c and vg_libpthread_freebsd.c, but they seemed pretty similar. Why the split? Other things: does FreeBSD have the same sysinfo page mechanism as Linux now? And which version of FreeBSD should I download and install to try your stuff out? Would it be the New Technology release 5.1? 5.2? J |
|
From: Doug R. <df...@nl...> - 2004-01-05 09:36:01
|
On Sun, 2004-01-04 at 22:05, Jeremy Fitzhardinge wrote: > On Sun, 2004-01-04 at 02:42, Doug Rabson wrote: > > Thats right. My FreeBSD system started valgrind with a stack that wasn't > > aligned to 16 bytes and this confused fix_auxv enough to corrupt the > > auxv entries given to stage2. > > Hm, I wonder if that code is ever needed... I'm not sure. I do know that the FreeBSD kernel doesn't try to align stacks properly before execve even for platforms that need special alignment. We rely on the C startup code (which we provide along with libc itself) to apply any extra alignment before calling main. > > > The main problem (I think) is that the default data size limit for > > FreeBSD is only 0.5G. We do overcommit by default but moving the brk > > this way generates valid mappings (lazy allocated zero-filled mappings > > but still valid) for the entire ~3G address space and this is greater > > than the limit. > > Yep. Then they all get unmapped again immediately afterwards. Would it > help to make a loop which increases brk by a bit, unmaps the pages, etc > until brk is at the right place? Probably not. The relavent kernel code (src/sys/vm/vm_unix.c if you want to read it) just subtracts the new brk value from the 'data section base' that the ELF loader set and compares it to RLIMIT_DATA. It care whether or not the pages are mapped. > > > Overriding brk, sbrk (and even malloc itself) is perfectly safe in > > FreeBSD and it should override all ways that the syscall can be called > > apart from a pathological use of syscall(3). I made sure that stage2 was > > able to call malloc safely with these changes although there was a > > problem with non-trivial use of malloc because stage2's copy of libc.so > > ended up being loaded very close to the end of stage2 which didn't leave > > much spare to expand the 'brk'. > > Hm, yes, the kernel gets to place it, so I guess it could do that. Can > we convince it to place it below stage2 (which is what happens on > Linux)? Actually, its /libexec/ld-elf.so.1 which gets to place libc.so and valgrind can control its placement the way it does now - by mapping other stuff over places it needs to avoid. > > > I think that RLIMIT_DATA in FreeBSD governs the virtual size of the > > brk-managed region. The libc implementation will not fall back to mmap > > if sbrk (or brk) fails. > > OK. And I presume brk/sbrk are not allowed to grow the brk segment > beyond the next mapping up. Correct. > > > > I think the best thing to do is use this as a basis for factoring all > > > the OS-specific code into OS-specific files. > > > > Right - I think that process has already been started. > > Hm. After looking at the patch in a bit more detail, the changes are > smaller than I would have guessed. I'm pleased to see that almost all > of vg_syscalls.c is common. I definitely don't want to duplicate common > stuff, nor do I want to have lots of ifdefs, so the division will need > to be more fine-grained than I first thought. There are a fair number of differences in the syscall ABIs but there was definately a lot of common ground there. I currently get a load of 'defined but not used' warnings for linux syscalls which aren't in the FreeBSD tables. Maybe separating the syscall pre and post stubs from the tables and dispatch code would be sufficient? > > I didn't really look at the differences between vg_libpthread.c and > vg_libpthread_freebsd.c, but they seemed pretty similar. Why the split? The main difference between linuxthreads and FreeBSD pthreads is that where in Linux, e.g. a mutex is declared: typedef struct pthread_mutex pthread_mutex_t; in FreeBSD it would be declared: typedef struct pthread_mutex *pthread_mutex_t; The reason for this is to remove sizeof(pthread_mutex) from the pthreads ABI which makes it easy to substitute different pthreads implementations without rebuilding client code. Currently I know of at least four pthreads implementations that use the same ABI. Coping with this difference was getting very clumsy with several ifdef sections in almost every function. Also the glibc-specific stuff ended up changing quite a bit too. In the end it seemed easier to fork the code. > > > Other things: does FreeBSD have the same sysinfo page mechanism as Linux > now? I'm not familiar with sysinfo at all, so probably not. > > And which version of FreeBSD should I download and install to try your > stuff out? Would it be the New Technology release 5.1? 5.2? My test system here is approximately a 5.2 prerelease. I would recommend installing 5.2-RC2. |