|
From: Paul F. <pj...@wa...> - 2021-02-22 11:02:16
|
Hi all Since 3.17 is starting to be in the air, I'd like to start the ball rolling for integrating FreeBSD upstream. I'm not necessarily targeting 3.17, I'd rather everything be done properly rather than rush anything. In this mail I'll cover the following - FreeBSD background - current status - testing - overview of changes - TODOs FreeBSD background ~~~~~~~~~~~~~ I won't delve into the history of FreeBSD, but just give an overview of the ecosystem. It runs on amd64/i386, ppc, SPARC and ARM though only amd64 and i386 are currently supported in Valgrind. i386 was recently demoted to tier 2 support in FreeBSD, but it still seems to remain quite popular with FreeBSD users. The FreeBSD development world is split into 3 parts: base, ports and doc. base includes the kernel and quite a large number of core tools. The ports are a package mechanism for anything not in base (for instance, GNU make). doc obviously is the documentation. FreeBSD developers can be either committers (people that have been vetted and have write access to the source repos) or maintainers (typically those doing work on ports who don't have write access). The FreeBSD release model for base is to have 2 actively supported releases and 1 or 2 development versions. At the moment (Feb 2021) the active releases are 11.4-RELEASE and 12.2-RELEASE. 13 is entering beta and git head just became 14-CURRENT. When 13 is released, 11 will be EOLed. The ports model, by default, is to use a quarterly release cycle. Users can opt for a 'latest' model, or can install packages from source. There aren't too many fundamental differences between Linux and FreeBSD. One thing of note is that the system compiler is clang, not GCC. This does make life a little harder. The Valgrind port has been going for many years, with maintainers coming and going, usually after fixing the latest breakage. For a long time there were no merges from upstream, and it was stuck on version 3.10 + patches. Current Status ~~~~~~~~~~~~~~ After one false start back in 2018, I started working on earnest on the port at the start of 2020. I created a github clone of the sourceware repo and merged the existing FreeBSD work (mainly from Phil Longstaff, but there were a few other repos with patches that I incorporated). I used the github issue tracking, and to date have created about 150 issues, of which 125 are closed. This repo became the source for the devel/valgrind-devel port in July 2020, and I became the maintainer. I mostly work on 12.2-RELEASE, and my current regtest results aren == 735 tests, 12 stderr failures, 0 stdout failures, 0 stderrB failures, 0 stdoutB failures, 0 post failures == This is with the clang system compiler. With GCC I only get 4 failures due to having the right DWARF extensions and better OMP support. On i386 things aren't so great. I get somewhere between 25 and 30 failures. My current list of serious issues is 1. Apps built with rust fail due to the use of a guard page. 2. clang 10 generates 2 LOAD RW elf sections whilst Valgrind is hard coded to only handle one. I've hacked a workaround that mostly works but I'd like to make a better fix. 3. There seems to be an issue with the stack frame synthesized for signals. On amd64 this just seems to occasionally provoke a kernel message concerning some flags, but there are a couple of failures on i386. Other than the rust problem I haven't had any bug reports for several months. Testing ~~~~~~~ For the moment all development and testing has been done on my home computers (plus help from one keen i386 user). I have two old workstations, one that boots FreeBSD and Linux. The other has Solaris on it - not used much these days. I also have a couple of MacBooks. Since my computers are quite old I don't get to actively test the full set of regression tests - those that depend on recent Intel/AMD op-codes get skipped. In order to test as many FreeBSD versions as possible (and to some extent Linux and Illumos) I use VirtualBox a lot. For some more automatic testing, I have made the 'nightly' scripts work on my PC. I've discussed with the FreeBSD core team setting up a machine to run tests automatically and this should be possible. Overview of Changes ~~~~~~~~~~~~~~~~~~~ I'll create bugzilla items with more digestible review chunks. 'git diff --stat origin' currently gives me 457 files changed, 85460 insertions(+), 683 deletions(-) A few of these are not for merging (project and github files). 85k lines may seem a lot, but most of those are either new source or expected files. The files over 1k lines are coregrind/m_syswrap/syswrap-amd64-freebsd.c | 1109 + coregrind/m_syswrap/syswrap-x86-freebsd.c | 1513 ++ memcheck/tests/freebsd/scalar.c | 2031 ++ include/vki/vki-freebsd.h | 3077 +++ memcheck/tests/freebsd/scalar.stderr.exp | 5237 ++++ .../tests/freebsd/scalar.stderr.exp-freebsd130 | 5240 ++++ memcheck/tests/freebsd/scalar.stderr.exp-x86 | 5298 ++++ none/tests/amd64/sse4-64.stdout.exp-freebsd-clang | 5618 ++++ none/tests/amd64/sse4-64.stdout.exp.freebsd | 5618 ++++ coregrind/m_syswrap/syswrap-freebsd.c | 6848 +++++ helgrind/tests/tc19_shadowmem.stderr.exp-clang-x86 | 26122 +++++++++++++++++++ which accounts for something like 62k of the lines. If my awk skills are not letting me down there are about 3k lines of non-expected changes. Otherwise there are A. configure/Makefile changes Detection of the FreeBSD version and setting FREEBSD_VERS in config.h (similar to macOS) B. Alignment errors produce a SIGBUS on FreeBSD (not a SIGSEGV). So I copied and pasted to make a gen_SIGBUS_if_not_XX_aligned and used #ifdefs to select. It would probably be cleaner to refactor into a gen_SIGNAL_if_not_XX_aligned and put the conditional code inside that, but to start with I wanted to not change any of the Linux code. C. massif, memcheck, cachegrind, callgrind, dhat, none, exp-bbv. Almost no changes, just adding FreeBSD to macro tests. D. DRD and helgrind. Changes for FreeBSD-specific intercepts. E. coregrind/include. The meat of the changes are here. New files for syscalls and initimg. Added a few libc functions. Specific functions for address space manager, sigframe creation. Otherwise just adding FreeBSD to macro tests. F. tests. This is the other part with many changes. First, there are new FreeBSD-specific tests (29 currently, including one 'scalar' test). Then there are a lot of changes to the existing tests. Some would not compile on FreeBSD due to missing headers. Annoyingly, FreeBSD has an issue with PTHREAD_STACK_MIN - creating threads with this set invariably crashes the application, and I've modified the source to get these cases to run. A few other test cases required modifications. Often when the source changes the expected also needs to change. Due to the UB nature of a lot of the tests, FreeBSD produces different results. Usually in these cases I've added a new expected. Since I've tried to get the regtests to be as clean as possible for all 4 combinations of amd64/i386 and GCC/clang this has resulted in about 100 new expected files. TODOs ~~~~~ I'd like to add some more filters to reduce the number of new expecteds. In several places I've modified scripts, using "if [ `uname -o` = "FreeBSD" ];", usually where there are dependencies on GNU tools (sed, make, cp) that don't always work with the FreeBSD versions. To avoid any headaches I've just conditially used the GNU versions (which have a 'g' prefix on FreeBSD). It would be cleaner to create autoconf .in files for these scripts and to have configure detect the right binary. This would also detect problems earlier (e.g., if a FreeBSD user hasn't installed gmake). Any of the serious issues in the 'current status' section. A+ Paul |
|
From: Paul F. <pj...@wa...> - 2021-02-23 21:17:42
|
Ouf, as we say over here. 12 patches uploaded to bugzilla. Time for bed. A+ Paul |