[Valgrind-developers] FreeBSD roadmap, part 1

[Paul F. <pj...@wa...>]

Hi all

Since 3.17 is starting to be in the air, I'd like to start the ball rolling
for integrating FreeBSD upstream. I'm not necessarily targeting 3.17,
I'd rather everything be done properly rather than rush anything.

In this mail I'll cover the following


- FreeBSD background

- current status

- testing

- overview of changes

- TODOs


FreeBSD background

~~~~~~~~~~~~~

I won't delve into the history of FreeBSD, but just give an overview of the
ecosystem. It runs on amd64/i386, ppc, SPARC and ARM though only amd64 and
i386 are currently supported in Valgrind. i386 was recently demoted to 
tier 2
support in FreeBSD, but it still seems to remain quite popular with FreeBSD
users.

The FreeBSD development world is split into 3 parts: base, ports and 
doc. base
includes the kernel and quite a large number of core tools. The ports are
a package mechanism for anything not in base (for instance, GNU make). doc
obviously is the documentation. FreeBSD developers can be either committers
(people that have been vetted and have write access to the source repos)
or maintainers (typically those doing work on ports who don't have write
access).

The FreeBSD release model for base is to have 2 actively supported releases
and 1 or 2 development versions. At the moment (Feb 2021) the active 
releases
are 11.4-RELEASE and 12.2-RELEASE. 13 is entering beta and git head just
became 14-CURRENT. When 13 is released, 11 will be EOLed. The ports model,
by default, is to use a quarterly release cycle. Users can opt for a 
'latest'
model, or can install packages from source.

There aren't too many fundamental differences between Linux and FreeBSD.
One thing of note is that the system compiler is clang, not GCC. This does
make life a little harder.

The Valgrind port has been going for many years, with maintainers coming and
going, usually after fixing the latest breakage. For a long time there were
no merges from upstream, and it was stuck on version 3.10 + patches.

Current Status
~~~~~~~~~~~~~~

After one false start back in 2018, I started working on earnest on the port
at the start of 2020. I created a github clone of the sourceware repo and
merged the existing FreeBSD work (mainly from Phil Longstaff, but there were
a few other repos with patches that I incorporated). I used the github issue
tracking, and to date have created about 150 issues, of which 125 are 
closed.
This repo became the source for the devel/valgrind-devel port in July 2020,
and I became the maintainer.

I mostly work on 12.2-RELEASE, and my current regtest results aren

== 735 tests, 12 stderr failures, 0 stdout failures, 0 stderrB failures, 
0 stdoutB failures, 0 post failures ==

This is with the clang system compiler. With GCC I only get 4 failures
due to having the right DWARF extensions and better OMP support.

On i386 things aren't so great. I get somewhere between 25 and 30 failures.

My current list of serious issues is
1. Apps built with rust fail due to the use of a guard page.
2. clang 10 generates 2 LOAD RW elf sections whilst Valgrind is hard 
coded to only handle one.
I've hacked a workaround that mostly works but I'd like to make a better 
fix.
3. There seems to be an issue with the stack frame synthesized for signals.
On amd64 this just seems to occasionally provoke a kernel message 
concerning some
flags, but there are a couple of failures on i386.

Other than the rust problem I haven't had any bug reports for several 
months.

Testing
~~~~~~~

For the moment all development and testing has been done on my home 
computers
(plus help from one keen i386 user). I have two old workstations, one that
boots FreeBSD and Linux. The other has Solaris on it - not used much 
these days.
I also have a couple of MacBooks. Since my computers are quite old I 
don't get to
actively test the full set of regression tests - those that depend on recent
Intel/AMD op-codes get skipped.

In order to test as many FreeBSD versions as possible (and to some extent
Linux and Illumos) I use VirtualBox a lot.

For some more automatic testing, I have made the 'nightly' scripts work on
my PC. I've discussed with the FreeBSD core team setting up a machine to run
tests automatically and this should be possible.

Overview of Changes
~~~~~~~~~~~~~~~~~~~

I'll create bugzilla items with more digestible review chunks.

'git diff --stat origin' currently gives me

  457 files changed, 85460 insertions(+), 683 deletions(-)

  A few of these are not for merging (project and github files).

85k lines may seem a lot, but most of those are either new source or 
expected files.

The files over 1k lines are
  coregrind/m_syswrap/syswrap-amd64-freebsd.c        |  1109 +
  coregrind/m_syswrap/syswrap-x86-freebsd.c          |  1513 ++
  memcheck/tests/freebsd/scalar.c                    |  2031 ++
  include/vki/vki-freebsd.h                          |  3077 +++
  memcheck/tests/freebsd/scalar.stderr.exp           |  5237 ++++
  .../tests/freebsd/scalar.stderr.exp-freebsd130     |  5240 ++++
  memcheck/tests/freebsd/scalar.stderr.exp-x86       |  5298 ++++
  none/tests/amd64/sse4-64.stdout.exp-freebsd-clang  |  5618 ++++
  none/tests/amd64/sse4-64.stdout.exp.freebsd        |  5618 ++++
  coregrind/m_syswrap/syswrap-freebsd.c              |  6848 +++++
  helgrind/tests/tc19_shadowmem.stderr.exp-clang-x86 | 26122 
+++++++++++++++++++

which accounts for something like 62k of the lines. If my awk skills are 
not letting me
down there are about 3k lines of non-expected changes.

  Otherwise there are

  A. configure/Makefile changes
  Detection of the FreeBSD version and setting FREEBSD_VERS in config.h
  (similar to macOS)

  B. Alignment errors produce a SIGBUS on FreeBSD (not a SIGSEGV). So I
  copied and pasted to make a gen_SIGBUS_if_not_XX_aligned and used
  #ifdefs to select. It would probably be cleaner to refactor into a
  gen_SIGNAL_if_not_XX_aligned and put the conditional code inside that,
  but to start with I wanted to not change any of the Linux code.

  C. massif, memcheck, cachegrind, callgrind, dhat, none, exp-bbv.
  Almost no changes, just adding  FreeBSD to macro tests.

  D. DRD and helgrind. Changes for FreeBSD-specific intercepts.

  E. coregrind/include. The meat of the changes are here. New files for
  syscalls and initimg. Added a few libc functions. Specific functions for
  address space manager, sigframe creation. Otherwise just adding FreeBSD
  to macro tests.

  F. tests. This is the other part with many changes. First, there are
  new FreeBSD-specific tests (29 currently, including one 'scalar' test).
  Then there are a lot of changes to the existing tests. Some would not
  compile on FreeBSD due to missing headers. Annoyingly, FreeBSD has an
  issue with PTHREAD_STACK_MIN - creating threads with this set invariably
  crashes the application, and I've modified the source to get these cases
  to run. A few other test cases required modifications. Often when the
  source changes the expected also needs to change.

  Due to the UB nature of a lot of the tests, FreeBSD produces different
  results. Usually in these cases I've added a new expected. Since I've 
tried
  to get the regtests to be as clean as possible for all 4 combinations of
  amd64/i386 and GCC/clang this has resulted in about 100 new expected 
files.

  TODOs
  ~~~~~

  I'd like to add some more filters to reduce the number of new expecteds.

  In several places I've modified scripts, using "if  [ `uname -o` = 
"FreeBSD" ];",
  usually where there are dependencies on GNU tools (sed, make, cp) that 
don't
  always work with the FreeBSD versions. To avoid any headaches I've just
  conditially used the GNU versions (which have a 'g' prefix on FreeBSD).
It would be cleaner to create autoconf .in files for these scripts and
to have configure detect the right binary. This would also detect problems
earlier (e.g., if a FreeBSD user hasn't installed gmake).

Any of the serious issues in the 'current status' section.


A+
Paul