[sdcc-devel] The future of 32-bit big-endian host regression testing

[Philipp K. K. <pk...@sp...>]

Dear SDCC developers,

3 bugs were found in SDCC 4.5.0 RC1 when the macOS ppc machine in the
farm came back online. Two of them serious ones (could result in silent
generatiomn of bad code). None of these bugs had been found on other
machines.

I am now reconsidering my previous opinion on regression testing on
32-bit big-endian hosts. Previously, when a lot of bugs were found by
regression testing on 64-bit big-endian host (ppc64), I assumed that
would be enough, and we'd no longer need the ppc machine in the farm,
that can't really keep up with the regression testing workload.

AFAIK, the macOS ppc machine is a Mac Mini G4. It takes two days to do a
third regression test (our testing infrastructure allowes for third
test, where only a third of the ports are tested (rotating) and for full
test, where all ports are tested). I expect the regression test load to
increase post-4.5.0 (new Rabbit ports nad memory models, possible new
pdk16 port, later possible dual-dptr mcs51 variant).

I have a Mac Mini G4 too (upgraded to the 1 GB RAM maxmimum and with an
IDE SSD). It usually sits in a box, and only gets taken out in
situations like now, where I need to do testing on a 32-bit big-endian
system. Mine runs OpenBSD, which isn't the fastest OS either.

So what options do we have going forward?

OS:
* macOS 10.5 was meant for ppc and can still be used on it. I am not
familiar with it, but Erik manages to keep it running for SDCC farm use.
* OpenBSD has ppc as a top-tier supported architecture.
* NetBSD has sparc64 as a top-tier architecture that can also run 32-bit
binaries.

Compilers:
* AFAIK, compilers have dropped 32-bit SPARC support.
* LLVM has dropped macOS Darwin ppc (i.e. macOS) support, but keeps ELF
ppc (i.e. OpenBSD, etc). OpenBSD releases tend to include a reasonably
current LLVM.
* GCC has both Darwin and ELF ppc support, and there are even options to
optimize for specific CPUs, such as the PowerPC 970 ("Apple G5"). An
older GCC is usually available to be installed on OpenBSD via the ports
tree.

Platform:
* Our old Mac Mini G4 - too slow.
* Emulation on amd64 - even slower.
* Virtualization on ppc64 - fast enough, OpenBSD isn't great as host or
guest for VMs. so this leaves the GNU/Linux host with macOS guest as
option. Works, but is hackish, and kvmppc hasn't seen much activity for
years; likely to break, requires work to keep going.
* sparc64 hardware: still expensive, and NetBSD doesn't even support the
fastest hardware well, so we'd be stuck with UltraSOARC IIIi, which is slow.
* The fastest ppc Mac - PowerMac G5 with two dualcore CPUs. Leaky
liquid-cooling, thus unreliable and few survive, making them expensive.
Also, putting something unreliable into the farm doen't seem like a good
idea.
* The second-fastest ppc Mac - Xserve G5 with two singlecore CPUs.
Cheap. More than twice the speed of the Mac Mini G4, but still slow by
today's standards. Should be able to keep up with the regression test
load today, and maybe the 4.6.0 one if we go from three-day to four-day
rotation. Somewhat noisy from what I've heard. Also quite deep (715 mm),
so it needs a real server rack and won't fit into the network cabinets
that house my other systems. OpenBSD also only supports up to 2 GB RAM
on macppc.

Maybe I forgot something, but to me, it looks like an Xserve G5 running
macOS or OpenBSD is currently the best option for 32-big-endian
regression testing.
macOS would be the faster option: AFAIK it is a faster OS than OpenBSD,
and AFAIK the GCC compiler we'd use there is better at optimizing for
ppc than LLVM.

Philipp

P.S.: The Apple G5 is actually a 64-bit processor. The OpenBSD macOS
port supporting it is 32-bit though. I think the latest macOS for G5 is
64-bit, but applications can be either 32-bit or 64-bit. Our snapshots
are currently targeting PowerPC 7400 ("Apple G4), which is 32 bits.