[Sbcl-devel] x86-64, amd64, contrib: sb-fiber

[Jesse B. <jes...@pr...>]

Howdy sbcl-devel,

Here is a revised patch to introduce fibers/coroutines via a sb-fiber contrib and what I believe to be idiomatic x86-64/arm64 runtime support. Subsequent to providing invaluable guidance on implementation, Charles Zhang was kind enough to review a version a few weeks ago and suggested that it was in good enough shape that I share it here to solicit feedback on interface shape, completeness and fitness to purpose.

There's been further work over the past few weeks, mainly an outcome of exposing a fiber-based HTTP proxy in a DMZ, where it was promptly incinerated by scanners, php probes, etc. The package has gained more consistent interrupt and condition support, and notably the ability to migrate fibers between threads, which should allow users good flexibility in scheduler design. Further to that, I'll say again that the main goal here has been to introduce the minimal operation set to support user applications: concrete scheduler and generator implementations are not provided. I'll share gists of what I'm using for those, if there is interest.

Some measurements on arm64, Apple M2, Mac14,2 are:

pingpong (resume, yield)         ~62 ns/op   (2 switches/iter -> ~31 ns/switch)
switch, 32 live bindings      ~97 ns/op   (binding-stack swap = ~2 ns/binding)
make-fiber, release           ~4.6 us/op   (stack mmap, struct alloc)

(Creation is mmap-bound; pooling is the obvious mitigation.)

A patch is attached, corresponding to https://github.com/jbouwman/sbcl/commit/1c5ba1fc4ee3246e4455e2af998b9e7eb004cbf2

  — Jesse