#834 Add WASI platform support

[PuQ]

For those interested in running gnuplot on WASI, there is an experimental patch (based on 6.1 [444208]) to make gnuplot work on WASI.

Non-goal:

• Functions depending on external libraries
• Readline support (WASI does not support termio)

Build dependencies:

• Clang 20.1+
• wasi-libc, can be installed with package managers
• wasi-compiler-rt, can be installed with package managers

Build (wasi-sysroot path may be different from this one):

./prepare
PKG_CONFIG_LIBDIR=/usr/share/wasi-sysroot/lib CC=clang ./configure --host=wasm32-wasi --with-wasi-sysroot=/usr/share/wasi-sysroot
cd src
make timestamp.h binonly

Test with Wasmtime:

wasmtime -Wexceptions gnuplot.wasm -V
wasmtime -Wexceptions gnuplot.wasm -e 'set term dumb; p sin(x), cos(x)'
wasmtime -Wexceptions --env HOME=/tmp --dir /tmp::/tmp gnuplot.wasm

[Ethan Merritt]

I am potentially very interested in this, but I am very much not up to speed on the wasm ecosystem. Furthermore so far as I can tell, the linux systems I normally work on do not provide any of the relevant support packages, so I am at least one or two major steps away from being able to try this out. Maybe you can provide some guidance...

• Would it be more reasonable to work from the GitHub repository code and build it up the clang/wasm/wasi development tools from scratch or to start by installing a different linux distro that offer relevant pre-built packages? I am not a fan of Fedora, but maybe Arch or Mint?
• There was previous work to build gnuplot for emscripten, but I am unclear on the relationship between emscripten and wasm/wasi.. Is there any connection?
• Maybe you could give a birds-eye summary of the use cases you see for gnuplot + wasi. In particular I am wondering how much of the existing terminal output code is relevant - would typical graphics output go via qt or kitty or svg or what? In the earlier proof-of-principle emscripten work the output went to svg, but it was not very well integrated.
• What exactly is the issue you mention with termio? I gather that wasi does not provide an emulation for the termios library, but gnuplot's builtin readline should fall back to using ioctls instead. Does that not work either?
• Could you attach the output from the configure script from your test run? I'd like to see how much of the code above a bare minimum was pulled in and therefore known to build successfully (run performance not guaranteed of course).

[PuQ]

wasi-sdk would be a good start, rather than installing another distro. It is a prebuilt bundle providing latest clang, wasi-libc and wasi-compiler-rt.

I don't have much experience in Emscripten, but from what I know, Emscripten and WASI are two platforms (wasm32-emscripten and wasm32-wasi for compilers), providing different libc for Wasm. Wasm is more like a common architecture.

WASI is also a set of system APIs designed to make more apps run anywhere including browsers/machines/servers. It provides POSIX-like system interfaces for Wasm to access OS (for gnuplot, that would be reading and writing local files, getting time and environment variables). Wasm apps call these APIs inside the WASI runtime, which are then executed by the runtime. In most cases, Wasm binaries built by Emscripten (usually with js files) are mainly run inside browsers, however, Emscripten also uses WASI APIs outside browsers (without js files).

Currently, WASI has three independent versions (0.1, 0.2 and 0.3). The widely used one is WASI 0.1 (more commonly known as wasip1, WASI Preview 1). Most Wasm runtimes also support WASI 0.1.

GUI terminals like qt and wxt, depending on lots of system libraries and complex syscalls, are almost impossible for WASI. Many raster terminals require system fonts via fontconfig, they may work, but I have never tried building fontconfig into gnuplot on WASI. I think svg would still be the best terminal on WASI as it is zero-dependency and vector-based.

WASI only has limited ioctl support, without terminal-related ioctls. In other words, there is no terminal API on WASI, and wasi-libc is just a C wrapper for these underlying WASI APIs. Since gnuplot lacks readline support on WASI, I think it would be well-suited as a plugin/extension for other software (like a plugin for Typst, it can be run on different platforms, including browsers, but using the same Wasm binary), maybe also a Web API backend.

I've made an asciinema recording showing how to build gnuplot using wasi-sdk, and here is a link to view it online.

[Ethan Merritt]

Now that I've looked at the patch, I wonder if 99% of it could be respun as "optional support for using memory-backed scratch space viafmemopen and open_memstream rather than using tempfiles. That seems like a perfectly reasonable option in any case, and it might then simplify WASI support into something like "you must select this option during configuration". Do I have that right?

Thanks for the session output video. That gives me an idea of what you ended up with and what is missing.

As to terminal output modes -

I think Qt explicitly supports WASM: (https://doc.qt.io/qt-6/wasm.html)
I'm still trying to understand exactly where WASI fits in on top of that, but I would have thought connecting the front and back ends of a Qt app (gnuplot + display window) would be a obvious example of a "system API". That honestly sounds like a more flexible target goal for gnuplot putput than trying to shuffle SVG files between apps and then trying to interact with it.

I can understand that building pango+cairo may be difficult, but I think at a minimum you would want to bring in gdlib. That would give you png/jpeg/sixel/kitty output terminals and also support for reading in image data. In a terminal session such as you show in the video, kitty output would go a long way toward providing the full range of graphics. OK fonts might still be a problem.

You say "I think it would be well-suited as a plugin/extension for other software", but that doesn't fit with my understanding of what a "plugin" is expected to provide. Gnuplot does not provide an API other than a text input stream and an output stream whose content depends on the terminal type. So how would it function as a plugin? Maybe I'm thinking about this the wrong way.

[Ethan Merritt]

Another question:

Is there a reason you use the sequence fclose(fp); fp = fmemopen(fbuf, fbuf_len, "r"); rather than simply rewind(fp);?
The documentation says fseek is usable after open_memstream(), and rewind is documented as being equivalent to fseek(stream, 0L, SEEK_SET).

Allowing the code to use rewind(fp) further simplifies the change and works for me in testing (not on WASI yet).

[PuQ]

Indeed, making fmemopen and open_memstream optional would be better. open_memstream creates a write-only stream, so I reopen the buffer with fmemopen to read from it. fmemopen can also be used for writing, but it requires a fixed-size buffer compared to open_memstream.

From what I can see in Qt's docs, Qt for WebAssembly is essentially Qt for Emscripten and is designed to run in browsers. WASI's graphics APIs are still in an early stage, and no mainstream runtime supports them, unlike Emscripten, which uses JavaScript to provide additional functionality.

I suddenly realized that instead of porting the existing Qt terminal to WASI, it might be possible to implement a new terminal based on browser technologies such as WebGL2. In that case, gnuplot.wasm could call external JavaScript functions to render graphs directly in the browser. Since there is already a canvas terminal, perhaps a quick proof of concept could be done by replacing fprintf(gpoutfile, "js_external_function(...)") with js_external_function(...).

The plugin I mean would execute gnuplot code inside editors, typesetting engines, or similar environments that embed a Wasm runtime, using in-memory streams based on functions like fmemopen, open_memstream, or fopencookie. It would call load_file to run a script stored in a buffer and redirect gpoutfile to another in-memory stream instead of using temporary files or pipes to run scripts and capture results, as most gnuplot bindings do.

[Ethan Merritt]

I have committed your original patchset and then committed a small modification to move the tempfile handling into a set of macros in syscfg.h rather than cluttering up the primary code with tests for #ifdef __wasi__. They also fix a memory leak that would have happened if an error is encountered while a tempfile is open.

After a bit of fumbling I got the wasi-sdk installed and used that for testing.

• The only real problem I had is that I don't understand the mechanism for granting permission to access filesystem directories, so the normal initialization files are not found and the postscript terminals cannot read the required prolog code for insertion into postscript output. The same issue prevents normal inclusion of the javascript fragments to support mousing in SVG output. For testing purposes it works to copy everything into the current directory first but that's not a real solution.

• I see that the problem with rewind() is not the fseek() operation; it's that the implementation of open_memstream() in wasi-sdk does not allow read access to the buffer. IMHO that's weird but I see that it is consistent with the opengroup.org specification for open_memstream. The linux libc implementation does not have this limitation so I didn't encounter the problem in my earlier testing. Anyhow it's now handled by the new macros.

• For testing, I added the libstbi image-handling routines into breaders.c (despite the name, libstbi can be used as an in-line collection of routines rather than a library). This allows the gnuplot.wasm executable to read binary data from image files (notably PNG but lots of others also). With this addition, the full set of unit tests in all.dem run successfully. Yay!

• I agree that an obvious future direction is to revamp or totally rewrite the canvas terminal. The current canvas.trm was written as a proof-of-priciple showcase for the graphics capabilities of the brand new (at the time) HTML5 canvas extensions. At that time the browser implementations did not provide access to their internal font-handling routines, so the gnuplot terminal was (and still is) limited to a single ancient set of vectorized character glyphs. A total rewrite has been on the project wishlist for years but no one ever stepped up to tackle it.
  Qt would still be my first choice, though, since simple Qt apps need not be browser-based.

• Oh, one other thing. The history code expects to work in conjunction with readline. It also expects to be able to read and write to a persistent history log. I have not given up on addressing those expectations but as it stands you cannot do much with the history commands and it uselessly dumps the session log to stderr as the program exits.

[Ethan Merritt]

Update

After a minor tweak to gnuplot's svg/domterm terminal driver, I can now run the WASI/WASM gnuplot executable by invoking wasmtime inside qtdomterm. See https://domterm.org/ and (somewhat old) https://opensource.com/article/18/1/domterminal. Domterm intercepts the SVG output and then renders it inside the widget. This gets me full graphics, fonts, etc., in a Qt widget. It's great. Screenshot attached.

Here's what I don't understand, though. It also makes the history and readline functions work. I thought that wasn't possible? I guess the qt layer must be doing the line editing and then passing only the edited line through to gnuplot's in stream. Maybe?

Domterm itself is mostly written in javascript. The qt version I tested with runs that javascript via a Qt widget, but you can also get it to run in a browser and various other configurations I have not explored with a mixed bag of output via more javascript or HTML.

As it stands now, the mousing code gnuplot generates for use with SVG is not accepted by domterm. I would guess that might be fixable. I will try to bring the domterm author into the loop.