ChrysaLisp 7.5 "Rosetta"

Code made once, as VP, and translated where it runs. A build compiles each file to VP one time and gives every CPU its native code from that, by translators that are themselves VP. A machine that lacks a function's native code makes it from the VP as the function is bound, and a task sent to another machine reads its code from the system that sent it.

This is the first GitHub release since 7.0, "Pipsqueak". Four tags came between, and this page has each of them in short: the shader language and the GPU, triangles, a file system of our own, machines that find each other and sync, the Storage system, Films with sound of our own decoding, sealed links over the internet, and voice calls between desktops.

Update every machine together, with make install. The link has a version now, and a machine of 7.4 or before does not link to one of 7.5. The translators are VP, the Lisp ones are gone, so a tree from before cannot build its way forward: make install starts from the new snapshot.zip, whose boot image carries every translator.

VP, made once

• The VP is the same on every platform. A host call is the same VP everywhere and the translator puts its args where the ABI wants them; a task's state is saved and loaded by VP instructions the translator expands; nothing in the VP knows an ABI, and the stack has no alignment in it, each host call lines it up at the call.

• A build has two stages. Each file is compiled to VP once for a build mode and kept, obj/vp/<mode>/; each platform's code is translated from it, only what has changed. A build of every platform walks the tree once.

• The translators are VP, lib/trans/<cpu>.vp: VP binary in, native code out, with their peephole passes. They give the same code to the byte as the Lisp translators did, the whole tree in 7 ms in place of

  1. ARM64, x86_64 for AMD64 and WIN64, RISC-V 64, LoongArch 64, and VP64 for the emulator.

• The VP spec, vp_vm.md, says what each instruction does at its edges: sets give 0 or 1, float branches follow IEEE on a NaN, constants and offsets of any size, loads and stores at their natural alignment only. tests/system/test_vp_spec.lisp holds every target to it. Writing it down mended RISC-V's long branches on equal operands, x86's logical and, and the optimizer forwarding a narrow store as the whole register.

• run_tui.sh -V, the VP64 emulator in validation mode, says each unaligned load or store.

Code that travels

• A function's VP binary is kept beside its native code, <name>.vpb. When a node binds a function that has no native code here, the loader, which holds its CPU's translator, makes it from the VP, with each function it calls, and keeps it as a build would have.

• @Source, one to a system. A task sent to another machine reads its Lisp from the system that sent it, so NetMon's and Mandelbrot's children need none of their files where they run.

• Function servers. A task from another system binds that system's natives: the hashes of a function and all it calls are asked for once, only VP this node does not have is sent, as VP binary, and its translations are cached by hash. They live in a server of their own that goes with its last task.

• The boot images are smaller. The VP64 one holds what an install needs before it has built anything, and carries all the translators; a native one holds the loader, the kernel and what they call, 44KB, and the rest is bound from obj/ as it is asked for. snapshot.zip is 161KB. A load goes on in the block the last one ended in, and a block that has run is never written again.

Films

• AVI, lib/media/avi.inc, read as an mp4's tracks are: idx1, OpenDML, or its chunks one after another where there is no index. Sound of MP3, AC-3, PCM, and Microsoft and IMA ADPCM, lib/media/adpcm.inc. Pictures of Motion JPEG, ours or the host's, H.264, and MPEG-4 Part 2 where the host decodes it.

• An MP3's picture is drawn from the frames its sound decoded, not decoded again, and the decoder takes a third less time a frame, the same samples bit for bit. HE-AAC decodes 13% quicker by SBR's filter banks done by the DCT-IV, and 15% quicker again by its hot loops native, the same samples.

• A film resized keeps its decoder and does not pause.

Storage

• A store on a flash stick is written 2.3 times as fast: blocks go in folders by when they were put, so a run of them writes one folder, not

  1. A copy into a store is four times as fast, its whole blocks written in order at the end of a run; 63 MB/s onto Flash.

• A store's stream keeps its last block and what it asked for ahead across a near seek: an MP3 read from a store plays with no gaps, frames 116 times faster.

• A member that steps aside for a desktop gives up its store's open part way, and one told to go while it starts is stopped, not left running and claimed. The Windows host seeks and stats in 64 bits, so a store image past 2GB is made and read there.

Files

• A file picker shared by the apps, one call to open it. A drawer: a folder open, its folders and files of the kinds asked for, a filter, the path above. Down the side the places: the tree, the user's folder, and the app's own folders. A preview of what is picked, a film's first frame and length, a font's glyphs. Objects of the store are picked as files are, store:path. Films, Fonts, PCB and Images open by it.

• Whiteboard: Insert adds a board file's things to the board. A .cwb keeps the keys it does not know and saves them back, and :script is kept for the scripts shapes and documents are to carry.

Networks

• The link has a version: the handshake is CLDOOR3 with +door_version 1, a node of another closes at once, and beacons say it. mesh says in one sentence what is not right of a machine it hears, and names the machines a door was closed on. The link protocol is written down, link_protocol.md, and the TCP link is tested against it byte by byte.

• A message is made with its src and offset 0: a message of one fragment had carried a reused block's old header onto the wire.

Tests and tools

• Tests wait for what they test, never a time, so a slow machine fails nothing. A run always says how it goes, and a module long past its usual time is said to be late as it runs.

• check, forward, includes and imports find more: forward finds a function named in (const) above its definition, and includes reads an .inc that is not a class's as part of the file. The lint is clean, relies with it.

• On a slow machine no app takes a whole node: Eyes takes at most a quarter of its node, Molecule draws its new atom images between frames, and the GUI waits as long as its last frame took.

• The eraser asks (poly-hit?) of two hulls, the GJK is gone, 25 to 55 times quicker. A sync -d takes the folders its removes leave empty, and a file changed twice in one second, to the same size, is no longer taken for unchanged by sync's cache of hashes.

• Gone: the profiler, whose figures were its own cost as much as the code's, use (time-it); debug mode and its app, the stack frames are kept; an app's (jit) of its lisp.vp, make install makes the apps.

Since 7.0, the tags between

Each has its own notes.

• 7.1. A shader language of our own, written as s-expressions, and the GPU to run it on. The GUI on SDL3, a Raspberry Pi running the desktop with nothing under it, and a session that keeps itself to itself.

• 7.2. Vertex and pixel shaders drawing triangles on the nodes or the GPU, and typed functions Lisp calls as native code. An exFAT file system of our own, SHA-256 and ChaCha20-Poly1305. Tests that run only what a change could have broken. Machines that find each other: the Net services make a mesh, and sync makes one machine's files another's.

• 7.3. The Storage system: the machines of a mesh keep one store between them, each block on three of them, read and written as a file is, or a function sent out to where the blocks are. A band of the scheduler for tasks that work for others, textures in the shader language, PNG and JPEG, deflate, HTTPS by TLS 1.3, and the Whiteboard made over.

• 7.4. Films with their sound: mp4, Matroska and WebM, the pictures by the host's decoders, the sound by our own, AAC with HE-AAC, AC-3 and E-AC-3, MP3 and FLAC. Links over the internet sealed, voice calls between desktops, TLS 1.2 beside 1.3.

The release check

Every stage passes on every platform, from cold, whatever the cache of test results had.

• An Apple M4 Max, a 2018 x86_64 MacBook Pro and a Raspberry Pi 4. make install from snapshot.zip, 4 s, 10 s and 42 s; every test with tests -a, 195 modules and 8,006 tests on each; the boot images of all six CPUs; the lint, the forward check and a restore, clean.

• The VP64 emulator's release image, 7,734 tests, those of the error checks a release build does not have left out.

• RISC-V and LoongArch under QEMU, each installed from the snapshot as a new machine is, its boot image the same to the byte as the M4's cross build, and 7,943 tests.

• The Windows host under Wine, make install and 7,936 tests.

Thanks

• To Andy Stout, for the issues that found a store left running and claimed, a test that hung with no user, an MP3's pictures decoded twice, the Linux host's warnings, and a big store's slow open; for input -f taking text==, and for putting the history right on the PlayStation assembler; and for offering VA-API on the hardware he has.

• To Martyn Blyss, for the Windows runs and the GPU triangles' first start there, and to Darren Pegg, for the shader language's names, and for the iOS and Android hosts he is building on the VP64 emulator.

Not ready

• Echo cancelling in voice calls has not been heard live; half duplex stays on.

• Opus is not here, and WebM's pictures are mostly VP8, VP9 or AV1, which the host is not given.

• Pictures by Media Foundation on Windows are built and not yet run on Windows itself; the Windows scripts are not run under Wine.

• A store of 256GB takes about 70 s to open, nearly all of it reading its folders of blocks.

Enjoy

Chris