ChrysaLisp Files

Overview

This release introduces native TCP/IP networking and direct machine-to-machine linking, asynchronous multi-stage streaming pipelines for image/media processing, an advanced 2D conical dilation and optical kerning engine for vector fonts, significant compiler backend upgrades (including a brand-new LoongArch64 translator and a completely rewritten x86_64 backend), and substantial ergonomics and performance enhancements across the core language runtime and CLI toolchain.

---

Key Highlights

1. Native TCP/IP Networking & Inter-Node Linking

• Native Cluster Linking (service/net, cmd/link): ChrysaLisp instances can now interconnect directly across physical machines over standard IP or DNS/mDNS hostnames.
• Decoupled Persistent Listener: Running link -l [port] keeps a server listener active indefinitely to accept reconnections and service multiple peers concurrently. Inbound and outbound connections spawn dedicated worker tasks (:host_net :conn, :host_net :in, :host_net :out).
• Kernel & Host Stack Separation: Low-level network socket I/O callbacks run on the host OS thread stack via sys_task :callback, avoiding machine stack exhaustion in lightweight ChrysaLisp tasks.
• Retirement of Legacy Bridging: The legacy ChrysaLib / hub_node shared-memory bridging layer has been completely removed from cmd/link.lisp and documentation in favor of self-hosted network links.
• Network Protocol Libraries (lib/net/):
• url.inc: RFC-compliant URL parsing, formatting, path/query extraction, and fast vectorized percent-encoding/decoding using lookup tables.
• http.inc: HTTP/1.1 client with automatic socket connection pooling (http-pool-checkout/http-pool-checkin), chunked transfer decoding, Content-Length framing, and streaming responses.
• json.inc: High-performance JSON parser and serializer with native escaping (\b, \q, unicode surrogate pair support) and seamless translation between JSON and ChrysaLisp collections (Pmap, List, Array).
• cmd/nettest.lisp: Test utility for testing end-to-end network connectivity, HTTP requests, and stream delivery.

2. Asynchronous Streaming Pipelines & Direct Cluster Code Execution

• Asynchronous Media Pipelines (lib/image/cpm.inc):
• CPM-load and CPM-save have been refactored from sequential, memory-buffered steps into asynchronous streaming pipelines using local child tasks (+kn_call_open).
• Symmetrical pipeline stages (cpm-load-stage-xxx and cpm-save-stage-xxx for pixmap data, RLE, and LZ4) stream data back-to-front through IPC stream mailboxes with zero intermediate buffer copies.
• Compression libraries (lib/streams/rle.inc, lib/streams/lz4.inc) are now loaded dynamically only when compressed streams are detected or created.
• Inline Task Execution:
• Added cluster-wide support for launching Lisp tasks directly from inline source code strings (strings prefixed with () without writing temporary files to disk.
• In the kernel, opt_run routes inline code strings directly to class/lisp/run; import in class/lisp/class.vp wraps the inline string in an sstream fed straight into the REPL.

3. Vector Typography & Optical Kerning Engine (cmd/ctf.lisp)

• Bidirectional Kerning & Triplet Checks: The font generator now supports both positive and negative kerning adjustments. Replaced naive cavity thresholding with a full 3-glyph triplet clearance check to prevent overlapping bounding envelopes between non-adjacent characters.
• Optimal Baseline Deadbanding: Added per-font default kerning baseline calculation (find-optimal-xkern). By optimizing the baseline deadband, .ctf font file sizes are reduced by up to 12% without altering VP rendering code.
• 2D Conical Dilation: Envelope distance calculations now apply 2D conical dilation (smooth-envelope) to prevent slice cavity collapses on complex letterforms (e.g., "Fa", "<<<", ">>>") while preserving italic glyph slopes.
• UI Integration: Adjusted Textfield widget cursor navigation and positioning metrics to align with the revised kerning profiles.

4. Compiler Backend & Architecture Translators

• New LoongArch64 Architecture Translator (lib/trans/la64.inc): Complete code generation backend supporting the 64-bit LoongArch ISA (LA64).
• Rewritten x86_64 Translator (lib/trans/x86_64.inc): Replaced the legacy reverse-engineered x64 translator with a clean, structured emitter supporting prepass instruction fusion (emit-prepass), ALU/memory operand combining, and optimized register allocation.
• ARM64 Backend Optimizations (lib/trans/arm64.inc):
• Enhanced 2-to-3 address ALU/shift peephole transformations.
• Prepass fusion for load/store pairs (LDP/STP), reducing compiled boot image size by ~6 KB.
• Generic VP Optimizer (lib/asm/vpopt.inc): Padded pmap instruction tables and unified instruction barrier checks into single fast lookups.
• Elimination of emit-tlabel: Removed redundant target label pseudo-instructions across the toolchain.

---

Runtime, Language & Core Library Changes

Variadic pinsert & perase

pinsert and perase now accept multiple keys or key/value pairs in a single invocation, matching def and undef:

:::lisp
;; Sets: multiple keys
(pinsert my-pset :key1 :key2 :key3)
(perase my-pset :key1 :key2)

;; Maps: multiple key/value pairs
(pinsert my-pmap :key1 "val1" :key2 "val2")
(perase my-pmap :key1 :key2)

Iterator Semantics & Functional Returns

• Early Exit in lines!: (lines!) now halts iteration immediately if the callback returns a truthy (non-nil) value, returning that value. If the stream reaches EOF or its bounded range, it returns :nil.
• Standardized (print) and (prin): Both functions now consistently return :nil instead of the last printed argument. This allows idioms like (lines! print stream) without triggering accidental early iteration exits.

Type System & Method Signature Matching

• Exact vs Subtype Matching in Signatures: :lisp :env_args_sig supports uppercase symbols to mandate exact class identity: lisp (signature '(:NUMS :nums)) ; Arg 1 must be exact :nums; Arg 2 may be any subclass of :nums
• :dim Updates: Tightened :dim method signatures to require exact :NUMS descriptors, followed by a system-wide sweep updating signatures across classes.
• Inlined Subclass Checks: :lisp :env_args_xxx methods now perform direct vtable hierarchy scans without machine stack consumption.

Strings, Numbers & System Routines

• VP-Lowered String Case Conversion: Lowered (to-upper) and (to-lower) to native VP code (:str :lisp_upper, :str :lisp_lower) for fast case-insensitive lookups.
• Native Escape Engine: Implemented native :str :escape in VP. :str :print now streams escaped output directly. Added support for \b (backspace) and \q (quote) escaping across :str :escape, :str :unescape, and json-parse.
• Subroutine Refactoring in :str :unescape: Converted read-hex-nibble from an inlined macro into a local subroutine call, reducing code bloat. Extended (call) in lib/asm/class.inc to support parameterless subroutine calls.
• Real to String Conversion: Added (real-to-str r [precision]) to root.inc, formatting floating-point values with controllable decimal precision and scientific notation.
• Register Macro Cleanup: emit-native-reg? is now a direct macro expanding to (pfind +emit_native_regs r) without runtime symbol type-checking overhead.

---

Tooling & Command Updates

Command / Component	Updates
cmd/trace.lisp	Added -i, --integrity flag to perform static verification of all compiled VP instructions and operands against instruction specs on disk.
cmd/grep.lisp	Added -n, --line-number and -i, --ignore-case options. Unified regular expression mode flags to -x.
cmd/edit.lisp	Added -i, --ignore-case search support.
cmd/sed.lisp	Added -i, --ignore-case replacement mode.
GUI Search Bars	All graphical desktop app search toolbars now support -i case-insensitive matching.
Docs Application	Added multi-column layouts, column alignment, and migrated Markdown rendering into a dedicated, reusable gui/md/lisp.inc (Md) widget.
Window Zooming	Standardized zoomable text applications to bind to the :zoom property on the primary window widget.

---

Bug Fixes & System Stability

• Vector Float Math: Fixed an illegal instruction bug in :reals :sum where (vp-xor-rr :f1 :f1) was improperly emitted instead of an integer-to-float conversion or float clear.
• Shared Memory Teardown (sys/link): Fixed resource leaks where unclosed links polled indefinitely. If transmission is blocked for the duration of lk_timeout, (:sys_link :out) terminates the link node, triggering coordinated shutdown of (:sys_link :in) and the parent (:sys_link :link). Shared memory segments are unmapped, closed, and removed from statics_sys_mail_links_array.
• JSON Escaping: Resolved edge cases in JSON string parsing where escaped backslashes and escaped quotes collided or dropped \b escape tokens.

---

New Documentation

• docs/ai_digest/net_stack.md: Architectural breakdown of the service/net socket service, lib/net/ protocol stack (url, http, json), and cmd/nettest.lisp.
• docs/ai_digest/async_pipelines.md: Guide covering cluster-wide inline Lisp tasks, node affinity/pinning, shared-memory stream concurrency, and cpm zero-copy image processing pipelines.
• docs/ai_digest/keeping_it_hot.md: Deep dive into ChrysaLisp execution cache behavior, register conventions, and compiler optimization strategies.
• docs/test.md: Formatting, column alignment, and component reference test document for the documentation reader.