#199 Swift compression: research summary and adoption path

[Anonymous]

Originally created by: Avazbek22

Status

Swift compression is deliberately not shipped. This issue records the research that was done, why we decided to wait, and the exact adoption path for whenever we (or a future contributor) decide to add it. Everything below was verified hands-on, not assumed.

Why Swift is not in the 14 supported languages

1. The Swift grammar shipped inside TreeSitter.DotNet 1.3.0 is unusable. It comes from the archived tree-sitter/tree-sitter-swift repository (pre-async/await era). Verified facts:
2. Its ABI is below the runtime minimum (13), so Parser refuses to load it: InvalidOperationException: The language version is incompatible with this library. The shipped tree-sitter-swift.dll is dead weight.
3. Its AST has no capturable function body node (statements are inlined into the declaration), so our bodies/preserve query contract cannot be expressed against it at all.
4. It is only built for Windows in the package.
5. The maintained community grammar is alex-pinkus/tree-sitter-swift (the de-facto standard, used across the tree-sitter ecosystem). We verified it end-to-end on win-x64: compiled with zig cc (~3.9 MiB), loaded through TreeSitter.DotNet 1.3.0 (ABI 14, runtime supports 13-15), parsed modern Swift (actors, async/await, @MainActor) with zero ERROR/MISSING nodes.
6. Audience economics. Swift is ~4-5% of developers, almost entirely on macOS/Xcode, while our core audience is Windows-first. Roughly +4 MiB per release artifact for a small audience was judged not worth it right now. This is a product decision, not a technical blocker.

Upstream work already in flight

• Binding PR replacing the dead submodule with alex-pinkus (also enables Linux/macOS builds): mariusgreuel/tree-sitter-dotnet-bindings#14. If merged and released, Swift becomes a normal "add a language pack" task with no vendoring.
• Kotlin grammar request for the binding: mariusgreuel/tree-sitter-dotnet-bindings#15 (would let us drop our vendored Kotlin binaries).
• Real scanner bug we found and fixed upstream: on LLP64 (Windows) 1UL << FAKE_TRY_BANG in OP_SYMBOL_SUPPRESSOR shifts a 32-bit long by 32 (UB; the mask degrades to the BLOCK_COMMENT bit), affecting operator suppression around postfix ! / try! in Windows builds. Fix: alex-pinkus/tree-sitter-swift#606 (issue [#607]). Until that lands, any local build must carry this one-line patch (1UL -> 1ULL).

Adoption path (when we want Swift)

Deep external research (including prebuilt channels: npm/PyPI prebuilds, xberg/kreuzberg language packs, nvim/Zed/Emacs ecosystems, WASM-via-wasmtime, SwiftSyntax/SourceKit) found no ready-made channel that satisfies our delivery contract (raw tree_sitter_swift shared libraries, six RIDs, verifiable provenance, no runtime downloads, no Node/Python wrappers). The cheapest reliable path is our own vendoring pipeline, already proven with Kotlin:

1. Source: official alex-pinkus release archive (releases ship SHA-256 digests and GitHub build-provenance attestations; prefer that over the with-generated-files branch). Pin tag + commit + archive digest.
2. Patch: apply the scanner fix (#606) unless already merged; record patch SHA-256 and patched-file SHA-256 in the lock.
3. Build: generalize tools/grammars/build-kotlin.ps1 into a manifest-driven builder; same zig toolchain, -O2, six RIDs (win-x64/arm64, linux-x64/arm64, osx-x64/arm64). Extend Infrastructure/Grammars/vendored/vendored-grammars.lock.json and the supply-chain tests (see VendoredGrammarSupplyChainTests) - they already enforce hash pinning.
4. Language pack (Infrastructure/Compression/Languages/Swift), modeled on Languages/Kotlin. Minimal v1 bodies:
   scheme (function_declaration body: (function_body) @body) (init_declaration body: (function_body) @body) (deinit_declaration body: (function_body) @body)
   Contract decisions (consistent with the rest of the engine):
5. init/deinit compress - Swift state lives in stored property declarations and the signature, like C#/Kotlin (unlike Python/Ruby/PHP where constructors are preserved).
6. Computed properties, accessors (get/set/willSet/didSet), subscripts, closures, result-builder bodies (incl. SwiftUI var body) stay whole in v1 - they are simply not captured; this matches our property-preservation contract.
7. Attributes, property wrappers, generics, async/throws/typed throws live in signatures and survive automatically.
8. Admission gates before calling it supported:
9. Corpus: SwiftNIO, Vapor, Alamofire, modern SwiftUI code, macro-heavy packages.
10. Scanner regression set on all six RIDs: try!, try?, prefix/postfix !, custom operators - plus serialized-AST parity across all six binaries (an independent build such as the xberg language-pack bundle can serve as a comparison oracle; do NOT ship their binaries).
11. Known upstream quirks to include: the open as? ... ?? ... parse regression; a report of ~6s query compilation for a complex third-party query (our queries are short and parent-anchored, but measure query-compile time anyway, target well under 500 ms).
12. Reverse-parse gate + full-file fallback behavior on malformed input (engine already guarantees this).
13. Payload: expect ~+4 MiB per RID; the grammar-delivery CI budget (currently 29 MiB) will need a deliberate bump to ~33-34 MiB.
14. If binding PR [#14] ships first: skip vendoring entirely; do a controlled migration (compare grammar commit, ABI, hashes, corpus output), keep only the language pack.

Revisit triggers

• mariusgreuel/tree-sitter-dotnet-bindings#14 merged and released -> Swift becomes cheap; reevaluate.
• Meaningful macOS/Swift user demand -> run the vendoring path above (~2-3 days).

[Anonymous]

Originally posted by: Avazbek22

Update: the scanner fix was merged upstream - alex-pinkus/tree-sitter-swift#606 is now in main.

Consequence for the adoption path above: step 2 (carrying a local 1UL -> 1ULL patch) becomes unnecessary once a source snapshot that includes both the fix and the generated files exists - either the next with-generated-files refresh (their automation updates it periodically) or the next tagged release. When picking the pin, verify the fix is present in src/scanner.c and drop the patch/patch-hash entries from the lock schema.

Also worth doing at that point: update the submodule pin in mariusgreuel/tree-sitter-dotnet-bindings#14 from 0.7.3-with-generated-files to the post-fix snapshot, so the binding PR ships the corrected grammar.