The CUE Data Constraint Language Files

This release brings significant performance improvements, three language changes, and initial support for Kubernetes CRDs.

Changes which may break some users are marked below with: :warning:

Evaluator

Performance

A particularly slow part of the new closedness algorithm has been rewritten for speed, resulting in improvements of up to 10x in wall times for some projects.

Significant progress has been made to the new evaluator's memory usage; a few large projects experiencing high memory usage on v0.13 now see reductions of up to 80%.

Various other optimizations and performance bug fixes have been made to the new evaluator, yielding speed-ups of 5-20% on a variety of CUE projects.

error builtin

A new error builtin is added to the language, which allows users to create custom error values with a specified message.

Don't simplify validators into concrete values

A new CUE_EXPERIMENT=keepvalidators experiment is introduced, already on by default, which prevents validators from being simplified into concrete values.

X == Y and == X

Firstly, there were several bug fixes related to ==. Most notably, [int] == [int] incorrectly resolved to true. It now correctly resolves to an error.

All other changes are enabled using the @experiment(structcmp) file-level attribute. Enabling this experiment allows all CUE values to be compared for equality. This includes comparing structs.

The same experiment also allows a unary == to enforce that a value is a specific concrete value.

Other changes

:warning: The file embedding experiment can no longer be disabled via CUE_EXPERIMENT=embed=0, having been introduced in CUE v0.10.0.

File embedding is now enabled when using the cue/cuecontext or cue/load APIs; earlier versions could not expose it via the Go API due to an import cycle.

:warning: The topological field sorting experiment can no longer be disabled via CUE_EXPERIMENT=toposort=0, having been introduced in CUE v0.11.0.

:warning: CUE_EXPERIMENT=cmdreferencepkg, introduced in v0.13.0 to require referencing tool packages to declare cue cmd tasks, is now on by default.

The new closedness algorithm has been adjusted to fix a number of "field not allowed" bugs, particularly in the form of regressions compared to the old evaluator.

A particularly slow part of the new closedness algorithm has been optimized, resulting in improvements of up to 10x in wall times for some projects.

cmd/cue

:warning: cue def --strict jsonschema:, deprecated in favor of cue def jsonschema+strict: in v0.11.0, is no longer supported.

Two bugs have been fixed in the new cue trim algorithm where disjunctions could incorrectly select defaults or be treated as ambiguous.

Encodings

cue get crd introduces support for extracting Kubernetes Custom Resource Definitions (CRDs) as CUE definitions; see cue help get crd.

The new encoding/jsonschema.ExtractCRDs Go API exposes the extractor used by cue get crd.

The JSON Schema decoder has gained support for dependencies, bringing the pass rate for the official JSON Schema test suite up from 79% to 81%.

Go API

cue/load now provides module information via the new Instance.ModuleFile field.

:warning: In order to support the above change, the mod/modfile.File.Format method has been moved to the function mod/modfile.Format: an API-breaking change but necessary to avoid a cyclic package dependency.

cue/parser gains a new Config API, which allows other packages like cue/load to properly apply the right parser options such as the CUE language version.

:warning: In order to support the above change, the cue/parser.Config.ParseFile and cue/build.ParseFile signatures have now changed (incompatibly) to add a parser.Config argument, enabling the CUE language version to be passed through.

Fix an issue where cue.Value.Decode did not work with *math/big.Float types.

Full list of changes since v0.13.0

* internal/core/adt: inline combinedFlags helpers by @mpvl in [dffc5c] * internal/core/adt: refactor combinedFlags from bit-packed uint32 to struct by @mpvl in [f44480] * ai: specify use of git codereview by @mpvl in [f34d9a] * ai: allow certain tools unconditionally by @mpvl in [80473c] * internal/core/adt: revert the top-level parallelism of EvalV{1,2} tests by @mvdan in [6c3270] * internal/core/adt: silence tests once again by @mvdan in [4f388b] * cue/load: buffer non-regular input files like we do for "-" by @mvdan in [fbb484] * internal/ci: make the writing of files reusable by @mvdan in [649bb6] * internal/ci: place input parameters inside an #in struct by @mvdan in [92d4ba] * cmd/cue: add testscript for issue 4009 by @mvdan in [fb631b] * ai: adjustments to code by @mpvl in [4bdc44] * ai: initial Claude.md by @mpvl in [5530af] * cue/token: add []byte content to token.File by @cuematthew in [3e5061] * internal/ci/base: add helpers for OS + Go version matrices by @mvdan in [f5ad0d] * tools/trim: only consider defaults when we're not ignoring the conjunct by @cuematthew in [b95b21] * internal/encoding/yaml: properly report disjunction errors by @kilpkonn in [c7cc1a] * pkg/encoding: add test cases for disjunction errors with Validate by @kilpkonn in [518993] * lsp/fscache: Force parsing mode to always be ParseComments by @cuematthew in [741955] * internal/core/adt: clear close check by @mpvl in [d8451d] * internal/core/adt: exclude more cases for adding defID by @mpvl in [b12697] * internal/core/adt: add tests for Issue 4000 by @mpvl in [b20058] * internal/core/adt: fix panic on remnant notifications by @mpvl in [faecce] * internal/core/adt: add cycle detection by @mpvl in [0740c0] * internal/core/adt: remove requirement type by @mpvl in [f0f4ce] * internal/core/adt: update tests by @mpvl in [83f3c8] * internal/lsp: Move much of our own code to internal/lsp by @cuematthew in [6d1013] * internal/ci: add base steps for the common Go checks and staticcheck by @mvdan in [a79c3d] * internal/ci: include more useful string values in base by @mvdan in [a78e43] * internal/ci: split Linux runners into "small" and "large" by @mvdan in [6d3530] * internal/ci: allow gerritHubHostname to be overridden by @myitcv in [5cc6d8] * internal/core/adt: new data structure for reqSets by @mpvl in [3f23e1] * internal/core/adt: add generation tracking to closedness info by @mpvl in [507751] * internal/core/adt: add generation tracking to prevent stale context usage by @mpvl in [19e2c7] * tools/flow: avoid looking like we're using `go run` for real by @mvdan in [5f4bd3] * update dependencies for v0.14.0-rc.1 by @mvdan in [0c84f9] * cue/stats: remove unused CloseIDElems counter by @mpvl in [930a98] * internal/core/adt: fix notification CloseInfo by @mpvl in [0ccf38] * cmd/cue/cmd: add tests for 3981 by @mpvl in [840ff2] * internal/core/adt: add tests for 3981 by @mpvl in [78e1a1] * internal/core/adt: ToDataAll rewrite by @mpvl in [2804f1] * internal/ci: allow additional cache modes by @jpluscplusm in [9afca7] * encoding/jsonschema: implement dependencies by @rogpeppe in [9620ea] * internal/cmd/cue-ast: use cue/load with SkipImports=true by @mvdan in [45ab65] * gopls/cache: add test for imports of non-ambiguous non-default modules by @cuematthew in [e4cdcd] * internal/core/adt: remove CloseIDElems counter by @mpvl in [903383] * internal/core/adt: add counters before rewrite by @mpvl in [c8256c] * internal/core/adt: hoist loop by @mpvl in [24cc41] * internal/core/adt: change signature by @mpvl in [5ee425] * internal/core/adt: remove replaceID.add field by @mpvl in [d987d8] * gopls/cache: rework normalizeImportPath by @cuematthew in [f30926] * internal/ci: start publishing the module by @mvdan in [05a3d6] * internal/ci: carve out into a separate module to be published by @mvdan in [17177e] * internal/ci: do a little bit more for trybot jobs in the base package by @mvdan in [1004c3] * internal/ci: apply the top-level githubactions.#Workflow schema by @mvdan in [d9aaac] * gopls/cache: ignore imports of stdlib by @cuematthew in [eac972] * internal/core/adt: revert CL 1208898 for evalv2 by @mvdan in [1789dd] * tools/trim: Work harder to discover disjunction branches by @cuematthew in [7f38bb] * internal/golangorgx: remove the unused tools/facts package by @mvdan in [60c36a] * internal/golangorgx/vendoring: remove by @mvdan in [4d6663] * cmd/cue/cmd: speed up use of sleeps to test concurrent commands by @mvdan in [13461f] * internal/ci: rename "ubuntu" runner to "linux" by @mvdan in [8250c1] * internal/golangorgx: delete unused and Go-specific packages by @mvdan in [dbed9a] * internal/golangorgx: remove unused gopls/util packages by @mvdan in [9ca960] * internal/gopls: rewrite gopls/cuelsp by @cuematthew in [a23afd] * gopls/lsprpc: Adjust gopls to cuelsp and simplify by @cuematthew in [ca86fb] * internal/gopls: remove unnecessary old command alias by @cuematthew in [4752c0] * internal/gopls: renaming of gopls to cuelsp by @cuematthew in [8935ee] * internal/gopls: delete a lot of code by @cuematthew in [3fb304] * all: make some txtar table tests parallel by @mvdan in [2d182a] * internal/golangorgx: remove gopls re-vendoring script by @mvdan in [3114a2] * update all direct dependencies by @mvdan in [2df1b5] * internal/core/adt: filter notifications by @mpvl in [971ee4] * internal/core/adt: keep track of notification counts by @mpvl in [18b7df] * internal/core/adt: run evaluator tests in parallel by @mvdan in [991e54] * internal/pkg: fix a race when contexts concurrently load std imports by @mvdan in [79798c] * internal/core/adt: clean up orphaned arcs by @mpvl in [a39797] * internal/core/adt: mark inlined nodes for cleanup by @mpvl in [e92f16] * internal/core/adt: prune parent processing by @mpvl in [1b7a71] * internal/core/adt: trigger more diffs in stats by @mpvl in [f38950] * internal/core/adt: clean up use of Vertex.state by @mpvl in [fb4f50] * internal/core/adt: basic memory management for evalv3 by @mpvl in [4ebd05] * internal/core/adt: check for nil state in parent disjunction by @mpvl in [4638be] * internal/core/adt: delay processing of pattern constraints by @mpvl in [ab40e7] * internal/core/adt: handle equal disjuncts by @mpvl in [654008] * internal/core/adt: do not set error for nil task state by @mpvl in [495f60] * internal/core/adt: reuse more lets by @mpvl in [b96ed2] * tools/trim: handle error cases in API by @mpvl in [bd237a] * internal/core/adt: allow unprocessed by @mpvl in [4d894f] * internal/core/adt: remove filter for let sharing by @mpvl in [521d44] * internal/core/adt: rearrange newNodeContext by @mpvl in [f42b36] * internal/core/adt: mark nodeContext.clone as v2 by @mpvl in [b7d9b6] * internal/core/adt: add benchmark for Issue 3334 by @mpvl in [99494a] * internal/core/adt: more eagerly update stats in tests by @mpvl in [2862a1] * internal/ci: skip "check git tags" when testing CLs by @mvdan in [510052] * internal/ci: use shorter Namespace runner profile aliases by @mvdan in [6f5079] * internal/ci: run tests with GOARCH=386 on protected branches only by @mvdan in [673b49] * internal/robustio: add stub js/wasm getFileID implementation by @cuematthew in [c13484] * internal/core/adt: normalize how the tests configure the runtime by @mvdan in [a6abf3] * internal/cuetxtar: do not load the root cuelang.org/go CUE module by @mvdan in [7a237e] * cue/load: deduplicate handling missing files for ad-hoc packages by @mvdan in [4cb33d] * cue/load: reject unsupported cue.mod files by inspecting open errors by @mvdan in [c80fda] * internal/robustio: Copy from upstream and consolidate use by @cuematthew in [454497] * internal/core/adt: remove unused test code by @mvdan in [ee299f] * internal/modpkgload: Make Package keep its set of files by @cuematthew in [6405ce] * internal/core: remove a few unnecessary trailing Fprintln calls in tests by @mvdan in [c2f57e] * internal/lsp/fscache: Create CUECacheFS and OverlayFS by @cuematthew in [955e7f] * encoding/jsonschema: parallelize vendor_external.go a bit by @mvdan in [3b1003] * internal/ci: drop "evict caches" nightly cron job by @mvdan in [206e82] * cmd/cue: implement cue get crd by @rogpeppe in [a6df86] * internal/ci: cache staticcheck via nscloud-cache-action by @mvdan in [ff4639] * encoding/jsonschema: implement ExtractCRDs by @rogpeppe in [e2835d] * internal/ci: assume the CI machines are reasonably modern by @mvdan in [7bddda] * cmd/cue: skip tests using sleeps in short mode by @mvdan in [f7e5f8] * internal/ci: skip the test cache via GOFLAGS=-count=1 by @mvdan in [d3da40] * internal/ci: switch to Namespace GitHub Actions runners by @mvdan in [f61538] * internal/core/adt: delay stringifying nodes in errors via fmt.Stringer by @mvdan in [7e4dff] * internal/core/adt: ensure emptyNode is inert by @mpvl in [2adf5e] * mod/modfiledata: rename File.ModulePath() to File.ModuleRootPath() by @cuematthew in [b8a14d] * mod/modpkgload: correct calculation of pkg.fromExternal by @cuematthew in [a8ba68] * mod/modpkgload: capture faulty behaviour of pkg.fromExternal by @cuematthew in [c23708] * cue/ast: Ensure ImportPath.Canonical copes with host-only paths by @cuematthew in [703c22] * mod/modpkgload: Make LoadPackages support nil shouldIncludePkgFile by @cuematthew in [19d3ab] * internal/core/adt: use custom int set implementation for visited by @rogpeppe in [12ec0e] * all: simplify three iterator uses with range-over-func by @mvdan in [bb2494] * internal/core/adt: remove cycle check by @mpvl in [d92e91] * internal/core/adt: add tests for 3978 by @mpvl in [81795a] * internal/core/adt: revert 1217228 by @mpvl in [47912f] * cmd/cue/cmd: fix versiontest test by @rogpeppe in [2bbc35] * doc/ref/spec.md: support struct and list for equality by @mpvl in [d58d1f] * internal/core/compile: add version check for builtins by @mpvl in [985e69] * internal/core/adt: add builtin for user-defined errors by @mpvl in [3ad866] * internal/core/adt: rename CallContext.Expr by @mpvl in [64ffd0] * doc/ref/spec.md: document error builtin by @mpvl in [be18b0] * internal/ci: update Go and goreleaser versions by @mvdan in [8d463d] * update golang.org/x/... dependencies by @mvdan in [e88d16] * internal/core/adt: support unary == and != for all types by @mpvl in [030658] * internal/core/adt: comparison of values of different types by @mpvl in [34fadd] * internal/core/adt: consider required fields for final values in Equal by @mpvl in [64d578] * internal/core/adt: support comparsion of structs by @mpvl in [2b7dfb] * internal/core/adt: fix error gobbling of internal nodes by @mpvl in [665f69] * internal/core/adt: add tests for Issue 3977 by @mpvl in [996e8b] * cue/load: pass language version down to parser by @rogpeppe in [d56775] * cue/load: use error registry in SkipImports mode by @rogpeppe in [5a0461] * cue/parser: enable cueexperiment.File throughout by @mpvl in [86f7a6] * internal/cueexperiment: support for per-file experiments by @mpvl in [52988e] * internal/core/adt: simplify recursive getReqSets calls by @mvdan in [2d4263] * cmd/cue: remove CUE_SYNTAX_OVERRIDE support by @rogpeppe in [450ba7] * internal/core/adt: do not repeat replaceIDs calls by @mvdan in [8241d9] * cue/load: actually populate build.Instance.ModuleFile correctly by @rogpeppe in [48ec27] * internal/core/adt: handle incomplete values in comparators by @mpvl in [c92504] * internal/core/adt: use replaceIDs incrementally in checkTypos by @mvdan in [16c782] * internal/core/adt: rewrite replaceIDs to scale better by @mvdan in [b4212a] * internal/core/adt: Equal should take defaults when appropriate by @mpvl in [361384] * internal/core/adt: limit closedness check in Equal by @mpvl in [e8e868] * internal/core/adt: merge validate package with adt by @mpvl in [26e159] * internal/core/adt: add tests for incomplete lists comparison by @mpvl in [e67898] * cue/parser: add Config type by @rogpeppe in [211309] * cue/load: include module file in build.Instance by @rogpeppe in [ab9ac5] * internal/core/adt: use pointers for snapshot and result in nodeContext by @mvdan in [1a2d2e] * internal/core/adt: add tests for Issue 3929 by @mpvl in [777723] * internal/core/adt: handle yields in Resolve by @mpvl in [81e88e] * internal/core/adt: add tests for Issue 3972 by @mpvl in [0ac391] * internal/core/adt: make small iota enums consistently uint8 by @mvdan in [ce44cf] * internal/core/adt: allow lookup across disjunction boundary by @mpvl in [20afef] * internal/core/adt: apply vertex rewriting to blocked tasks by @mpvl in [d71305] * internal/core/adt: add tests for 3967 by @mpvl in [5c5f97] * cuecontext: revert support for CUE_EXPERIMENT by @mpvl in [2a7093] * internal/pkg: only use %s with stringifiable builtin errors by @mvdan in [fc4ca4] * cue/parser: prohibit definition identifiers for package names by @rogpeppe in [548c23] * internal/core/export: avoid panic on non-regular identifiers for packages by @rogpeppe in [8bbf9b] * cmd/cue: add test cases for issue 3968 by @rogpeppe in [e0010d] * cmd/cue: add testscript for `exp gengotypes` output filenames by @mvdan in [b20d1a] * all: swap math/rand for math/rand/v2 where it affects behavior by @mvdan in [82ff07] * all: remove a bit of dead code by @mvdan in [ab546c] * cmd/cue: properly document the keepvalidators experiment by @mvdan in [398cda] * all: swap math/rand for math/rand/v2 where it doesn't affect behavior by @mvdan in [db84c6] * cmd/cue: fix error on cue mod rename with a bad module by @rogpeppe in [9f9eba] * cmd/cue: add test case for modrename with a bad module by @rogpeppe in [90d9b0] * internal/core/adt: also create embed scope for disjunctions by @mpvl in [c92590] * internal/core/adt: add tests for 3964 by @mpvl in [2597e9] * internal/core/adt: fix default logic by @mpvl in [8dc494] * cueexperiment: export Config type and use it by @mpvl in [33a179] * internal/core/adt: add tests for 3958 by @mpvl in [5d162d] * README: minor updates by @jpluscplusm in [eb5408] * internal/core: start removing pre-toposort field ordering code by @mvdan in [6fd9f7] * internal/core/adt: don't check removed scopes by @mpvl in [86ef53] * internal/core/adt: track scope of closed() by @mpvl in [c6abd6] * internal/core/adt: do not remove embed groups by @mpvl in [191a0e] * internal/core/adt: adjustments to closedness algo by @mpvl in [6746a8] * internal/core/adt: change ellipsis to fix hidden bug by @mpvl in [e9a4d1] * internal/core/adt: update tests for 3833 and 3839 before fix by @mpvl in [f7e46e] * internal/core/adt: add tests for Issue 3920 by @mpvl in [75660e] * internal/core/adt: also typo check disjuncts by @mpvl in [dba18d] * internal/core/adt: add tests for Issue 3957 by @mpvl in [deed94] * internal/core/adt: remove most validator simplifications by @mpvl in [487e52] * internal/core/adt: copy bounds.txtar by @mpvl in [97927b] * cmd/cue: allow "access denied" testscript to pass on Wine by @mvdan in [250524] * cmd/cue: tweak two more testscripts to be portable by @mvdan in [975831] * cue/ast: improve ImportPath.Canonical by @rogpeppe in [12f319] * cue/ast: add more tests for ImportPath.Canonical by @rogpeppe in [3444f2] * cue: fix decoding into big.Float by @haoqixu in [dcc308] * cmd/cue: add false, echo, and cat testscript commands by @mvdan in [d1210a] * cmd/cue: drop debug lines from testscripts by @mvdan in [c0c607] * cue: add tests for issue 3927 by @haoqixu in [7cfda3] * cue: use multiple sizes in BenchmarkLargeValueMarshalJSON by @mvdan in [4e0e42] * update all other dependencies by @mvdan in [b226e1] * update golang.org/x/... by @mvdan in [c947a0] * all: rely on iterators being available since Go 1.23 by @mvdan in [100a7c] * internal/golangorgx: remove files build-tagged for old Go versions by @mvdan in [15536a] * internal/golangorgx/tools/analysisinternal: remove by @mvdan in [830ad6] * cue/cuecontext: always enable embedding by @rogpeppe in [e2e071] * internal/golangorgx/tools/typesinternal: remove by @mvdan in [6024ad] * internal/golangorgx/gopls/cache/methodsets: remove by @mvdan in [881004] * internal/golangorgx/tools/tokeninternal: remove by @mvdan in [518f13] * cue/load: make NoFilesError use position arguments for paths by @mvdan in [5b6d50] * cmd/cue: remove dev.txtar testscript by @mvdan in [d0cb1d] * cmd/cue: deprecate --strict flag in favor of the jsonschema filetype tag by @mvdan in [00d3a8] * internal/cueexperiment: enable cmdreferencepkg by default by @mvdan in [16444f] * cmd/cue: turn all "cmd" testscripts away from legacy id/kind fields by @mvdan in [8b11b9] * internal/cueexperiment: toposort is always on now by @mvdan in [06de12] * internal/cueexperiment: keep record of how experiments advanced by @mvdan in [fa77cf] * internal/cueexperiment: file embedding is always on now by @mvdan in [23f252] * all: ensure we init cuedebug and cueexperiment before every use by @mvdan in [fc8460] * cmd/cue: consistently use the same top-level command context by @mvdan in [60e570] * cmd/cue: add more CUE_DEBUG=http tests by @mvdan in [6119be] * internal/core/adt: partially revert CL 1214950 by @mpvl in [e1fd26] * internal/core/adt: add tests for 3946 by @mpvl in [d2adad] * internal/cueversion: avoid dependency on golang.org/x/mod by @rogpeppe in [94630d] * internal/core/adt: do not always filter disjuncts by @mpvl in [50e6b3] * internal/core/adt: add tests for 3947 by @mpvl in [a1f096] * cmd/cue: remove stale testscript txtar files for CUE_EXPERIMENT=modules=0 by @mvdan in [b7b7cc] * internal/cueexperiment: tweak the docs for all experiment flags by @mvdan in [198171] * cmd/cue: consistently use tab indentation in help texts by @mvdan in [9113b8] * cmd/cue: drop evalv3=0 testscripts by @mvdan in [a27b03] * cmd/cue: implement mirror --dry-run by @rogpeppe in [707f96] * internal/cueversion: bump languageVersion for v0.14 by @mvdan in [9eb4c1] * internal/mod/modfiledata: factor out of mod/modfile by @rogpeppe in [da6a63]