Agent Substrate Files

What's Changed

⚠️ Breaking Changes

• snapshot rootfs uppers as one tar per container by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2110
• actorevent: write each actor event to one place, with a shared OTEL_*_EXPORTER parser by @baizhenyu in https://github.com/agent-substrate/substrate/pull/2193
• ateom-microvm: snapshot durable-dir volumes as one tar per volume by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2283

Bug Fixes

• Clean up actors when ateom terminates by @sfunkenhauser in https://github.com/agent-substrate/substrate/pull/1888
• Select kvm-clock for amd64 micro-VM guests by @swiftdiaries in https://github.com/agent-substrate/substrate/pull/1945
• Reject repointing an actor that owns a snapshot to a different storage location by @laoj2 in https://github.com/agent-substrate/substrate/pull/2014
• ateapi: stop logging the raw PostgreSQL connection string (which might contain db password) at startup by @git286 in https://github.com/agent-substrate/substrate/pull/1833
• atecontroller: add a zero-count bucket to empty exponential histograms by @ericcurtin in https://github.com/agent-substrate/substrate/pull/1933
• Support workers with multiple IPs by @sfunkenhauser in https://github.com/agent-substrate/substrate/pull/2052
• router: report no_capacity only for a ResourceExhausted cause by @ericcurtin in https://github.com/agent-substrate/substrate/pull/1929
• fix(serverboot): honor OTEL_METRICS_EXPORTER=none by @QuentinBisson in https://github.com/agent-substrate/substrate/pull/1878
• atelet: try each matching image pull credential in turn by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2104
• Support --platform build flag in orchestrator image by @maxsmythe in https://github.com/agent-substrate/substrate/pull/1901
• ateapi: reach an atelet again after its node restarts by @ericcurtin in https://github.com/agent-substrate/substrate/pull/2118
• atelet: measure what a snapshot image costs, not how long it is by @chw120 in https://github.com/agent-substrate/substrate/pull/931
• Only require a pause image for gVisor sandboxes by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2144
• fix: reload the pod-identity CA pool for client-cert verification by @rashadism in https://github.com/agent-substrate/substrate/pull/1930
• fix: reload frozen CA pools in atelet, ateapiauth, and the worker-side atelet dialer by @rashadism in https://github.com/agent-substrate/substrate/pull/1932
• Preserve system-info registrations during stale lifecycle cleanup by @swiftdiaries in https://github.com/agent-substrate/substrate/pull/1944
• [Flaky test fix] TestSandboxAssetPrewarmDownloads - atelet: make the sandbox prewarm worker joinable, fixing the race by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/1940
• fix(objectstorage): stop initial range downloads on close by @ktsoator in https://github.com/agent-substrate/substrate/pull/2176
• contextlogging: redact protobuf attributes in the shared slog handler by @git286 in https://github.com/agent-substrate/substrate/pull/1942
• WorkerPool can set the ServiceAccount of its worker pods by @danielle-mustillo in https://github.com/agent-substrate/substrate/pull/2219
• router: report a shed request as no_capacity after a full fleet by @JeffLuoo in https://github.com/agent-substrate/substrate/pull/2138

Features: Networking and Egress

• Add gcp secret manager as a credential provider in its own go module by @yufan-su in https://github.com/agent-substrate/substrate/pull/1991
• egress policy: add an actor JWT value source to CredentialHeader by @thompsonmax in https://github.com/agent-substrate/substrate/pull/1960
• Implement passthrough egress policy by @yanavlasov in https://github.com/agent-substrate/substrate/pull/2019
• docs: how an actor exports telemetry under the egress lockdown by @git286 in https://github.com/agent-substrate/substrate/pull/2120
• Passthrough dial address resolved by the gateway by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2045
• Update agentgateway and enable ingress and egress test coverage by @EItanya in https://github.com/agent-substrate/substrate/pull/2158
• router authn/z: Prefactor; extract router flag binding to function by @ahmedtd in https://github.com/agent-substrate/substrate/pull/2161
• remove non mitm dataplanes by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2015
• Router no longer logs request credentials or query strings by @git286 in https://github.com/agent-substrate/substrate/pull/2206
• kubectl-ate: share the missing-actor check across egress-policy commands by @ericcurtin in https://github.com/agent-substrate/substrate/pull/2191
• graduate credential injection from experimental by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2101
• ateapi: describe no_capacity without "free worker" by @JeffLuoo in https://github.com/agent-substrate/substrate/pull/2137
• SystemInfo: Implement system-roots.ate.dev trust bundle by @ahmedtd in https://github.com/agent-substrate/substrate/pull/2049

Features: Security and Identity

• feat(credentials): narrow a namespace grant by secret name or label by @jjamroga in https://github.com/agent-substrate/substrate/pull/1859
• ClusterTrustBundle v1 support by @howardjohn in https://github.com/agent-substrate/substrate/pull/1924
• atelet: pull image credentials from the kubelet credential provider by @BenTheElder in https://github.com/agent-substrate/substrate/pull/917
• ateapi: Implement Policy Governance CRUD Access Policy APIs by @SHRUTI6991 in https://github.com/agent-substrate/substrate/pull/1965
• oidcdiscovery: build the OpenID discovery document by @thompsonmax in https://github.com/agent-substrate/substrate/pull/1963
• kubectl-ate: add --validity flag to admin make-ca-pool by @pintonunes in https://github.com/agent-substrate/substrate/pull/2058
• localjwtauthority: thumbprint key IDs and a configurable signing algorithm by @thompsonmax in https://github.com/agent-substrate/substrate/pull/1961
• security: only run sandbox runtime assets from the static files dir by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2269
• only enable guest debug console with --guest-debug by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2291
• ateapi: [Part 2] Runtime Authorization check for Actor and ActorTemplate CRUD by @SHRUTI6991 in https://github.com/agent-substrate/substrate/pull/1980

Features: Observability

• benchmarking/locust: harvest server-side telemetry from Prometheus by @Nishanth29 in https://github.com/agent-substrate/substrate/pull/1725
• ateomstats: add per-activation usage state and the usage emitter by @baizhenyu in https://github.com/agent-substrate/substrate/pull/2005
• otlprelay: use a short temp dir for the test socket by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2270

Features: Workers and Actors

• workerservice: validate RequestActorSuspend requests by @shrutiyam-glitch in https://github.com/agent-substrate/substrate/pull/2023
• atelet: use a glibc base image to support credential providers by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2108
• ateapi: prefer less-loaded workers via power of two choices by @chw120 in https://github.com/agent-substrate/substrate/pull/1915
• atelet,ateom: restore local pause checkpoints directly from LocalSnap by @chw120 in https://github.com/agent-substrate/substrate/pull/1876
• csi: add publishContext to actor volumes by @hajiler in https://github.com/agent-substrate/substrate/pull/1578
• Fix grpc status error when creating actors and tags by @juli4n in https://github.com/agent-substrate/substrate/pull/2263
• fake-dataplane: add a fake dataplane skeleton by @haiyanmeng in https://github.com/agent-substrate/substrate/pull/2272
• ocispec: raise actor RLIMIT_NOFILE from 1024 to 1048576 by @ygao-g in https://github.com/agent-substrate/substrate/pull/2242

Features: Install and Operations

• benchmarking: agent-session workload — a scripted coding agent that suspends while the LLM thinks by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/1934
• ci: add automated release notes generation by @stshrive in https://github.com/agent-substrate/substrate/pull/2018
• cordon-control-plane: share one pool, give postgres its own by @haiyanmeng in https://github.com/agent-substrate/substrate/pull/2056
• feat(benchmarking): add substrate spawn benchmark by @sairajp-rewind in https://github.com/agent-substrate/substrate/pull/1969
• Let glutton actors consume CPU, driven from boomer dynconfig by @maxsmythe in https://github.com/agent-substrate/substrate/pull/1875
• ate-setup: add the uniform configuration package by @stshrive in https://github.com/agent-substrate/substrate/pull/2012

Documentation

• Add Yolo the Octopus logo and quick navigation links to README by @brandonroyal in https://github.com/agent-substrate/substrate/pull/2239

Other Changes

• atepg: name the row in list decode errors by @ericcurtin in https://github.com/agent-substrate/substrate/pull/2000
• ategcs: rename and move to pkg/ for reuse outside atelet by @ILLUM1N0X in https://github.com/agent-substrate/substrate/pull/2013
• Remove reserved fields from proto and update back-compat instruction in AGENTS.md by @laoj2 in https://github.com/agent-substrate/substrate/pull/1985
• remove ssl_cert_dir and add clearer guidance for actor trust store by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2044
• ateom: share atunnel wiring in ateomtunnel by @ericcurtin in https://github.com/agent-substrate/substrate/pull/1959
• Delete worker pods in terminal states by @sfunkenhauser in https://github.com/agent-substrate/substrate/pull/2051
• benchmarking(agentsession): define the session script in YAML, selectable per run by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/2046
• Introduce a dedicated type for API errors. by @juli4n in https://github.com/agent-substrate/substrate/pull/2072
• benchmarking(agentsession): run a script from a file mounted into the worker by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/2047
• ateom-microvm: use pflag like the other binaries by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2096
• Egress gateway no longer fetches an actor's egress policy twice when requests race by @thompsonmax in https://github.com/agent-substrate/substrate/pull/2095
• atelet: take the DATA snapshot file set from ateom by @HavenXia in https://github.com/agent-substrate/substrate/pull/2028
• SystemInfo: Allow multiple trustbundle names in one file by @ahmedtd in https://github.com/agent-substrate/substrate/pull/2048
• Remove onResume.fromData and the DATA_ON_GOLDEN restore scope by @shrutiyam-glitch in https://github.com/agent-substrate/substrate/pull/2105
• Delete CLAUDE.md in favor of AGENTS.md. by @juli4n in https://github.com/agent-substrate/substrate/pull/2125
• Don't crash actors on transport errors when calling atelet by @laoj2 in https://github.com/agent-substrate/substrate/pull/2060
• Add AGENTS.md files to key API source directories by @juli4n in https://github.com/agent-substrate/substrate/pull/2131
• benchmarking: add workerMemory to size benchmark worker pods by @maxsmythe in https://github.com/agent-substrate/substrate/pull/2109
• atepg: move expired-lease cleanup off the lock acquisition path by @mvanhorn in https://github.com/agent-substrate/substrate/pull/1880
• Introduce separate PostgreSQL roles for schema ownership and normal application access by @iplay88keys in https://github.com/agent-substrate/substrate/pull/1752
• Finish moving DNS to its own package ateomnet/dns by @bowei in https://github.com/agent-substrate/substrate/pull/2115
• benchmarking(agentsession): classify router HTTP errors like their gRPC origins by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/2145
• benchmarking: size the runner pod per test, and fan out the agentsession shutdown by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/2139
• podcertcontroller: skip ClusterTrustBundle updates when nothing changed by @ericcurtin in https://github.com/agent-substrate/substrate/pull/2164
• move Star History section to top of readme by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2167
• metrics: rename the scheduler outcome no_free_worker to no_capacity by @JeffLuoo in https://github.com/agent-substrate/substrate/pull/1895
• docs: fix k8s downwardAPI volume doc broken url by @doniacld in https://github.com/agent-substrate/substrate/pull/2192
• Make ate-apiserver report errors using apierror. by @juli4n in https://github.com/agent-substrate/substrate/pull/2129
• objectstoresnapshot/v1: add node and control snapshot plugin API by @ILLUM1N0X in https://github.com/agent-substrate/substrate/pull/2070
• When the call to atelet fails with UNAVAILABLE, propagate the error to clients by @juli4n in https://github.com/agent-substrate/substrate/pull/2196
• atelet: show the real error when an actor directory cannot be reset by @ericcurtin in https://github.com/agent-substrate/substrate/pull/2142
• ci: report e2e-test from the matrix job directly by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2225
• Fix error reporing for AccessPolicy resources. by @juli4n in https://github.com/agent-substrate/substrate/pull/2214
• MintActorJWT no longer takes the actor's UID by @thompsonmax in https://github.com/agent-substrate/substrate/pull/2207
• envoy: fix one connection per downstream plus cleanups by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2213
• benchmarking: Prometheus can now run on the tainted load-generator pool by @Nishanth29 in https://github.com/agent-substrate/substrate/pull/2215
• e2e: run the identity suite with the others by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2227
• ateom-microvm: replace shell commands with Go by @destijl in https://github.com/agent-substrate/substrate/pull/2147
• benchmarking: add a personal-assistant agent-session script and a dwell op by @aditya-shantanu in https://github.com/agent-substrate/substrate/pull/2230
• CSI volume support: follow-up improvements by @anishgang in https://github.com/agent-substrate/substrate/pull/1729
• ci: cache Go and envoy-dataplane builds in e2e by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2229
• egress: make egress dial the port the actor connected to and ignore authority ports by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2228
• Set up bundled PostgreSQL identities in ate-setup by @iplay88keys in https://github.com/agent-substrate/substrate/pull/2025
• tarutil: refuse device nodes and unvetted xattrs on extract by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2111
• fix comment on pool creation in ci by @LiorLieberman in https://github.com/agent-substrate/substrate/pull/2231
• Mask actor_jwt in snapshot plugin requests in logs by @git286 in https://github.com/agent-substrate/substrate/pull/2265
• atenet-egress: raise mitm_internal circuit breakers above Envoy's 1024 default by @yufan-su in https://github.com/agent-substrate/substrate/pull/2255
• e2e: run demo tests in parallel, give each sandbox class the whole node, build for host arch only by @BenTheElder in https://github.com/agent-substrate/substrate/pull/2252
• Refactor: atunnel DNS server returns a Stop()able stub by @bowei in https://github.com/agent-substrate/substrate/pull/2251
• benchmarking: tolerate ate.dev/sandboxClass=shared on the benchmark WorkerPool by @Nishanth29 in https://github.com/agent-substrate/substrate/pull/2284
• ateapi: report missing StorageClasses consistently during volume creation by @cloudonly in https://github.com/agent-substrate/substrate/pull/2233
• fix(atenet): return the final resume error after budget expiry by @ktsoator in https://github.com/agent-substrate/substrate/pull/2248
• netns: serialize setSysctl's /proc/sys remount by @yufan-su in https://github.com/agent-substrate/substrate/pull/2292
• benchmarking: cap the runner Job name at 63 characters by @maxsmythe in https://github.com/agent-substrate/substrate/pull/2184
• ate-setup: save used settings locally after installation by @stshrive in https://github.com/agent-substrate/substrate/pull/2223
• Worker: Report node HardwareIdentity during Worker registration by @shrutiyam-glitch in https://github.com/agent-substrate/substrate/pull/2282
• Cleanup actor on nodes even when the worker is crashed by @msau42 in https://github.com/agent-substrate/substrate/pull/2222
• AGENTS.md: keep JWT and JWK code on the standard library by @thompsonmax in https://github.com/agent-substrate/substrate/pull/2301
• GOVERNANCE: fix role name in Becoming a Maintainer section by @haiyanmeng in https://github.com/agent-substrate/substrate/pull/2305
• atenet: bind Envoy admin to loopback on router and egress gateway by @destijl in https://github.com/agent-substrate/substrate/pull/2310
• ateapi: remove SnapshotConfig.on_pause, pause captures the on_commit scope by @dberkov in https://github.com/agent-substrate/substrate/pull/2309

New Contributors

• @ericcurtin made their first contribution in https://github.com/agent-substrate/substrate/pull/2000
• @QuentinBisson made their first contribution in https://github.com/agent-substrate/substrate/pull/1878
• @mvanhorn made their first contribution in https://github.com/agent-substrate/substrate/pull/1880
• @pintonunes made their first contribution in https://github.com/agent-substrate/substrate/pull/2058
• @doniacld made their first contribution in https://github.com/agent-substrate/substrate/pull/2192
• @rashadism made their first contribution in https://github.com/agent-substrate/substrate/pull/1930
• @destijl made their first contribution in https://github.com/agent-substrate/substrate/pull/2147
• @ktsoator made their first contribution in https://github.com/agent-substrate/substrate/pull/2176
• @anishgang made their first contribution in https://github.com/agent-substrate/substrate/pull/1729
• @brandonroyal made their first contribution in https://github.com/agent-substrate/substrate/pull/2239
• @cloudonly made their first contribution in https://github.com/agent-substrate/substrate/pull/2233
• @danielle-mustillo made their first contribution in https://github.com/agent-substrate/substrate/pull/2219

Full Changelog: https://github.com/agent-substrate/substrate/compare/v0.3.0...v0.4.0

Committers in this release

• @aditya-shantanu
• @ahmedtd
• @anishgang
• @baizhenyu
• @BenTheElder
• @bowei
• @brandonroyal
• @chw120
• @cloudonly
• @danielle-mustillo
• @dberkov
• @destijl
• @doniacld
• @EItanya
• @ericcurtin
• @git286
• @haiyanmeng
• @hajiler
• @HavenXia
• @howardjohn
• @ILLUM1N0X
• @iplay88keys
• @JeffLuoo
• @jjamroga
• @juli4n
• @ktsoator
• @laoj2
• @LiorLieberman
• @maxsmythe
• @msau42
• @mvanhorn
• @Nishanth29
• @pintonunes
• @QuentinBisson
• @rashadism
• @sairajp-rewind
• @sfunkenhauser
• @SHRUTI6991
• @shrutiyam-glitch
• @stshrive
• @swiftdiaries
• @thompsonmax
• @yanavlasov
• @ygao-g
• @yufan-su