| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| README.md | 2026-07-22 | 99.4 kB | |
| v1.6.0 source code.tar.gz | 2026-07-22 | 7.6 MB | |
| v1.6.0 source code.zip | 2026-07-22 | 10.4 MB | |
| openapi.json | 2026-07-22 | 2.6 MB | |
| cozypkg-checksums.txt | 2026-07-22 | 564 Bytes | |
| cozypkg-windows-arm64.tar.gz | 2026-07-22 | 22.9 MB | |
| cozypkg-darwin-amd64.tar.gz | 2026-07-22 | 26.7 MB | |
| cozypkg-darwin-arm64.tar.gz | 2026-07-22 | 24.8 MB | |
| cozypkg-linux-amd64.tar.gz | 2026-07-22 | 25.4 MB | |
| cozypkg-linux-arm64.tar.gz | 2026-07-22 | 22.9 MB | |
| cozypkg-windows-amd64.tar.gz | 2026-07-22 | 25.6 MB | |
| initramfs-metal-amd64.xz | 2026-07-22 | 155.2 MB | |
| kernel-amd64 | 2026-07-22 | 20.4 MB | |
| nocloud-amd64.raw.xz | 2026-07-22 | 349.0 MB | |
| metal-amd64.raw.xz | 2026-07-22 | 349.0 MB | |
| metal-amd64.iso | 2026-07-22 | 543.5 MB | |
| cozystack-operator-hosted.yaml | 2026-07-22 | 2.8 kB | |
| cozystack-operator-generic.yaml | 2026-07-22 | 2.9 kB | |
| cozystack-operator-talos.yaml | 2026-07-22 | 2.9 kB | |
| cozystack-crds.yaml | 2026-07-22 | 20.0 kB | |
| Totals: 20 Items | 1.6 GB | 0 | |
Cozystack v1.6.0
Cozystack v1.6.0 replaces the Ubuntu + kubeadm bootstrap of tenant Kubernetes workers with Talos Linux driven by Cluster API, completes the etcd-operator v1alpha2 migration with in-place adoption of live clusters, adds OIDC single sign-on for tenant kube-apiservers and per-instance Grafana, introduces the tenant-facing SecurityGroup firewall API, makes tenant resource quotas hierarchical so a sub-tenant can no longer escalate past its parent's budget, gives Keycloak an optional KMS-encrypting database proxy, a separate admin hostname and S3 backups, propagates operator-provided wildcard certificates down to every tenant termination point, and reworks the release pipeline around immutable tags and rc→stable promotion so a stable release is the byte-identical promotion of the release candidate that was tested. A ten-PR sweep also makes application deletion actually reclaim its storage instead of leaking PVCs. The release rolls up every fix from v1.5.1, v1.5.2 and v1.5.3.
Platform components bumped in this release: Talos v1.13.0 → v1.13.6 (KVM guest-escape CVE fixes), etcd-operator v0.4.5 → v0.5.2 (new etcd-operator.cozystack.io/v1alpha2 API), Cilium 1.19.3 → 1.19.5, KubeVirt v1.8.4, Velero 1.17.0 → 1.18.1, Vertical Pod Autoscaler 1.3.0 → 1.5.0, Harbor 2.14.2 → 2.15.1, Keycloak 26.5.2 → 26.6.3, LINSTOR 1.33.2 → 1.33.3 with linstor-csi v1.11.2, FoundationDB operator v2.13.0 → v2.30.0, HAMi 2.8.1 → 2.9.0, Percona MongoDB operator 1.21.1 → 1.22.0, OpenBao v2.5.0 → v2.5.1, CoreDNS chart 1.43.2 → 1.46.0, csi-driver-nfs 4.11.0 → 4.13.3, OpenCost 1.111.0 → 1.120.3, NATS 2.11.8 → 2.11.10, linstor-scheduler v0.3.2 → v0.3.6, and ouroboros 0.8.0 → 0.8.1.
Note: Items marked (shipped in v1.5.x) were also released in the v1.5.1, v1.5.2 or v1.5.3 patch releases.
⚠️ Breaking Changes and Required Actions
This is the largest upgrade surface since v1.0. Read this section in full before upgrading — three of the items below can block or wedge the upgrade if their preconditions are not met.
The platform migration targetVersion moves from 45 (v1.5.x) to 54, which means migrations 45 through 53 run as pre-upgrade hooks on the way to v1.6.0.
Pre-upgrade checks
Run these against the management cluster before applying the v1.6.0 Platform Package.
1. etcd adoption needs a reachable backup target (migration 50 — hard-fails the upgrade).
Migration 50 adopts every legacy etcd.aenix.io/v1alpha1 cluster onto the new v1alpha2 operator and takes a mandatory pre-adoption etcd snapshot first. If it cannot resolve the platform-managed snapshot target it exits 1 and halts the upgrade (#2859, [#3270], [#3335]).
:::sh
# Do you have legacy etcd clusters at all? If this prints nothing, migration 50 is a no-op.
kubectl get etcdclusters.etcd.aenix.io -A
# The snapshot target: the cozy-default-etcd strategy and the projected credentials Secret.
kubectl get etcds.strategy.backups.cozystack.io cozy-default-etcd
kubectl get secret cozy-backups-creds -n cozy-velero \
-o jsonpath='{.data.AWS_ACCESS_KEY_ID}{"\n"}'
# The bucket behind it must be healthy.
kubectl get buckets.apps.cozystack.io cozy-backups -n tenant-root
If the Secret or the strategy is missing and you have legacy etcd clusters, either repair the cozy-backups Bucket / backupstrategy-controller first, or accept adopting live etcd without a snapshot by taking the documented escape hatch (see Manual actions required).
2. SeaweedFS naming audit (chart refuses to render for two classes of tenant).
The v1.5.0 SeaweedFS chart bump to 4.31 renamed workloads from chart-based (seaweedfs-*) to release-based (seaweedfs-system-*) names. v1.6.0 pins fullnameOverride: seaweedfs and adopts the running set in place, but two states cannot be adopted automatically and the chart fails the render rather than guess (#3282, [#3339]).
:::sh
# Step 0 — seaweedfs-db ownership. A missing row means the filer metadata is ALREADY lost.
kubectl get cluster.postgresql.cnpg.io -A \
-o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,OWNER:.metadata.annotations.meta\.helm\.sh/release-name,KEEP:.metadata.annotations.helm\.sh/resource-policy'
# Step 1 — classify every SeaweedFS instance (read-only).
hack/seaweedfs-naming-audit.sh
Correction, added after release: the copy of that script shipped in v1.6.0 fails open, so a clean result from it is not evidence. Every kubectl call in the v1.6.0 version is silenced with 2>/dev/null, so a timeout, an RBAC denial or any transient API error produces an empty "all clean" table byte-identical to an honestly clean fleet — on the one script whose output gates a runbook step that deletes PVCs. Fixed in [#3436] and backported as [#3474], both merged 2026-07-28, so v1.6.1 and later carry a version that prints FATAL and exits non-zero instead. Run the audit from a v1.6.1-or-later checkout, read the exit code rather than the table, and re-run it if the only run you have was against the v1.6.0 copy.
Class L needs no action. Class S (installed fresh on 1.5.x) and class MIXED (both naming generations present) must be recovered before upgrading — follow docs/operations/seaweedfs-431-rename-recovery.md. A cluster that went 1.4.x → 1.6.0 directly never renamed and is unaffected.
3. Tenant Kubernetes clusters still on v1.30.
v1.30 leaves the Talos↔Kubernetes support matrix and the chart now refuses to render it. Migration 46 patches live CRs from v1.30 to v1.31, but a GitOps-managed CR is re-overwritten by the next source reconcile (#2931).
:::sh
kubectl get kuberneteses.apps.cozystack.io -A \
-o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,VERSION:.spec.version'
Any row showing v1.30 that is reconciled from Git must have spec.version bumped to v1.31 (or newer) in Git, before or together with the platform upgrade.
4. Hand-made tenant StorageClasses that collide with propagated LINSTOR classes.
Remote-accessible LINSTOR StorageClasses are now created inside each tenant cluster under the same name. A manually created tenant StorageClass with a colliding name (typically replicated) blocks the propagated class and stalls the tenant CSI release (#2872).
:::sh
# Inside each tenant cluster:
kubectl get storageclass replicated -o jsonpath='{.metadata.annotations.meta\.helm\.sh/release-name}{"\n"}'
Delete any such class that is not Helm-managed before upgrading. Infra classes that must stay node-local need an explicit linstor.csi.linbit.com/allowRemoteVolumeAccess: "false" — an absent annotation is treated as remote-accessible.
5. Deprecated etcd tenant-module backup values.
The backup.* block on the etcd tenant module is removed in this release. Confirm nothing still relies on it (#2859).
:::sh
kubectl get etcds.apps.cozystack.io -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\t"}{.spec.backup.enabled}{"\n"}{end}'
6. If you are coming from v1.4.x, the v1.5.0 requirement still applies: the management cluster (and any tenant cluster enabling the Flux addon) must be on Kubernetes 1.33+.
Breaking changes
-
Tenant Kubernetes workers move from Ubuntu + kubeadm to Talos Linux. What changed: worker bootstrap is now driven by
cluster-api-bootstrap-provider-talos(CABPT) with atalos-csr-signersidecar in the Kamaji control-plane pod; workers boot the Talos openstack image streamed in by CDI. Who is affected: every existing tenant Kubernetes cluster. What to do: nothing mandatory — existing machines roll over to Talos workers automatically on the first reconcile after upgrade — but plan for a full worker-pool replacement per tenant cluster, and expect worker disks to be reprovisioned and container images re-pulled. The separatedisk-kubeletPVC is gone;nodeGroups[*].diskSizenow sizes the single Talos system disk (@myasnikovdaniil in [#2931]). -
Kubernetes v1.30 removed from the tenant support matrix. What changed: the
versionenum dropsv1.30; the chart fails the render with an explicit guard. Who is affected: tenants still pinned to v1.30. What to do: migration 46 bumps live CRs tov1.31; GitOps-managed CRs must be bumped in Git (see pre-upgrade check 3) (#2931). -
Worker MachineHealthCheck remediation is now ON by default. What changed: the worker MHC
maxUnhealthymoved from a hard-coded0(remediation effectively disabled) tonodeHealthCheck.maxUnhealthy, defaulting to"50%"— CAPI now deletes and replaces unhealthy worker Machines. Who is affected: every tenant Kubernetes cluster. What to do: setnodeHealthCheck.maxUnhealthy: "0%"to keep the previous behaviour until your fleet is stable on Talos workers (#2931). -
The default
md0node group is no longer merged into every cluster. What changed:nodeGroupsdefaults to{}and the built-inmd0is applied only when no node groups are configured, so it can finally be removed or replaced. Who is affected: clusters that defined their own node groups and were silently also gettingmd0. What to do: nothing — migration 47 pinsmd0explicitly on existing CRs to preserve the live topology; remove it afterwards if you do not want it. Migration 47 is fail-closed: a read or patch failure aborts the upgrade rather than let Helm prune a livemd0MachineDeployment (@myasnikovdaniil in [#2936]). -
Fresh tenant clusters with
nodeGroups: {}come up with zero workers. What changed: the chart no longer managesMachineDeployment.spec.replicas— the cluster-autoscaler owns it alone, seeded fromminReplicas: 0. Who is affected: new clusters that enableaddons.ingressNginx.enabled: trueon a default CR. What to do: either supply a node group withroles: [ingress-nginx]andminReplicas >= 1, or let the autoscaler bring upmd0once the ingress-nginx pods goPending. Upside for existing clusters:helm upgradeno longer drains workers back to a hardcodedreplicas: 2on every platform bump (@IvanHunters in [#3233]). -
etcd moves to
etcd-operator.cozystack.io/v1alpha2. What changed: the legacyetcd.aenix.io/v1alpha1StatefulSet model is replaced by a Membership-API lifecycle; the vendored upstream chart is replaced by a Cozystack-authored one at appVersionv0.5.2, and CRDs move into a dedicatedetcd-operator-crdspackage. Who is affected: every cluster running the etcd tenant module or tenant Kubernetes (Kamaji datastores). What to do: existing clusters are adopted in place by migration 50 — no data move, no pod restart — provided the pre-upgrade snapshot target resolves (see pre-upgrade check 1). A pre-upgrade hook also deletes the pre-1.6 operator Deployment, whosespec.selectoris immutable and would otherwise fail the Helm upgrade (@androndo in [#2859], @myasnikovdaniil in [#3270]). -
The deprecated
backup.*values on theetcdtenant module are removed. What changed:backup.enabled,schedule,destinationPath,endpointURL,region,forcePathStyle,s3AccessKey,s3SecretKeyand the job-history limits no longer exist; a newversionvalue (default3.6.11) pins the etcd release. Who is affected: tenants that configured legacy per-release S3 backups on the etcd module. What to do: migrate to theBackupClassflow driven by theEtcdstrategy (strategy.backups.cozystack.io/v1alpha1) before upgrading (#2859). -
Deleting a managed application now permanently deletes its storage. What changed: ten packages gained post-delete cleanup that reclaims previously-orphaned PVCs and Secrets. Deletion is irreversible — this is the point of the change, but it is a real behaviour change from "delete leaks the volume, and you could recover from it". Who is affected: ClickHouse (keeper + data + log PVCs), Qdrant (data PVC), OpenBAO (data PVC), the tenant
monitoringmodule (VictoriaMetrics/VictoriaLogs storage PVCs), the tenantseaweedfsmodule (volume PVCs), the tenantetcdmodule (data-etcd-*PVCs), Harbor (jobservice/trivy PVCs — this force-deletes them, overridingpersistence.resourcePolicy=keep), MariaDB (operator-generated Secrets), Bucket and Gateway (ACME solver resources). What to do: back up before deleting; snapshot anything you care about. Migrations 48 and 51 backfill the release label onto pre-existing ClickHouse keeper PVCs and monitoring storage PVCs so already-deployed clusters are covered too. Harbor's CNPG database PVCs are not touched (@scooby87 in [#3072], [#3075], [#3073], [#3094], [#3095], [#3170], [#3077], [#3071], [#3076], [#3093]). -
Tenant resource quotas are now hierarchical. What changed: a tenant's declared quota is the budget for its whole sub-tree — a child that declares a quota carves a fixed slice out of the parent's remaining budget, a child without one shares the parent's pool instead of being unbounded, and a child quota exceeding the parent's remaining budget is rejected at admission. Who is affected: multi-level tenant trees where a sub-tenant previously declared more than its parent, or nothing at all. What to do: review sub-tenant quotas before upgrading; an overcommitted pool emits a
QuotaOvercommittedevent. Set--tenant-quota-buffer-percentoncozystack-controllerfor a temporary grace buffer so workloads already over a freshly-binding quota keep running during rollout (@kvaps in [#3086]). -
Worker node disks now default to the application-level (DRBD
replicated) StorageClass. What changed: a node group leavingstorageClassempty previously picked the management-cluster default StorageClass; it now falls back to the applicationstorageClass, because linstor-csi v1.11.2 rejects ReadWriteMany volumes on a non-DRBD class and worker VMs need RWX to live-migrate. Who is affected: tenant clusters relying on the empty-storageClass default. What to do: nothing, but the rendered worker template changes, so each tenant's worker MachineDeployment rolls once on the first reconcile after upgrade (@lexfrei in [#3129], [#2987]). -
authentication.oidc.keycloakInternalUrlnow defaults to the in-cluster Keycloak Service. What changed: the default ishttp://keycloak-http.cozy-keycloak.svc:8080/realms/cozyinstead of empty, so OIDC backend calls no longer leave the cluster. Who is affected: clusters that never set the value explicitly. What to do: nothing, but the dashboard and linstor-gui gatekeepers roll once. Set the value explicitly for a non-standard Keycloak placement, or to an empty string to restore external OIDC discovery (@lexfrei in [#3289]). -
Air-gapped tenant workers:
registries.mirrorspassthrough is gone. What changed: the Helm-rendered*-patch-containerdSecret that plumbed the platform-wideregistries.mirrorsconfig to tenant workers has no consumer in the Talos machineconfig and was removed. Who is affected: tenants relying onregistries.mirrorsfor in-guest container-image pulls. What to do: the Talos OS image and installer are overridable — settalos.imageFactoryURLandtalos.installerRepositoryto your mirror — but in-guest registry mirroring is a Phase 2 follow-up (#2931, [#3244]). -
SecurityGroupis served with a membership-group shape. The newsdn.cozystack.io/v1alpha1 SecurityGroupAPI landed inside this release withspec.targetRefreplaced by aspec.attachments[]membership model before release, so there is no upgrade path to worry about — but anyone trackingmainbetween 2026-06-30 and 2026-07-16 must rewrite their objects (@lexfrei in [#2922]). -
The
network.cozystack.ioAPI group (ExposureClass/ServiceExposure) was introduced and removed within this cycle. It never shipped in a release and is not part of v1.6.0; nativeServicetype: LoadBalancer+loadBalancerClasscovers the same ground, exposed aspublishing.loadBalancerClass. Only clusters trackingmainthat explicitly setpublishing.exposureClassneed to provision their own MetalLB/Cilium address pool and switch topublishing.loadBalancerClass; any orphanedcozystack-<class>pool can be deleted by hand (@lexfrei in [#3081], removed by @kvaps in [#3218]).
Manual actions required
-
BEFORE the upgrade — recover SeaweedFS tenants classified
SorMIXED. The chart refuses to render for them, which blocks the tenant's upgrade until an operator resolves it. Runhack/seaweedfs-naming-audit.shand followdocs/operations/seaweedfs-431-rename-recovery.md. Do not guess which generation holds the data — the runbook exists because a duplicate that briefly served writes is indistinguishable from one that never scheduled (#3339). -
BEFORE the upgrade — bump GitOps-managed tenant
KubernetesCRs offv1.30. Migration 46 only patches live objects; Git wins the next reconcile and the HelmRelease then fails the version guard (#2931). -
BEFORE the upgrade — delete colliding hand-made tenant StorageClasses (see pre-upgrade check 4) (#2872).
-
DURING the upgrade, only if migration 50 halts — if the cluster intentionally has no backup storage (backups disabled, external S3 without staged credentials, or SeaweedFS not yet ready) and you accept adopting live etcd without a safety snapshot, take the documented escape hatch:
sh
kubectl edit package.cozystack.io cozystack.cozystack-platform
yaml
spec:
components:
platform:
values:
migrations:
etcdAdoptSkipBackup: true
Flux re-renders the platform chart and re-runs the hook, adopting every legacy etcd without a snapshot. A botched adoption on a tenant control-plane etcd is unrecoverable without a snapshot. Set the flag back to false once the upgrade completes, or the next etcd migration will skip its snapshot too (#2859, [#3335]).
-
AFTER the upgrade — review worker MachineHealthCheck tolerance. Remediation is now on by default at
50%; setnodeHealthCheck.maxUnhealthy: "0%"(cluster-wide) or per node group if you want the old behaviour while the fleet settles on Talos (#2931, [#3053]). -
AFTER the upgrade — re-check metrics scrape targets on ports 10250. cert-manager's webhook, external-secrets' webhook and metrics-server's listener all moved off port 10250, which the kubelet also serves (#3359, [#3360], [#3361]).
-
No action, but expect one rollout each: tenant worker pools (Talos rollover and the storageClass fallback), the dashboard and linstor-gui gatekeepers (OIDC internal URL default), the VPA admission controller (cert-manager-issued TLS), and the linstor-scheduler admission Deployment (migration 52 deletes it so the new immutable selector can be created cleanly).
Deprecations
- Legacy per-app S3 backup fields. The
etcdmodule'sbackup.*block is now removed (above); the equivalent legacy fields on Postgres and ClickHouse remain deprecated in favour of the platform-managedBackupClassflow introduced in v1.5.0. - Classic per-host ACME certificates for platform services. Still the default, but
publishing.certificates.wildcard(DNS-01) andpublishing.certificates.wildcardSecretName(operator-provided) are now the recommended path at scale, and the operator-provided certificate propagates to per-tenant termination points automatically (#2988, [#2990]). auto-release.yamlcron patch releases are gone — stable releases are cut only by explicit rc→stable promotion (#3017).
Feature Highlights
Talos Linux Tenant Kubernetes Workers
Tenant Kubernetes worker nodes no longer boot Ubuntu and bootstrap through kubeadm. Phase 1 of the Kubernetes-app split design replaces that path with Talos Linux, driven by cluster-api-bootstrap-provider-talos (CABPT v0.6.12) registered as a second bootstrap provider alongside kubeadm, and a clastix/talos-csr-signer sidecar embedded in the Kamaji control-plane pod.
Talos PKI (an Ed25519 CA plus trustd TLS) is generated through cert-manager with stable, lookup-and-reuse random Talos secrets; the Kamaji control-plane provider is bumped with an upstream-bound patch that exposes KamajiControlPlane.spec.network.additionalServicePorts so trustd (50001/TCP) can be published on the apiserver Service. Workers boot the Talos openstack raw image via a CDI DataVolume streamed from the image factory, with the system disk exposed as virtio-blk using blockSize.custom: logical=512, physical=4096 so 4Ki-native backends such as LINSTOR/DRBD behave under QEMU's O_DIRECT writes while SeaBIOS still boots. The separate kubelet disk is gone — Talos lays out EPHEMERAL itself on the single system disk that diskSize now sizes.
Existing tenants roll over automatically: old machines are replaced by Talos workers without manual intervention, and platform migration 45 pins the outgoing KubeadmConfigTemplate objects with helm.sh/resource-policy: keep so Helm cannot prune them out from under the still-live kubeadm MachineSet mid-rollover.
Two behaviour changes are worth planning for: the tenant kubernetes HelmRelease now reports Ready as soon as Helm completes (DisableWait) and no longer blocks on worker or addon readiness — worker-rollout health is tracked by WorkloadMonitor — and worker MachineHealthCheck remediation is enabled by default at maxUnhealthy: "50%". See the Breaking Changes section above for the full action list (@myasnikovdaniil in [#2931]).
Follow-ups shipped in the same release make the rollover practical: per-node-group maxUnhealthy / nodeStartupTimeout overrides so a stateful group can sit at 0% while a stateless one tolerates 50% (#3053), overridable talos.imageFactoryURL and talos.installerRepository for air-gapped, mirrored or rate-limited environments (#3244), the TalosConfigTemplate moved out of a post-install hook into the main install phase so worker MachineDeployments scale reliably (#3145), and MachineDeployment.spec.replicas handed entirely to the cluster-autoscaler so platform bumps stop draining workers (#3233).
etcd-operator v1alpha2 with In-Place Adoption
Cozystack's etcd stack moves to the donated etcd-operator.cozystack.io/v1alpha2 API — a Membership-API lifecycle replacing the StatefulSet model — served by a Cozystack-authored chart at appVersion v0.5.2, with CRDs split into their own etcd-operator-crds package so they can be installed ahead of the controller.
The interesting part is the upgrade. Platform migration 50 runs as a pre-upgrade hook, while the legacy operator is still the running one, and uses the etcd-migrate tool baked into the migrations image to rewrite ownership, labels and CRs so the new operator takes over the live data plane on its first reconcile — no data move, no pod restart. Before touching anything it takes a mandatory snapshot of every adopted cluster to the platform-managed cozy-backups bucket, under a system key prefix with tenant-invisible credentials, and it fails loudly rather than adopt without one. It also re-issues each cluster's server and peer certificates with the new operator's native member wildcard SAN first, because in secretRef TLS mode the operator never mints certs — an adopted cluster whose certs lack the native domain would silently fail TLS the first time a member is replaced, while still reporting Available=True.
The in-cluster 1.5 → 1.6 path needed three more fixes to actually work, all folded into [#3270]: a chart-managed transitional etcd-headless Service so adopted members stay resolvable at their legacy per-pod DNS names until they roll onto the native subdomain; a pre-upgrade hook that deletes the pre-1.6 operator Deployment, whose spec.selector is immutable and would otherwise fail the whole HelmRelease with field is immutable; and a raised 256Mi memory floor so the operator does not OOM before the VPA admission webhook can scale it. Backup strategies also derive their S3 endpoint, scheme and TLS mode from the provisioned bucket Secret instead of requiring it hand-set.
The tenant etcd module's deprecated backup.* values are removed in the same change — use a BackupClass bound to the Etcd strategy instead (@androndo in [#2859], @myasnikovdaniil in [#3270], [#3335]).
OIDC Single Sign-On for Tenant Kubernetes and Grafana
Two symmetric Phase 1 features let a tenant opt an individual workload into the platform's Keycloak cozy realm through one flat selector, with no platform-level configuration.
Tenant kube-apiserver. Each Kubernetes CR gains spec.oidc.mode: System | CustomConfig | None (default None). System trusts the cozy realm via a per-cluster public client and audience binding; CustomConfig accepts a tenant-supplied structured AuthenticationConfiguration. spec.oidc.users[] drives one ClusterRoleBinding per user inside the tenant cluster (admin → cluster-admin, view → view). In System mode a <release>-oidc-kubeconfig Secret carrying a ready-to-use kubectl oidc-login exec block is surfaced through the dashboard, so a tenant user goes from "cluster exists" to "kubectl works with my SSO identity" without an operator in the loop (@IvanHunters in [#3044]). The underlying passthrough — controlPlane.apiServer.extraArgs, extraVolumes and extraVolumeMounts on KamajiControlPlane — is also exposed directly for anyone wanting to hand-roll authentication (@lexfrei in [#3123]).
Grafana. Each Monitoring CR gains the same spec.oidc.mode selector. System wires a per-instance confidential Keycloak client with audience binding; authorization is app-side, with spec.oidc.users: [{email, role: Admin|Editor|Viewer}] reconciled into Grafana's Main Org. by a chart-owned post-install/post-upgrade Job that pre-provisions users, adds them to the org, PATCHes their role and prunes stale members. CustomConfig accepts a tenant-supplied [auth.generic_oauth] payload, either inline as a map (the chart merges the two contract-critical settings on top) or as a Secret with an auth.ini key — in which case spec.oidc.users is unsupported and the chart fails the render on that combination rather than silently ignoring it. The admin_user/admin_password Secret remains a documented break-glass path in every mode (@IvanHunters in [#3176]).
Both are documented in-repo at docs/oidc-tenant.md and docs/oidc-grafana.md.
SecurityGroup: Tenant-Managed Network Policy
A new tenant-facing, namespace-scoped firewall resource — sdn.cozystack.io/v1alpha1 SecurityGroup — lets tenants manage their applications' network policy without any access to the cilium.io API group.
A SecurityGroup is a membership group: it owns a membership label (securitygroup.sdn.cozystack.io/<name>) that a new securitygroup-controller stamps onto the pods of every managed application listed in spec.attachments, and removes when an attachment is dropped. The group projects 1:1 onto a CiliumNetworkPolicy in the same namespace whose endpointSelector is exactly that membership label — so one SecurityGroup can cover several applications at once, and fromSG/toSG peers resolve group-to-group references live in the Cilium dataplane. A finalizer guards the backing policy so member labels are always stripped before it is removed, re-asserted by the aggregated-API REST storage on every write so a full-replace PUT cannot orphan them.
Ingress and egress rules only ever add allowed traffic; an empty list does not isolate the member pods, because effective connectivity remains the union of every policy selecting a pod including the platform's blanket-allow baseline. Default-deny enforcement is tracked separately as future work (@lexfrei in [#2922]).
Hierarchical Tenant Resource Quotas
tenant.spec.resourceQuotas previously became a plain per-namespace ResourceQuota that ignored the tenant tree entirely — a super-admin inside a quota'd tenant could create a sub-tenant with a larger quota, or no quota at all, and consume more than was allocated. A tenant's declared quota is now the budget for its whole sub-tree.
Enforcement is two cooperating layers, mirroring OpenShift's ClusterResourceQuota split. A declaration-time gate in the aggregated apiserver deterministically rejects a sub-tenant whose declared quota exceeds the parent's remaining budget. A runtime enforcer in cozystack-controller maintains a per-namespace tenant-quota-allocated ResourceQuota that clamps each pool member to its share of the parent budget, aggregating sub-tree usage so members collectively stay within it — Kubernetes enforces the most restrictive ResourceQuota in a namespace, so this binds without fighting Flux over the chart-rendered tenant-quota.
A parent quota lowered below the sum of its children emits a QuotaOvercommitted event, and --tenant-quota-buffer-percent on cozystack-controller temporarily inflates pool budgets during rollout so workloads already over a freshly-introduced quota keep running. No API types and no tenant chart changes — the feature works with the existing field (@kvaps in [#3086]).
Keycloak: Encryption Proxy, Admin Hostname, and Backups
Keycloak gets four independent additions this release, all opt-in.
An optional KMS-encrypting database proxy provides column-level encryption for PII at rest, backed by either a static KEK or Vault Transit, and off by default (@kvaps in [#3003]). A follow-up adds Vault Kubernetes and AppRole auth for the proxy, a dedicated ServiceAccount, and a shared DEK set covering both migration and HA (@sircthulhu in [#3239]).
An opt-in ingress.adminHost serves the admin console and Administration REST API on a separate hostname and route, with publishing.ingressNameAdmin (_cluster.expose-ingress-admin) attaching that route to a separate — for example private — Gateway or ingressClass. Disabled by default; existing deployments are unchanged (@sircthulhu in [#2954]).
S3 backups for the Keycloak CNPG database land via Barman, configurable under backup.* and disabled by default (@kvaps in [#3174]). Finally, the realm login theme is now selectable and its theme image shipped through platform branding values (branding.loginTheme, branding.loginThemeImage) (@lexfrei in [#3124]), and the database host, port and URL properties are overridable so the connection can be routed through an external proxy (@sircthulhu in [#2907]).
Wildcard Certificates, End to End
v1.5.0 let operators point platform services at a pre-existing wildcard TLS Secret through publishing.certificates.wildcardSecretName, but only for the root tenant. That certificate is now replicated by the platform controller into every tenant namespace that terminates TLS, so per-tenant ingress controllers and Gateways serve it automatically — no extra operator input, and no cross-namespace Secret read (@lexfrei in [#2990]).
For operators who would rather have the platform mint the wildcard, a new opt-in publishing.certificates.wildcard (default false) issues a single *.<root-host> certificate via a DNS-01 solver on the default ingress-nginx path, instead of a per-host ACME certificate — the practical fix for hitting Let's Encrypt rate limits at scale (@lexfrei in [#2988]).
Immutable Tags and rc→stable Promotion
A stable release is now a renamed release candidate: the bytes shipped as vX.Y.Z are bit-for-bit the bytes built and end-to-end tested as vX.Y.Z-rc.N. No tag is ever force-moved and stable is never rebuilt — it is promoted by retagging the rc's existing images by digest.
The enabler was decoupling the operator's self-reported version from its image: it now reads COZYSTACK_VERSION from the environment (threaded through cozystackOperator.platformVersion), falling back to the build-time value, so the same bits can report any release name. All five force-retag sites in the release workflows are removed and replaced with write-once or fast-forward-only semantics, and the cron-driven auto-release.yaml patch-tag workflow is deleted outright — stable tags now exist only through explicit promotion. A promote-rc.yaml workflow validates the rc, skopeo copys every digest-pinned image reference to the stable tag, restamps the version-stamped assets, and opens a release-X.Y.Z PR whose merge cuts the write-once stable tag (@myasnikovdaniil in [#3017]). A companion Cut Pre-release Tag workflow covers the rc side of the same flow (#3253).
Application Deletion Reclaims Its Storage
Deleting a managed application used to leave its volumes and operator-generated Secrets behind. A ten-PR sweep across the catalog closes that, adding post-delete cleanup hooks and PVC retention policies so a deleted application actually releases what it held: ClickHouse keeper and data/log PVCs, Qdrant's data PVC, OpenBAO's data PVC, the tenant monitoring module's VictoriaMetrics/VictoriaLogs storage and TLS Secrets, the tenant seaweedfs module's volume PVCs and cert/db Secrets, the tenant etcd module's data-etcd-* PVCs and five TLS Secrets, Harbor's jobservice and trivy PVCs plus ACME solver resources, MariaDB's operator-generated password Secrets, Bucket's ACME solver resources, Gateway's ACME account Secret, and the Postgres init Job.
Two migrations backfill the release label onto storage provisioned before this change — 48 for ClickHouse keeper PVCs, 51 for monitoring storage PVCs — because the respective operators only stamp volumeClaimTemplates labels onto PVCs they create after the change and never re-label existing ones. Both are best-effort by design: the worst case if a relabel is skipped is the pre-existing leak, which is no worse than the status quo.
This makes deletion destructive. See the Breaking Changes section (@scooby87 in [#3069], [#3071], [#3072], [#3073], [#3075], [#3076], [#3077], [#3078], [#3093], [#3094], [#3095], [#3170]).
Platform Components
- Talos Linux: v1.13.0 → v1.13.6 (management-cluster image and matchbox assets), with tenant workers on the same v1.13.6. Closes the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes. (release notes) (@lexfrei in [#3008], [#3240], [#3269])
- Kernel 6.18.29 → 6.18.38, carrying the KVM shadow-paging use-after-free and nested-guest escape fixes. Nested virtualization remains available.
- containerd 2.2.5, runc 1.4.3, etcd 3.6.11, Kubernetes 1.36.1 in the bundled components.
- Reliability fixes across the range: DNS server and listeners recreated on host-DNS runner restart, correct
intmarshalling of kube-scheduler config, relaxed LUKS header and hostname validation, more resources marked sensitive, higher etcd open-file limit,FailurePauseTimeouthonoured before reboot. -
System extensions refreshed to the 20260622 set, including DRBD 9.3.2 and ZFS 2.4.3. Applying this rolls each tenant worker pool once, as the worker template is renamed to carry the new image.
-
etcd-operator: v0.4.5 (vendored upstream chart) → v0.5.2 (Cozystack-authored chart,
etcd-operator.cozystack.io/v1alpha2). CRDs split into a newetcd-operator-crdspackage. See Feature Highlights and Breaking Changes for the adoption path. (sources) (@androndo in [#2859], @myasnikovdaniil in [#3270]) -
Cilium: 1.19.3 → 1.19.5. (release notes) (@lexfrei in [#2972])
- Fixes
CiliumNetworkPolicyfromNodes/toNodeswhenpolicy-default-local-clusteris on (the 1.19+ default), and a wildcard-namespace bypass for selectorlessipBlockrules. - Gateway API: TLS-passthrough routes no longer fail silently on gateways with mixed HTTP/HTTPS/TLS listeners, weighted backend splitting is fixed for
TLSRoutepassthrough, stale route parent statuses are pruned, and a controller panic on a malformedparametersRefis prevented. - Datapath: fixes node connectivity disruption when ClusterIP/LoadBalancer VIPs overlap node-local addresses, host-proxy packet routing to pods, a corrupted
NamespaceSelectoronCiliumEgressGatewayPolicy, and an endpoint-ID reuse race that removed a live endpoint's BPF state. -
Removed: the
loadBalancer.standaloneHelm option. -
KubeVirt: updated to v1.8.4. Fixes VMIs stuck in
Scheduledon Kubernetes 1.36 (strict CRD numeric-format validation of VMI checksum status fields). (release notes) (@lexfrei in [#2940]) - Also fixes a gRPC connection leak in
virt-handler'sGetLauncherClientthat caused unbounded memory growth, socket accumulation and goroutine leaks, and closes CVE-2026-35469 via amoby/spdystreambump. -
Adds missing metrics, recording rules and alerts for the virt components.
-
Velero: 1.17.0 → 1.18.1 (chart 11.0.0 → 12.0.3), with velero plugins moved to velero-1.18-compatible releases and
kubevirt-velero-pluginv0.8.0 → v0.9.0. (release notes) (@lexfrei in [#2974]) - Concurrent backup processing — multiple backups now run simultaneously without interfering, a direct win on multi-tenant clusters.
- Cache volumes for data movers on restore (CSI snapshot data movement and fs-backup), so data-mover pods no longer fail on limited node ephemeral disk, and restore throughput improves.
- Incremental backup size is now observable for data-mover backups.
-
v1.18.1 fixes list-scoping bugs that made restores, PodVolumeRestores and CSI PVC backups list across all namespaces instead of the install namespace, and migrates
VolumeGroupSnapshottov1beta2for Kubernetes 1.34+. -
Vertical Pod Autoscaler: 1.3.0 → 1.5.0 (chart 10.0.0 → 11.1.1). (release notes) (@lexfrei in [#2975])
- In-place pod resizing is promoted to Beta, with fallback to eviction when an in-place update fails.
- Deprecation:
UpdateMode: Autois deprecated upstream, as is the humanized-memory flag. - New metrics: failed eviction counters,
update_modelabels on updater metrics, VPA name/namespace on resource-update counters; Prometheus bearer-auth support. -
Dependencies bumped for Kubernetes 1.34; several race conditions in the recommender and updater fixed.
-
Harbor: 2.14.2 → 2.15.1 (chart 1.18.2 → 1.19.1), plus PodDisruptionBudgets for core, portal, jobservice, nginx, registry, trivy and exporter. (release notes) (@scooby87 in [#2966])
- New: tag-deletion option for garbage collection, per-project upstream-connection limits (
max_upstream_conn) with UI, and per-endpoint CA certificate support for registry endpoints. - Audit logging to the database can now be disabled at initialization.
-
2.15.1 fixes proxy-cache serve-local on remote-not-found, prevents background polling from renewing session TTL, redacts
redis_url_regfrom GC attributes, and refreshes photon base packages for CVEs. -
Keycloak: 26.5.2 → 26.6.3. (release notes) (@lexfrei in [#2964])
- JWT Authorization Grant for external-to-internal token exchange using externally signed JWT assertions.
- Federated client authentication, removing the need to manage individual client secrets in Keycloak.
- Workflows for automating realm administrative tasks such as user and client lifecycle management.
-
Zero-downtime patch releases — rolling updates within a minor stream without service downtime.
-
LINSTOR: linstor-server 1.33.2 → 1.33.3, linstor-csi v1.10.6 → v1.11.2. (sources) (@lexfrei in [#2987])
-
Upgrade impact: linstor-csi v1.11.2 rejects ReadWriteMany volumes on a non-DRBD storage class, which is why an empty tenant worker
nodeGroups[*].storageClassnow falls back to the application-levelreplicatedclass. See Breaking Changes. -
linstor-scheduler: v0.3.2 → v0.3.6 (chart 0.2.3 → 0.3.1), re-vendored to upstream with the native admission webhook. (sources) (@lexfrei in [#3201])
- The admission webhook no longer strips unknown Pod fields — the
imagevolume source and native-sidecarrestartPolicy— which had broken KubeVirt containerDisk VMs and native sidecars. - The webhook now reloads its TLS certificate after rotation instead of silently failing.
-
Migration 52 deletes the old
linstor-scheduler-admissionDeployment so the new, differently-selected one can be created (a Deployment'sspec.selectoris immutable);failurePolicy: Ignoremeans the brief gap does not block Pod creation. -
FoundationDB operator: v2.13.0 → v2.30.0 (chart 0.2.0 → 0.3.0). (release notes) (@lexfrei in [#2992])
-
Seventeen upstream releases of operator fixes and reconciliation improvements; the v2.30.0 release itself is a Go 1.26 toolchain and modernization pass.
-
HAMi: 2.8.1 → 2.9.0. (release notes) (@lexfrei in [#2993])
- HAMi-DRA (NVIDIA) is ready for use; HAMi-core performance optimized and a HAMi-core mode added for Ascend devices.
- Volcano vGPU device plugin synced to 0.19 with CDI support; new vGPU-monitor
--metrics-bind-addressflag and PrometheusServiceMonitorsupport in the chart. -
Resource-quota checking in the webhook, plus configurable webhook
namespaceSelector/objectSelector. -
Percona Server for MongoDB operator: 1.21.1 → 1.22.0, with the managed CR moved to
crVersion: 1.22.0. (release notes) (@lexfrei in [#2968]) - Sets the Kubernetes
appProtocol: mongoon all services, so service meshes (Istio and friends) recognize MongoDB traffic and mTLS/cluster formation work without workarounds. -
Physical and logical restore into a cluster with different replica-set names, via
replsetRemappingonPerconaServerMongoDBRestore. -
OpenBao: v2.5.0 → v2.5.1 (chart 0.25.3 → 0.25.7). (release notes) (@lexfrei in [#2991])
- Security: closes CVE-2025-68121 (Go 1.25.7) and CVE-2026-24051 (OpenTelemetry SDK 1.40.0).
- Fixes Auto Unseal failing when upgrading to v2.5.0 across AliCloud KMS, AWS KMS, Azure Key Vault, GCP Cloud KMS and OCI KMS.
- PKI no longer returns 500 until reload on a cancelled cache-invalidation context; irrevocable-lease revocation now decrements the lease count.
-
Note: pre-v2.5.0 corrupt namespace identity groups are removed during unseal and must be recreated by an admin — look for
deleting corrupt groupin startup logs. -
CoreDNS: chart 1.43.2 → 1.46.0 (upstream appVersion 1.12.3 → 1.13.1; the shipped image is deliberately held at v1.12.4 for kube-dns alignment). (release notes) (@lexfrei in [#3004])
-
Upstream fixes a Corefile infinite loop on unclosed braces and an import-cycle issue, normalizes panics on invalid origins, and prevents reload deadlocks.
-
csi-driver-nfs: 4.11.0 → 4.13.3. (release notes) (@lexfrei in [#2967])
-
Security: closes CVE-2025-52881 and CVE-2025-58181; CSI sidecar images upgraded.
-
OpenCost: 1.111.0 → 1.120.3 (chart 1.41.0 → 2.5.23). (sources) (@lexfrei in [#3000])
-
NATS: 2.11.8 → 2.11.10 (chart 1.3.13 → 1.3.16). (sources) (@lexfrei in [#2976])
-
ouroboros: 0.8.0 → 0.8.1. The controller rollout strategy is now rendered identically in every
controller.mode, so switching a live release into external-dns mode no longer fails server-side apply withspec.strategy.rollingUpdate: Forbidden(sources) (@lexfrei in [#3323]). -
Managed Kubernetes patch versions refreshed to v1.35.6, v1.34.9, v1.33.13 and v1.32.13; v1.30 removed from the support matrix (@lexfrei in [#3014]).
-
Grafana 11.6.15, victorialogs-datasource v0.14.1 → v0.28.0, and alerta-web 9.1.0 in the monitoring stack (@lexfrei in [#3011]).
-
Kamaji rebuilt on 26.3.6-edge with the datastore unused-deletion deadlock fix and Go stdlib/module CVEs closed (#3033, [#3146]).
-
New system packages:
etcd-operator-crds(v1alpha2 CRDs, installed ahead of the controller) andsecuritygroup-controller(backs the newSecurityGroupAPI).
Major Features and Improvements
-
[kubernetes] Bootstrap tenant workers with Talos via CABPT: See Feature Highlights — Ubuntu + kubeadm worker bootstrap is replaced by Talos Linux driven by
cluster-api-bootstrap-provider-talosand atalos-csr-signersidecar in the Kamaji control-plane pod; existing clusters roll over automatically (@myasnikovdaniil in [#2931]). -
[platform] Migrate to etcd-operator v1alpha2: See Feature Highlights — new
etcd-operator.cozystack.io/v1alpha2Membership API, Cozystack-authored operator chart at v0.5.2, snapshot/BackupClass-driven backup flow, and in-place adoption of legacy clusters via migration 50 (@androndo in [#2859]). -
[etcd] Complete the v1alpha2 transition for in-cluster 1.5→1.6 upgrades: Keeps the legacy
etcd-headlessService alive so adopted members stay resolvable, deletes the pre-1.6 operator Deployment via a pre-upgrade hook to get past the immutable selector, raises the operator's memory floor so it does not OOM before the VPA scales it, and makes the adoption migration robust in-cluster (@myasnikovdaniil in [#3270]). -
[api] Add the SecurityGroup network policy resource (
sdn.cozystack.io): See Feature Highlights — a tenant-facing, namespace-scoped firewall that attaches to managed applications by reference and projects 1:1 onto aCiliumNetworkPolicy, letting tenants manage network policy withoutcilium.ioaccess (@lexfrei in [#2922]). -
[tenant] Recursive (hierarchical) resource quotas for sub-tenants: See Feature Highlights — a sub-tenant's quota is carved out of its parent's remaining budget and can no longer exceed it; sub-tenants without a quota share the parent's pool (@kvaps in [#3086]).
-
[apps/kubernetes] Per-cluster OIDC selector for the tenant kube-apiserver (Phase 1): See Feature Highlights —
spec.oidc.mode: System | CustomConfig | Noneplusspec.oidc.users[], with a ready-to-useoidc-kubeconfigSecret surfaced through the dashboard (@IvanHunters in [#3044]). -
[monitoring] Per-instance OIDC selector for the Grafana instance (Phase 1): See Feature Highlights — the same flat
spec.oidc.modeselector for Grafana, with app-side role reconciliation into Grafana's Main Org (@IvanHunters in [#3176]). -
[kubernetes] Expose tenant apiserver authentication in app values: Adds
controlPlane.apiServer.extraArgs,extraVolumesandextraVolumeMountspassthrough toKamajiControlPlane, so operators can enable OIDC or a structuredAuthenticationConfigurationon tenant clusters by hand. Volume sources are restricted to ConfigMap and Secret, since the control-plane pod runs on the management cluster (@lexfrei in [#3123]). -
[kubernetes] Propagate remote-accessible LINSTOR StorageClasses to tenant clusters: Infra-cluster LINSTOR StorageClasses whose
allowRemoteVolumeAccessis not"false"are auto-created inside each tenant under the same name, with the class named bystorageClass(defaultreplicated) becoming the tenant default and the legacykubevirtclass kept as an alias so existing PVCs are unaffected. Node-local classes are not propagated. Delete any hand-made colliding tenant class before upgrading (@myasnikovdaniil in [#2872]). -
[platform, ingress] Propagate the operator wildcard certificate to per-tenant termination points: See Feature Highlights —
publishing.certificates.wildcardSecretNameis now replicated into every tenant namespace that terminates TLS, with no extra operator input and no cross-namespace Secret read (@lexfrei in [#2990]). -
[platform] ACME wildcard certificate on the default ingress-nginx path: Opt-in
publishing.certificates.wildcard(defaultfalse) issues one*.<root-host>certificate via a DNS-01 solver for system services instead of a per-host ACME certificate, avoiding Let's Encrypt rate limits at scale (@lexfrei in [#2988]). -
[keycloak] Optional KMS-encrypting database proxy: Column-level PII encryption at rest with a static KEK or Vault Transit; off by default (@kvaps in [#3003]). Extended with Vault Kubernetes and AppRole auth, a dedicated ServiceAccount, and a shared DEK set covering migration and HA (@sircthulhu in [#3239]).
-
[keycloak] Opt-in separate hostname and route for the admin console:
ingress.adminHostserves the admin console and Administration REST API on their own hostname, andpublishing.ingressNameAdminattaches that route to a separate (e.g. private) Gateway or ingressClass. Disabled by default (@sircthulhu in [#2954]). -
[keycloak] S3 backup for the CNPG database: Optional Barman-based S3 backups of the Keycloak database under
backup.*, disabled by default (@kvaps in [#3174]). -
[keycloak] Configurable login theme via platform branding: Select the realm login theme and ship its theme image through
branding.loginTheme/branding.loginThemeImage(@lexfrei in [#3124]). -
[keycloak] Allow overriding the database host/port/properties:
db.host,db.portanddb.urlPropertiesplusextraVolumes/extraVolumeMounts, so the database connection can be routed through an external proxy (@sircthulhu in [#2907]). -
[keycloak] Enable the UPDATE_EMAIL required action in the cozy realm: Users can now change their own email address from the account portal (@kvaps in [#3175]).
-
[release] Immutable tags and rc→stable promotion: See Feature Highlights — a stable release is the byte-identical promotion of its release candidate, with every force-retag site removed and the cron patch-release workflow deleted (@myasnikovdaniil in [#3017]). A companion Cut Pre-release Tag workflow covers the rc side (#3253).
-
[operator] Support private OCI registry credentials for the platform source: New
--platform-source-secretflag (andcozystackOperator.platformSourceSecretvalue) authenticates the platform package source against private OCI registries (kubernetes.io/dockerconfigjson) and private Git repositories (basic auth, bearer token, or SSH key). An empty value preserves the existing public-registry behaviour (@IvanHunters in [#2472]). -
[apps/kubernetes] Overridable Talos image factory URL and installer repository:
talos.imageFactoryURLandtalos.installerRepository(both defaulting to the public factory) let air-gapped, mirrored or rate-limited environments serve the worker OS image and installer from their own infrastructure (@myasnikovdaniil in [#3244]). -
[kubernetes] Per-node-group nodeHealthCheck overrides: Optional
nodeGroups[name].maxUnhealthyandnodeGroups[name].nodeStartupTimeout, falling back to the cluster-widenodeHealthCheck.*when unset — so a stateful group can sit at0%while a stateless one tolerates50%. An invalid override fails the render naming the offending group (@myasnikovdaniil in [#3053]). -
[kamaji] Default to 2 replicas and drop the telemetry webhook: The Kamaji controller now runs two replicas with soft pod anti-affinity, and the telemetry handler and webhook entry are removed — cutting apiserver admission p99 for
TenantControlPlanemutating/validating webhooks by roughly 70% on multi-tenant clusters (@mattia-eleuteri in [#2671]). -
[kubevirt-cdi] Make cloneStrategyOverride configurable: CDI's clone strategy is now settable from values (default
csi-clone), so operators on storage backends without efficient CSI cloning can select a different strategy (@mattia-eleuteri in [#2926]). -
[cozy-lib] CA-only TLS trust-anchor helper for tenants: A new
cozy-lib.tls.caCertSecrethelper renders aca.crt-only Opaque Secret surfaced through the tenant-secret API, so a tenant can verify a per-app TLS endpoint without being granted read access to any object that also carries a private key — the foundation the per-app TLS series builds on (@lexfrei in [#2989]). -
[dashboard] Vendor the cozystack-ui console into the monorepo: The console SPA is now built from in-tree source under
packages/system/dashboard/images/consolerather than a separate repository. The shipped image and digest are unchanged for cluster operators (@myasnikovdaniil in [#2963]).
Improvements (minor)
-
[api] Return typed apierrors from TenantNamespace registry helpers: The TenantNamespace API now returns
401 Unauthorizedfor requests without a user identity and a structured500 InternalErrorfor RBAC lookup failures, instead of a generic 500 (@lexfrei in [#2870]). -
[platform] Factor the cozystack-version stamp into a shared migration helper: Every migration now stamps
cozystack-versionthrough one labeled helper, so theplatform.cozystack.io/no-deletelabel can no longer be silently dropped by a label-less apply — which would have removed the ConfigMap from the deletion-protection policy (@lexfrei in [#2980], @myasnikovdaniil in [#3118]). -
[dashboard] Show the LoadBalancer external IP on the app Services tab: Externally published services now display their assigned IP in the console instead of leaving the operator to look it up with kubectl (@scooby87 in [#3172]).
-
[dashboard] Render error and unknown-type states instead of an infinite spinner: The console now shows a not-found message on a failed application GET and an unknown-type message for an unrecognized resource (@myasnikovdaniil in [#3098]).
-
[objectstorage-controller] Converge BucketClaim readiness and speed up COSI failover:
BucketClaimreadiness now converges to the backend Bucket, and single-replica COSI provisioner failover is faster (@lexfrei in [#3034]). -
[etcd] Pin tenant etcd to v3.5.31: Restores consistent watches by letting kube-apiserver enable
RequestWatchProgresson Kubernetes 1.35 tenants (@lexfrei in [#3122]).
Bug fixes
-
[seaweedfs] Close the 4.31 rename fallout on the 1.5.x→1.6 upgrade path: The 1.6 upgrade no longer renames a SeaweedFS instance away from its data. The naming guard now runs in the chart a platform upgrade actually re-renders and refuses when both pre- and post-4.31 naming generations exist;
hack/seaweedfs-naming-audit.shanddocs/operations/seaweedfs-431-rename-recovery.mdguide recovery, and the refusal is expected for tenants that passed through 1.5.x. Cluster-scoped COSI RBAC is named per namespace again (the 4.31 release-based names collided across tenants), and theseaweedfs-dbhand-over now runs for every instance name — previously an instance not namedseaweedfshad its filer metadata database pruned on upgrade. New migration 53 repairs clusters that already ran the old hand-over (@myasnikovdaniil in [#3339]). -
[seaweedfs] Adopt legacy workloads in place across the 4.31 rename: Pins
fullnameOverride: seaweedfsso upgrading past the vendored 4.31 chart bump adopts the existing workloads and volumes instead of standing up a second, empty cluster beside them (@myasnikovdaniil in [#3282]). -
[seaweedfs] Point S3 consumers at the reachable S3 service: The S3 ingress backend, the iceberg ingress backend and the in-cluster COSI provisioner endpoint all resolved a stale name or the wrong port, so the external S3 endpoint returned 503 through ingress-nginx and in-cluster COSI paths hit port 443 or the filer service. All consumers now resolve
seaweedfs-s3on port 8333 (shipped in v1.5.2) (@lexfrei in [#3119]). -
[seaweedfs] Configure the postgres2 connection pool for the filer: Removes roughly 2s of per-request S3 latency caused by PostgreSQL connection churn (shipped in v1.5.3) (@mattia-eleuteri in [#2906]).
-
[seaweedfs] Raise master probe timeouts above the cluster-status backoff: A master with a transiently empty raft view is no longer killed into a permanent CrashLoop, which could time out the whole SeaweedFS install (@lexfrei in [#3068]).
-
[seaweedfs] Soften master anti-affinity for small clusters: The hard master anti-affinity becomes a preference, so the S3 stack installs on clusters with fewer than three nodes (@lexfrei in [#3197]).
-
[kubernetes] Make the default md0 node group removable: The built-in
md0node group is applied only when no node groups are configured, so it can finally be removed or replaced; migration 47 pinsmd0explicitly on existing clusters to preserve their topology (@myasnikovdaniil in [#2936]). -
[apps/kubernetes] Honour nodeGroup minReplicas in MachineDeployment.spec.replicas: The chart stops managing
spec.replicas— the cluster-autoscaler owns it, seeded by CAPI's defaulting webhook from the min-size annotation and preserved through SSA field-manager ownership.helm upgradeno longer drains workers back to a hardcodedreplicas: 2on every platform bump (@IvanHunters in [#3233]). -
[kubernetes] Run the Talos reconcile Job in the main install phase: Creating the worker
TalosConfigTemplatefrom a main-phase Job instead of a post-install hook makes worker MachineDeployments scale reliably during install (@lexfrei in [#3145]). -
[kubernetes] Default worker disk storageClass to the application storageClass: A node group leaving
storageClassempty now falls back to the application-level (DRBDreplicated) class rather than the management-cluster default, so live migration of worker VMs gets the ReadWriteMany volume it requires (@lexfrei in [#3129]). -
[kubernetes] Make node group resources optional when instanceType is set: A node group sized purely by
instanceTypeno longer fails schema validation with "missing property 'resources'"; the unsupported combination ofinstanceTypeplus explicitresourcesis rejected at render time instead (shipped in v1.5.2) (@lexfrei in [#3121]). -
[kubernetes] Clear the Kamaji datastore-secret finalizer to unblock namespace deletion: The leftover
finalizer.kamaji.clastix.io/datastore-secreton the datastore-config Secret is now cleared during tenant teardown, so deleting a Kubernetes app or tenant no longer leaves the namespace stuckTerminating(@scooby87 in [#3078]). -
[kubernetes] Gate the vmop cert-manager-crds dependency on certManager.enabled: Clusters running
monitoringAgents.enabled=truewithcertManager.enabled=falsehad victoria-metrics-operator permanently blocked on a HelmRelease that would never exist, cascading tomonitoring-agentsandvertical-pod-autoscalernever becoming ready (shipped in v1.5.2) (@myasnikovdaniil in [#3130]). -
[apps/kubernetes] Bound the pre-delete hook's tenant HelmRelease wait: Tenant Kubernetes teardown no longer hangs when the cluster has no working nodes — the pre-delete hook bounds its wait for the in-tenant HelmReleases and force-clears their Flux finalizers on timeout (@myasnikovdaniil in [#2826]).
-
[kamaji] Backport the datastore unused-deletion deadlock fix: Deleting a Tenant whose etcd DataStore had no more referencing TenantControlPlanes left the DataStore stuck
Terminating, hanging the etcd Helm uninstall and wedging the tenant namespace. Backports clastix/kamaji#1122 as a source patch on the pinned build (shipped in v1.5.2) (@myasnikovdaniil in [#3146]). -
[tenant] Inherit the full ancestor label chain from the parent namespace: Tenant namespaces now carry the complete
tenant.cozystack.io/<ancestor>chain (includingtenant-root) at any nesting depth, restoring root-ingress reachability to nested tenants; migration 49 backfills the labels on existing clusters (@IvanHunters in [#2912]). -
[tenant] Grant the watch verb to the cleanup Role: Tenant deletion no longer hangs on the pre-delete hook, which needed
watchto wait; the hook also moves to a maintained kubectl image (@lexfrei in [#2923]). -
[tenant] Gate VM-resource HelmReleases on victoria-metrics-operator readiness: The tenant
etcd,ingressandmonitoringreleases create VictoriaMetrics-operator CRs behind afailurePolicy: Failwebhook; on cold installs they could reconcile before the webhook was serving and fail with "connection refused". They nowdependsOnthe operator (shipped in v1.5.2) (@lexfrei in [#3141]). -
[monitoring] Surface missing VictoriaLogs and gate the dead-end vlinsert ExternalName: A missing platform VictoriaLogs is now reported as not-ready instead of being masked, and the dangling
vlinsert-genericExternalName no longer silently drops logs. Adds amonitoring.rootEnabledplatform value that operators must set to match their actualTenant/rootspec.monitoring(@scooby87 in [#3215]). -
[mariadb] Re-guard replication so single-replica passes the operator webhook: Replication is enabled only for
replicas > 1, matching the mariadb-operator validating webhook, so single-replica instances can be created again; the dashboard, RBAC and backups now target the bare service for single-replica setups (shipped in v1.5.2) (@scooby87 in [#3180]). -
[mariadb] Derive my.cnf sizing from the instance's own resources:
my.cnfis now sized from the instance's own CPU, memory and volume instead of hardcoded values assuming a large dedicated host — small presets previously advertisedmax_connections=4096they could not serve; the limit now scales with memory from the server default of 151 (@lexfrei in [#3345]). -
[mariadb] Widen the startup probe budget so bootstrap cannot be killed: A slow first boot can no longer leave the datadir permanently unusable (@lexfrei in [#3344]).
-
[mariadb] Bind the metrics exporter to IPv4: The exporter's readiness probe is reachable again and the metrics endpoint populates reliably (@lexfrei in [#3064]).
-
[kafka] Set entity-operator resources to avoid OOM under the tenant LimitRange: The topic- and user-operator containers carried no requests/limits, so the 128Mi tenant
LimitRangedefault OOMKilled them on startup — leavingKafkaTopic/KafkaUserobjects unreconciled and topics stuck on the topic-operator finalizer during teardown (shipped in v1.5.2) (@myasnikovdaniil in [#2934]). -
[harbor] Preserve the jobservice and trivy storageClass on upgrade: The wrapper never forwarded
storageClassto the jobservicejobLogPVC, so it picked up the cluster default via admission — and because PVCstorageClassNameis immutable, that produced rendered-vs-live drift on every upgrade. Both now preserve an existing PVC's class via lookup (shipped in v1.5.2) (@myasnikovdaniil in [#2930]). -
[postgres] Purge a stale recovery cluster on repeat in-place restore: A repeat in-place restore no longer silently succeeds without restoring — a recovery cluster left over from an earlier completed restore is purged so the restore re-bootstraps from the backup (@IvanHunters in [#3318]).
-
[keycloak] Point HTTPRoute backends at the service port: Fixes Gateway API routing to Keycloak, which targeted the wrong port (@lllamnyp in [#3235]).
-
[kubevirt-instancetypes] Restore persistent EFI/TPM state: v1.5.1 stripped persistent EFI/TPM from the
windows.11,windows.2k22andwindows.2k25preferences because the RWO backend-storage PVC pinned VMs to a node and blocked live migration. RWO-Filesystem backend storage already live-migrates on KubeVirt v1.4+, so persistence is restored (dropped in v1.5.1 via [#3006], restored in v1.5.3) (@kvaps in [#3154]). -
[gateway] Pin identical allowedRoutes.kinds on port 443 listeners: All port-443 listeners now carry identical
allowedRoutes.kinds=[HTTPRoute,TLSRoute], preventing Cilium (cilium#45559) from collapsing the HTTPS-terminate and TLS-passthrough listeners and silently dropping HTTPRoutes (@shreyaabaranwal in [#3084]). -
[platform] Route OIDC backend calls through the in-cluster Keycloak by default:
authentication.oidc.keycloakInternalUrlnow defaults to the in-cluster Keycloak Service, so enabling OIDC on a fresh install no longer leaves the dashboard down while the root ingress is not yet serving (@lexfrei in [#3289]). -
[platform] Order backupstrategy-controller after cozystack-basics: On a fresh install the
cozy-backupsBucket could racetenant-rootnamespace creation and fail, which then raced the install-wait deadline and failed unrelated fresh installs (shipped in v1.5.2) (@lexfrei in [#3037]). -
[platform] Make
*-rdsources depend on cozystack-engine: Fixes an install-ordering race where resource-definition sources reconciled before the engine that serves them (@lexfrei in [#3041]). -
[cozystack-basics] Validate legacy Ingress hostnames against the tenant apex: Legacy Ingress hostnames in tenant namespaces are now constrained to the tenant's apex domain, closing a cross-tenant hostname-claim gap (@lexfrei in [#3200]).
-
[operator] Force source-watcher drift on a stuck ArtifactGenerator status:
cozystack-platformcould time out at 15m on fresh install when a PackageSource's derived ArtifactGenerator stalled inReady: Unknowndespite its artifacts being produced — an upstreampatch.Helperrace (fluxcd/pkg#934) that loses the Ready-condition write under etcd load. The reconciler now detects the stuck signature and forces source-watcher onto its drifted-branch reconcile (@IvanHunters in [#3182]). -
[fluxcd] Omit empty distribution.artifact in FluxInstance for guest clusters: Guest Kubernetes clusters with the fluxcd addon enabled no longer fail to install; an empty distribution artifact is omitted so the operator uses its embedded manifests instead of rendering an invalid null (@IvanHunters in [#3284]).
-
[flux] Tune source-watcher concurrency and CPU: Prevents the platform install from timing out under load (@lexfrei in [#3163]).
-
[vertical-pod-autoscaler] Migrate admission-controller TLS to cert-manager: The in-chart
genCA/genSignedCerttemplates are not idempotent, so every Flux reconcile minted a fresh certificate, flipped a checksum annotation and materialised a new ReplicaSet — producing 10+ ReplicaSets in minutes under load and livelocking the admission controller on its own election lease. A namespace-scoped cert-manager Issuer and Certificate replace them, with pod rotation keyed off the actualtls.crtcontent (@IvanHunters in [#3178]). -
[cert-manager] Raise the cainjector memory limit: CA injection into webhook
caBundles no longer fails under memory pressure, which previously surfaced as intermittentx509: certificate signed by unknown authorityadmission errors (@IvanHunters in [#3199]). -
[cert-manager] Move the webhook off the kubelet's port: The admission webhook no longer listens on port 10250, which the kubelet also serves — a connection misrouted to a node answered with the kubelet's certificate, failing every cert-manager admission call cluster-wide with a misleading x509 error (@lexfrei in [#3359]).
-
[external-secrets] Move the webhook off the kubelet's port: Same class of collision; the Service also targets the port by name so the move rolls out without an admission gap (@lexfrei in [#3360]).
-
[metrics-server] Move the listener off the kubelet's port: The default made hostNetwork mode unable to start at all; the new port also keeps node-port collisions out of the metrics path (@lexfrei in [#3361]).
-
[cilium] Pass IPv6 to the kernel stack when the IPv6 datapath is disabled: Node IPv6 (ICMPv6 Neighbor Discovery, BGP unnumbered over link-local) now works with the host firewall enabled — Cilium host policies apply to IPv4 only and node IPv6 is not filtered (@lexfrei in [#2871]).
-
[kube-ovn] Raise the ovs-ovn CPU limit and request: The co-located
ovn-controller,ovsdb-serverandovs-vswitchddaemons get CPU headroom (limit 4 cores, request 100m) under burst installs (@lexfrei in [#3050]). -
[linstor] Set controller resource requests and relax the liveness probe: The linstor-controller no longer crash-loops under node CPU contention, which previously left its Service with no endpoints and blocked RWX volume provisioning (@lexfrei in [#3043]).
-
[opensearch-operator] Gate leader election on replica count: Leader election is disabled on the single-replica operator Deployment, so a transient apiserver or etcd latency spike no longer crashloops the manager at install time; it re-enables automatically past one replica (@lexfrei in [#3040]).
-
[monitoring-agents] Relax fluent-bit probes for loaded installs: The DaemonSet no longer crash-loops on a loaded install when its
:2020HTTP server binds late behind a busy kube-API (@lexfrei in [#3045]). -
[velero] Add a startupProbe so slow startup does not crashloop the install gate: Velero's slow first start no longer trips the liveness probe and fail the platform install (@lexfrei in [#3138]).
-
[kubevirt-cdi] Order behind cozystack-basics: The
cdi-clone-dvRoleBinding now findscozy-public, fixing a cold-install ordering failure (@lexfrei in [#3137]). -
[vm-default-images] Refresh stale Alpine and openSUSE 16.0 image URLs: Alpine moves 3.21.6 → 3.21.7 and the openSUSE Leap 16.0 URL is corrected, so default-image imports stop 404ing (@myasnikovdaniil in [#3063]).
-
[apps/vpn] Remove the invalid
foofield from the urls Secret: A stray field broke Helm upgrades under server-side apply (@IvanHunters in [#3281]). -
[dashboard] Unbreak CORS on an expired session for Kubernetes API calls: The dashboard SPA no longer fails with CORS errors after the
kc-accesscookie expires — oauth2-proxy returns 401 (not 302) for/apiand/apis, and the KeycloakClient declareswebOrigins: ["+"](@IvanHunters in [#2788]). -
[dashboard] Join pod phase on (namespace, pod) instead of uid: Fixes the Capacity Planning and Namespaces dashboards failing with "duplicate time series" errors when static pods are present (@myasnikovdaniil in [#2782]).
-
[cozy-lib, harbor, mariadb] Digest-pin the cleanup-hook kubectl image and route it through cozy-lib.images-registry: The post-delete cleanup hook image is now digest-pinned and resolved through the cluster images registry via a new
cozy-lib.imagehelper, so mirrored and air-gapped installs can resolve it and uninstall no longer depends on a moving Docker Hub tag (@scooby87 in [#3171]). -
[migrations] Derive the etcd-adoption snapshot target from the projected bucket credentials: Migration 50 resolves its S3 endpoint and scheme from the claimed bucket's projected Secret rather than requiring them hand-set, and makes the
etcdAdoptSkipBackupescape hatch actually reachable from the Package CR (@myasnikovdaniil in [#3335]).
Security
-
[talos] Bump Talos to v1.13.6 (CVE-2026-53359): Ships Linux 6.18.38 with the kernel fixes for the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes (@lexfrei in [#3240]).
-
[kubernetes] Bump tenant worker Talos to v1.13.6: The same kernel fixes reach tenant worker VMs — the KVM shadow-paging use-after-free and nested-guest escape into the worker VM. Nested virtualization remains available; applying this rolls each tenant worker pool once (@lexfrei in [#3269]).
-
[seaweedfs] Enforce read-only bucket access by bumping cosi-driver to v0.3.1: The bundled COSI driver hardcoded read-write S3 permissions for every
BucketAccessand ignored the-readonlyBucketAccessClass, so credentials meant to be read-only could still upload and delete objects. The driver now honoursaccessPolicy: readonlyand issues Read/List-only credentials (shipped in v1.5.2) (@lexfrei in [#3136]). -
[platform] Raise the Go toolchain to 1.26 across first-party images: Clears standard-library security advisories in every Cozystack-built binary (@lexfrei in [#2941]).
-
[deps] Bump golang.org/x/crypto to v0.52.0: Closes 13 known OSV advisories (GO-2026-5005 through GO-2026-5033) (@myasnikovdaniil in [#3019]).
-
[kamaji] Close Go stdlib and module CVEs in the shipped binary: Rebuilds the pinned Kamaji binary on a patched toolchain and module set (@lexfrei in [#3033]).
-
[build] Refresh first-party Alpine runtime bases to 3.24: Clears the musl and zlib advisories across Cozystack-built images (@lexfrei in [#3009]).
-
[kilo] Rebuild the kilo image on current upstream and Alpine 3.24: Clears the libpng advisories (@lexfrei in [#3007]).
-
[ci] Harden GitHub Actions workflows for OpenSSF Scorecard: Pins every action by commit SHA, scopes per-job
GITHUB_TOKENpermissions to least privilege, and adds azizmorgate (pre-commit plus PR workflow) that blocks future regressions (@myasnikovdaniil in [#3223], [#2849], @kvaps in [#3227]). -
[ci] Enable Renovate digest pinning and OSV vulnerability alerts: Base images and GitHub Actions are pinned to digests, indirect Go dependencies are bumped, and OSV alerts are enabled — improving the Scorecard Pinned-Dependencies and Vulnerabilities scores (@myasnikovdaniil in [#3025], [#3026]).
Dependencies & version updates
See Platform Components above for the components with user-visible upstream changes. The remaining version work in this release:
- [linstor] Bump linstor-server to 1.33.3 and linstor-csi to v1.11.2: Carries the RWX-on-non-DRBD rejection that drives the worker storageClass fallback (see Breaking Changes) (@lexfrei in [#2987]).
- [images] Bump opencost, multus and clickhouse-backup: OpenCost to 2.5.23, multus-cni to v4.3.0 and clickhouse-backup to 2.7.2 (@lexfrei in [#3000]), with clickhouse-backup later refreshed to v2.7.4 (@app/renovate in [#3226]).
- [images] Refresh stale utility image references: kube-rbac-proxy v0.22.0, matchbox v0.11.0 and alpine/k8s 1.36.2 (@lexfrei in [#2969]).
- [kubernetes] Re-pin the deleted centos:stream9 digest for kubevirt-csi-driver: Upstream deleted the pinned base-image digest, breaking the kubevirt-csi-driver build; re-pinned twice as the upstream tag moved again (@IvanHunters in [#3303], @androndo in [#3386]).
- [deps] Renovate updates: kubevirt/csi-driver digest to 27b52aa (#3066), actions/checkout to v6.0.3 (#3133), and a batch of container image and action digests (#3232) (@app/renovate).
System Configuration
A sweep of platform components that previously ran without resource requests or limits — and therefore inherited namespace LimitRange defaults or ran unbounded — now declare them explicitly:
- [reloader] Set resources requests/limits on the reloader container (@matthieu-robin in [#2631]).
- [metrics-server] Set resources requests/limits on the metrics-server container (@matthieu-robin in [#2630]).
- [external-dns] Set resources requests/limits on the external-dns container (@matthieu-robin in [#2629]).
- [bootbox] Add resources requests/limits to the matchbox container (@matthieu-robin in [#2625]).
- [bucket] Add resources requests/limits to the s3manager container (@matthieu-robin in [#2624]).
- [goldpinger] Set resources requests/limits on the goldpinger DaemonSet (@matthieu-robin in [#2623]).
- [cert-manager] Set resources requests/limits on cert-manager pods (@matthieu-robin in [#2616]).
- [kubernetes] Add CPU/memory limits to the csi-resizer sidecar (@matthieu-robin in [#2614]).
Development, Testing, and CI/CD
-
[e2e] Migrate the app suite from BATS to Chainsaw and wire it into CI: The managed-application end-to-end suite moves to Chainsaw, giving declarative, per-resource assertions in place of shell scripting (@myasnikovdaniil in [#2826]).
-
[e2e] Diff-driven install selector
select-install.sh: Computes the minimal package install set for a diff-scoped E2E run by walking the forward PackageSource dependency closure, with a--validategraph-consistency mode (@myasnikovdaniil in [#3279]). -
[ci] Require an API-owner review for sizeable API changes: A PR that introduces a new API group, a new resource, or a breaking change to an existing API schema now requires a review from an API owner. The detector discovers CRDs and APIService registrations by content rather than directory name, and builds from the base-branch tip when the merge base predates the gate (@lllamnyp in [#3167], [#3184], @lexfrei in [#3193]).
-
[ci] Guard migrations.targetVersion against a silent skip: A CI check now catches a migration added without bumping
targetVersion, which would otherwise ship a migration that never runs (@IvanHunters in [#2771]). -
[build] Parallel matrix image builds on ephemeral runners: Image builds fan out across an ephemeral-runner matrix instead of serializing on one host (@myasnikovdaniil in [#2983]).
-
[ci] Move remaining workflows off the persistent self-hosted runner: All CI jobs now run on ephemeral or GitHub-hosted runners; release tag builds warm-start from the shared registry build cache (@kvaps in [#3268]).
-
[ci] Overlay current-main images for packages a PR did not rebuild: PR e2e and the installer artifact now run current-main images for untouched packages instead of the last release's, so a PR is tested as current-main plus its own changes (@myasnikovdaniil in [#3148]).
-
[ci] Move the install gate to a 32 vCPU runner: Ends the CPU oversubscription that produced most of the install-phase flakes (@lexfrei in [#3251]).
-
[e2e] Stop the install pipeline flaking: A combined pass over VPA OOM/churn and wait budgets across the install pipeline (@kvaps in [#2957]).
-
[e2e] Make the in-sandbox Talos image cache reachable by tenant workers: Fixes deterministic
kubernetes-*E2E failures where worker CDI importers could not reach the sandbox image cache (@lexfrei in [#3254]). -
[e2e] Escalate the cilium leak healer to an agent restart on leaked-IP reuse: The only reliable recovery from a Cilium endpoint-manager IP leak under pod churn (@lexfrei in [#3051]), with the reserved:ingress IP variant skipped (#3142).
-
[testing] Refresh the e2e-sandbox tools and base: Ubuntu 24.04, kubectl 1.33.13, talosctl 1.13.5, Helm 3.21.2 (@lexfrei in [#3010]).
-
[tests] Make bats assertions and hooks actually run under cozytest: Negative assertions now fail as intended and dead
setup/teardownhooks execute, closing a class of silently-passing tests (@lexfrei in [#3132], [#3134], [#3131]). -
Additional E2E and CI work: capture host→pod CNI data-plane on failure (#3047), capture LINSTOR satellite ErrorReports in cozyreport (#3209), wait for LINSTOR pool free capacity between tenant Kubernetes tests (#3210), gate the LINSTOR node-list probe on a ready controller endpoint (#3250), drain tenant VMs/PVCs between kubernetes tests (#3159), widen tenant-root monitoring bring-up waits (#3160), widen etcd convergence and FoundationDB status budgets (#3036, [#3039]), raise the kafka/zookeeper preset off the 128Mi OOM threshold (#3052), bump the sandbox tenant storage quota to 200Gi (#3206), make the hairpin DNS check tolerate a tenant node recycle (#3241), size the e2e tenant apiserver to the chart default (#3351), fix the etcd metrics-probe attach race (#3350), drop the serviceexposure Chainsaw suite left after the feature removal (#3336), drop duplicate package test targets (#3352), route PR base-image pulls through mirror.gcr.io (#3042), remove the redundant Release E2E workflow (#2978), prune stale and redundant workflows (#2982), fix the backport workflow's
conflict_resolutionnesting (#3155), drop the pr-labelereditedtrigger (#3211), map dashboard/console and cert-manager/external-secrets/metrics-server scopes in the labeler (#3117, [#3362]), addneeds-triageon new issues and new area labels (#3165), forcechore(deps)commits and anarea/dependencieslabel from Renovate (#3015), resolve the nightly packages digest without image-specific skopeo operations (#3332), mirror and retag images whose digest is embedded in the valuestagor whose registry host lives outsiderepository— ten images across eight packages were being silently skipped, so nightly published references that 404 at install and promoted releases did not carry the:<version>tags they claimed (#3375), pin the console image builder to pnpm 11.1.2 (#3099), and block hard-wrapped prose in markdown and GitHub bodies (#3297).
Documentation
-
[docs/security] Add a design-level threat model: New
docs/security/threat-model.mddescribing trust boundaries and attacker models for the platform (@lexfrei in [#3217]). -
[docs/security] Add a security self-assessment: New
docs/security/self-assessment.mdprepared for the CNCF incubation due diligence (@tym83 in [#3224]), alongside an alignment pass on the existing security docs and a new incident-response document (#3225). -
[docs] Route bugs, features, and proposals to the right repository: Issue templates and contribution docs now direct reporters to the correct Cozystack repository (@myasnikovdaniil in [#2854]).
-
[vm-instance] Document attaching a VM to an external VLAN: New
docs/vm-external-vlan.mdcovering the Linux-bridge and bridge-typeNetworkAttachmentDefinitionpath (@lexfrei in [#3196]). -
[harbor] Fix the SeaweedFS discovery key described as an annotation: It is the
namespace.cozystack.io/seaweedfsnamespace label (@lexfrei in [#3293]). -
[readme] Add the OpenSSF Scorecard badge (@tym83 in [#3237]).
-
[docs/agents] Add a downstream repository checklist to the PR template (@lexfrei in [#3298]).
-
[docs/release] Publish the patch-release changelogs: The generated changelogs for v1.4.5, v1.5.1 and v1.5.2 land in
docs/changelogs/so the release history onmainis complete (@app/cozystack-ci in [#3083], [#3024], [#3162]). -
[website] Multi-language proof-of-concept (ru, de, zh-cn, hi): Wires four additional languages into the site — localized landing page, getting-started guide, support page and navigation — with key-parity and translation-freshness lint, non-ratified localizations excluded from search indexing, and SEO blocks per language (@tym83 in cozystack/website#593).
-
[website] Tenant network isolation post (Cilium eBPF + Kube-OVN VPC): Explains how Cozystack isolates tenant traffic across the two data planes (@tym83 in cozystack/website#614, cover image in cozystack/website#620).
-
[website] CVE-2026-53359 (Januscape) advisory and Talos fix guide: A security advisory for the KVM guest-to-host escape plus a step-by-step Talos remediation guide (@tym83 in cozystack/website#603, cozystack/website#604, cozystack/website#612).
-
[website] CVE-2026-43499 (GhostLock) exposure assessment: Documents whether and how Cozystack clusters are exposed (@tym83 in cozystack/website#613).
-
[website] Call for independent Cozystack adopters for the CNCF Incubation review (@tym83 in cozystack/website#624).
-
[website] Simplify the backup guides around the default BackupClass: Corrects the admin override path and the platform-package
backupStoragedocumentation for bothnextand v1.5 (@androndo in cozystack/website#622). -
[website] Document host firewall and node IPv6 behavior: Explains that Cilium host policies apply to IPv4 only and node IPv6 is passed to the kernel stack (@lexfrei in cozystack/website#574).
-
[website] Document the
existingSecretcertificate mode andwildcardSecretName: Corrects the publishing reference and documents the third certificate mode (@lexfrei in cozystack/website#615, cozystack/website#619). -
[website] Drop the manual KubeVirt patch step for GPU passthrough: The platform now auto-wires
permittedHostDevices, so the manual patch documented since v1.4 is obsolete; includes pre-upgrade migration steps for hand-edited entries (@lexfrei in cozystack/website#556). -
[website] Explain why packages live in
appsvsextra(@kvaps in cozystack/website#594). -
[website] cozyvalues-gen in the developer guide; sync the external apps guide with the example repo (@myasnikovdaniil in cozystack/website#610).
-
[website] talm: encrypted user values and dry-run secret redaction (@lexfrei in cozystack/website#590).
-
[website] Refresh the telemetry snapshot to June 2026 and fix per-period tenants (@tym83 in cozystack/website#598).
The following documentation shipped in the v1.5.x patch releases and is included for completeness:
- [website] Add a containerized GPU workloads guide: Operations guide for the
containervariant ofcozystack.gpu-operator(shipped in v1.5.2) (@lexfrei in cozystack/website#555). - [website] Warn against changing
op:on the Talos machine files block: Talos rejectsop: createfor files outside/var, so the/etc/lvm/lvm.confentry must keepop: overwrite(shipped in v1.5.2) (@lexfrei in cozystack/website#575). - [website] Document talm chart drift detection and strict mode (shipped in v1.5.2) (@lexfrei in cozystack/website#577).
- [website] Document
publishing.certificateswildcard options (shipped in v1.5.2) (@lexfrei in cozystack/website#588). - [website] Correct stale version pins and prevent next.yaml drift (shipped in v1.5.2) (@myasnikovdaniil in cozystack/website#589).
- [website] Cozystack v1.5.0 release announcement (shipped in v1.5.2) (@tym83 in cozystack/website#592).
Other Repositories
talm v0.33.0 / v0.33.1
- [talm] Detect vendored chart and preset-template drift:
talmnow warns when the vendored charts in a project drift from the talm binary's embedded library, and detects preset-template drift via a new.talm-preset.lockfile (commit it).talm init --updatebecomes an exact re-sync that names the drifted paths, andstrictCharts: true/--strict-chartsescalates warnings to hard failures — blocking on unverifiable or missing drift baselines rather than passing silently. CRLF line endings are treated as checkout artifacts, not drift (@lexfrei in cozystack/talm#216). - [talm] Disable and then restore KVM nested virtualization in the Talos presets: v0.33.0 disabled nested virtualization in the
cozystackpreset as an immediate mitigation for CVE-2026-53359, pinninggrubUseUKICmdline=falseso the kernel args actually apply on Talos v1.12+. v0.33.1 reverts that mitigation now that the Talos v1.13.6 kernel carries the upstream fix — so nested virtualization is available again on a patched fleet (@lexfrei in cozystack/talm#224, cozystack/talm#225).
talm v0.32.0 (shipped in v1.5.2)
- [talm] First-class encrypted user values, honored at template and apply: A value file named
*.encrypted.yamlreferenced fromtemplateOptions.valueFilesis age-decrypted in memory at bothtalm templateandtalm apply, withtalm init --encrypt/--decryptcoveringvalues-secret.yaml↔values-secret.encrypted.yaml. Secret-bearing fields are redacted by default from rendered node files, stdout and the apply drift preview, with--show-secrets/--show-secrets-in-driftto reveal them (@lexfrei in cozystack/talm#222).
ansible-cozystack v1.5.1 / v1.5.2 / v1.5.3
- [ansible-cozystack] Make prepare and k3s.orchestration runnable as separate invocations: The example playbooks can now be run as independent steps rather than requiring a single combined invocation (@kvaps in cozystack/ansible-cozystack#52).
- [ansible-cozystack] Track the cozy-installer and k3s versions: The installer image is bumped to v1.5.1, v1.5.2 and v1.5.3 in step with the Cozystack patch releases, and k3s to v1.36.2+k3s1 (@app/renovate in cozystack/ansible-cozystack#56, cozystack/ansible-cozystack#57, cozystack/ansible-cozystack#58, cozystack/ansible-cozystack#59).
Contributors
We'd like to thank all contributors who made this release possible:
- @androndo
- @IvanHunters
- @kvaps
- @lexfrei
- @lllamnyp
- @mattia-eleuteri
- @matthieu-robin
- @myasnikovdaniil
- @scooby87
- @shreyaabaranwal
- @sircthulhu
- @tym83
New Contributors
We're excited to welcome our first-time contributor:
- @shreyaabaranwal - First contribution!
Full Changelog: https://github.com/cozystack/cozystack/compare/v1.5.0...v1.6.0