Download Latest Version metal-amd64.iso (543.5 MB)
Email in envelope

Get an email when there's a new version of cozystack

Home / v1.6.0
Name Modified Size InfoDownloads / Week
Parent folder
README.md 2026-07-22 99.4 kB
v1.6.0 source code.tar.gz 2026-07-22 7.6 MB
v1.6.0 source code.zip 2026-07-22 10.4 MB
openapi.json 2026-07-22 2.6 MB
cozypkg-checksums.txt 2026-07-22 564 Bytes
cozypkg-windows-arm64.tar.gz 2026-07-22 22.9 MB
cozypkg-darwin-amd64.tar.gz 2026-07-22 26.7 MB
cozypkg-darwin-arm64.tar.gz 2026-07-22 24.8 MB
cozypkg-linux-amd64.tar.gz 2026-07-22 25.4 MB
cozypkg-linux-arm64.tar.gz 2026-07-22 22.9 MB
cozypkg-windows-amd64.tar.gz 2026-07-22 25.6 MB
initramfs-metal-amd64.xz 2026-07-22 155.2 MB
kernel-amd64 2026-07-22 20.4 MB
nocloud-amd64.raw.xz 2026-07-22 349.0 MB
metal-amd64.raw.xz 2026-07-22 349.0 MB
metal-amd64.iso 2026-07-22 543.5 MB
cozystack-operator-hosted.yaml 2026-07-22 2.8 kB
cozystack-operator-generic.yaml 2026-07-22 2.9 kB
cozystack-operator-talos.yaml 2026-07-22 2.9 kB
cozystack-crds.yaml 2026-07-22 20.0 kB
Totals: 20 Items   1.6 GB 0

Cozystack v1.6.0

Cozystack v1.6.0 replaces the Ubuntu + kubeadm bootstrap of tenant Kubernetes workers with Talos Linux driven by Cluster API, completes the etcd-operator v1alpha2 migration with in-place adoption of live clusters, adds OIDC single sign-on for tenant kube-apiservers and per-instance Grafana, introduces the tenant-facing SecurityGroup firewall API, makes tenant resource quotas hierarchical so a sub-tenant can no longer escalate past its parent's budget, gives Keycloak an optional KMS-encrypting database proxy, a separate admin hostname and S3 backups, propagates operator-provided wildcard certificates down to every tenant termination point, and reworks the release pipeline around immutable tags and rc→stable promotion so a stable release is the byte-identical promotion of the release candidate that was tested. A ten-PR sweep also makes application deletion actually reclaim its storage instead of leaking PVCs. The release rolls up every fix from v1.5.1, v1.5.2 and v1.5.3.

Platform components bumped in this release: Talos v1.13.0 → v1.13.6 (KVM guest-escape CVE fixes), etcd-operator v0.4.5 → v0.5.2 (new etcd-operator.cozystack.io/v1alpha2 API), Cilium 1.19.3 → 1.19.5, KubeVirt v1.8.4, Velero 1.17.0 → 1.18.1, Vertical Pod Autoscaler 1.3.0 → 1.5.0, Harbor 2.14.2 → 2.15.1, Keycloak 26.5.2 → 26.6.3, LINSTOR 1.33.2 → 1.33.3 with linstor-csi v1.11.2, FoundationDB operator v2.13.0 → v2.30.0, HAMi 2.8.1 → 2.9.0, Percona MongoDB operator 1.21.1 → 1.22.0, OpenBao v2.5.0 → v2.5.1, CoreDNS chart 1.43.2 → 1.46.0, csi-driver-nfs 4.11.0 → 4.13.3, OpenCost 1.111.0 → 1.120.3, NATS 2.11.8 → 2.11.10, linstor-scheduler v0.3.2 → v0.3.6, and ouroboros 0.8.0 → 0.8.1.

Note: Items marked (shipped in v1.5.x) were also released in the v1.5.1, v1.5.2 or v1.5.3 patch releases.

⚠️ Breaking Changes and Required Actions

This is the largest upgrade surface since v1.0. Read this section in full before upgrading — three of the items below can block or wedge the upgrade if their preconditions are not met.

The platform migration targetVersion moves from 45 (v1.5.x) to 54, which means migrations 45 through 53 run as pre-upgrade hooks on the way to v1.6.0.

Pre-upgrade checks

Run these against the management cluster before applying the v1.6.0 Platform Package.

1. etcd adoption needs a reachable backup target (migration 50 — hard-fails the upgrade).

Migration 50 adopts every legacy etcd.aenix.io/v1alpha1 cluster onto the new v1alpha2 operator and takes a mandatory pre-adoption etcd snapshot first. If it cannot resolve the platform-managed snapshot target it exits 1 and halts the upgrade (#2859, [#3270], [#3335]).

:::sh
# Do you have legacy etcd clusters at all? If this prints nothing, migration 50 is a no-op.
kubectl get etcdclusters.etcd.aenix.io -A

# The snapshot target: the cozy-default-etcd strategy and the projected credentials Secret.
kubectl get etcds.strategy.backups.cozystack.io cozy-default-etcd
kubectl get secret cozy-backups-creds -n cozy-velero \
  -o jsonpath='{.data.AWS_ACCESS_KEY_ID}{"\n"}'

# The bucket behind it must be healthy.
kubectl get buckets.apps.cozystack.io cozy-backups -n tenant-root

If the Secret or the strategy is missing and you have legacy etcd clusters, either repair the cozy-backups Bucket / backupstrategy-controller first, or accept adopting live etcd without a snapshot by taking the documented escape hatch (see Manual actions required).

2. SeaweedFS naming audit (chart refuses to render for two classes of tenant).

The v1.5.0 SeaweedFS chart bump to 4.31 renamed workloads from chart-based (seaweedfs-*) to release-based (seaweedfs-system-*) names. v1.6.0 pins fullnameOverride: seaweedfs and adopts the running set in place, but two states cannot be adopted automatically and the chart fails the render rather than guess (#3282, [#3339]).

:::sh
# Step 0 — seaweedfs-db ownership. A missing row means the filer metadata is ALREADY lost.
kubectl get cluster.postgresql.cnpg.io -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,OWNER:.metadata.annotations.meta\.helm\.sh/release-name,KEEP:.metadata.annotations.helm\.sh/resource-policy'

# Step 1 — classify every SeaweedFS instance (read-only).
hack/seaweedfs-naming-audit.sh

Correction, added after release: the copy of that script shipped in v1.6.0 fails open, so a clean result from it is not evidence. Every kubectl call in the v1.6.0 version is silenced with 2>/dev/null, so a timeout, an RBAC denial or any transient API error produces an empty "all clean" table byte-identical to an honestly clean fleet — on the one script whose output gates a runbook step that deletes PVCs. Fixed in [#3436] and backported as [#3474], both merged 2026-07-28, so v1.6.1 and later carry a version that prints FATAL and exits non-zero instead. Run the audit from a v1.6.1-or-later checkout, read the exit code rather than the table, and re-run it if the only run you have was against the v1.6.0 copy.

Class L needs no action. Class S (installed fresh on 1.5.x) and class MIXED (both naming generations present) must be recovered before upgrading — follow docs/operations/seaweedfs-431-rename-recovery.md. A cluster that went 1.4.x → 1.6.0 directly never renamed and is unaffected.

3. Tenant Kubernetes clusters still on v1.30.

v1.30 leaves the Talos↔Kubernetes support matrix and the chart now refuses to render it. Migration 46 patches live CRs from v1.30 to v1.31, but a GitOps-managed CR is re-overwritten by the next source reconcile (#2931).

:::sh
kubectl get kuberneteses.apps.cozystack.io -A \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,VERSION:.spec.version'

Any row showing v1.30 that is reconciled from Git must have spec.version bumped to v1.31 (or newer) in Git, before or together with the platform upgrade.

4. Hand-made tenant StorageClasses that collide with propagated LINSTOR classes.

Remote-accessible LINSTOR StorageClasses are now created inside each tenant cluster under the same name. A manually created tenant StorageClass with a colliding name (typically replicated) blocks the propagated class and stalls the tenant CSI release (#2872).

:::sh
# Inside each tenant cluster:
kubectl get storageclass replicated -o jsonpath='{.metadata.annotations.meta\.helm\.sh/release-name}{"\n"}'

Delete any such class that is not Helm-managed before upgrading. Infra classes that must stay node-local need an explicit linstor.csi.linbit.com/allowRemoteVolumeAccess: "false" — an absent annotation is treated as remote-accessible.

5. Deprecated etcd tenant-module backup values.

The backup.* block on the etcd tenant module is removed in this release. Confirm nothing still relies on it (#2859).

:::sh
kubectl get etcds.apps.cozystack.io -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\t"}{.spec.backup.enabled}{"\n"}{end}'

6. If you are coming from v1.4.x, the v1.5.0 requirement still applies: the management cluster (and any tenant cluster enabling the Flux addon) must be on Kubernetes 1.33+.

Breaking changes

  • Tenant Kubernetes workers move from Ubuntu + kubeadm to Talos Linux. What changed: worker bootstrap is now driven by cluster-api-bootstrap-provider-talos (CABPT) with a talos-csr-signer sidecar in the Kamaji control-plane pod; workers boot the Talos openstack image streamed in by CDI. Who is affected: every existing tenant Kubernetes cluster. What to do: nothing mandatory — existing machines roll over to Talos workers automatically on the first reconcile after upgrade — but plan for a full worker-pool replacement per tenant cluster, and expect worker disks to be reprovisioned and container images re-pulled. The separate disk-kubelet PVC is gone; nodeGroups[*].diskSize now sizes the single Talos system disk (@myasnikovdaniil in [#2931]).

  • Kubernetes v1.30 removed from the tenant support matrix. What changed: the version enum drops v1.30; the chart fails the render with an explicit guard. Who is affected: tenants still pinned to v1.30. What to do: migration 46 bumps live CRs to v1.31; GitOps-managed CRs must be bumped in Git (see pre-upgrade check 3) (#2931).

  • Worker MachineHealthCheck remediation is now ON by default. What changed: the worker MHC maxUnhealthy moved from a hard-coded 0 (remediation effectively disabled) to nodeHealthCheck.maxUnhealthy, defaulting to "50%" — CAPI now deletes and replaces unhealthy worker Machines. Who is affected: every tenant Kubernetes cluster. What to do: set nodeHealthCheck.maxUnhealthy: "0%" to keep the previous behaviour until your fleet is stable on Talos workers (#2931).

  • The default md0 node group is no longer merged into every cluster. What changed: nodeGroups defaults to {} and the built-in md0 is applied only when no node groups are configured, so it can finally be removed or replaced. Who is affected: clusters that defined their own node groups and were silently also getting md0. What to do: nothing — migration 47 pins md0 explicitly on existing CRs to preserve the live topology; remove it afterwards if you do not want it. Migration 47 is fail-closed: a read or patch failure aborts the upgrade rather than let Helm prune a live md0 MachineDeployment (@myasnikovdaniil in [#2936]).

  • Fresh tenant clusters with nodeGroups: {} come up with zero workers. What changed: the chart no longer manages MachineDeployment.spec.replicas — the cluster-autoscaler owns it alone, seeded from minReplicas: 0. Who is affected: new clusters that enable addons.ingressNginx.enabled: true on a default CR. What to do: either supply a node group with roles: [ingress-nginx] and minReplicas >= 1, or let the autoscaler bring up md0 once the ingress-nginx pods go Pending. Upside for existing clusters: helm upgrade no longer drains workers back to a hardcoded replicas: 2 on every platform bump (@IvanHunters in [#3233]).

  • etcd moves to etcd-operator.cozystack.io/v1alpha2. What changed: the legacy etcd.aenix.io/v1alpha1 StatefulSet model is replaced by a Membership-API lifecycle; the vendored upstream chart is replaced by a Cozystack-authored one at appVersion v0.5.2, and CRDs move into a dedicated etcd-operator-crds package. Who is affected: every cluster running the etcd tenant module or tenant Kubernetes (Kamaji datastores). What to do: existing clusters are adopted in place by migration 50 — no data move, no pod restart — provided the pre-upgrade snapshot target resolves (see pre-upgrade check 1). A pre-upgrade hook also deletes the pre-1.6 operator Deployment, whose spec.selector is immutable and would otherwise fail the Helm upgrade (@androndo in [#2859], @myasnikovdaniil in [#3270]).

  • The deprecated backup.* values on the etcd tenant module are removed. What changed: backup.enabled, schedule, destinationPath, endpointURL, region, forcePathStyle, s3AccessKey, s3SecretKey and the job-history limits no longer exist; a new version value (default 3.6.11) pins the etcd release. Who is affected: tenants that configured legacy per-release S3 backups on the etcd module. What to do: migrate to the BackupClass flow driven by the Etcd strategy (strategy.backups.cozystack.io/v1alpha1) before upgrading (#2859).

  • Deleting a managed application now permanently deletes its storage. What changed: ten packages gained post-delete cleanup that reclaims previously-orphaned PVCs and Secrets. Deletion is irreversible — this is the point of the change, but it is a real behaviour change from "delete leaks the volume, and you could recover from it". Who is affected: ClickHouse (keeper + data + log PVCs), Qdrant (data PVC), OpenBAO (data PVC), the tenant monitoring module (VictoriaMetrics/VictoriaLogs storage PVCs), the tenant seaweedfs module (volume PVCs), the tenant etcd module (data-etcd-* PVCs), Harbor (jobservice/trivy PVCs — this force-deletes them, overriding persistence.resourcePolicy=keep), MariaDB (operator-generated Secrets), Bucket and Gateway (ACME solver resources). What to do: back up before deleting; snapshot anything you care about. Migrations 48 and 51 backfill the release label onto pre-existing ClickHouse keeper PVCs and monitoring storage PVCs so already-deployed clusters are covered too. Harbor's CNPG database PVCs are not touched (@scooby87 in [#3072], [#3075], [#3073], [#3094], [#3095], [#3170], [#3077], [#3071], [#3076], [#3093]).

  • Tenant resource quotas are now hierarchical. What changed: a tenant's declared quota is the budget for its whole sub-tree — a child that declares a quota carves a fixed slice out of the parent's remaining budget, a child without one shares the parent's pool instead of being unbounded, and a child quota exceeding the parent's remaining budget is rejected at admission. Who is affected: multi-level tenant trees where a sub-tenant previously declared more than its parent, or nothing at all. What to do: review sub-tenant quotas before upgrading; an overcommitted pool emits a QuotaOvercommitted event. Set --tenant-quota-buffer-percent on cozystack-controller for a temporary grace buffer so workloads already over a freshly-binding quota keep running during rollout (@kvaps in [#3086]).

  • Worker node disks now default to the application-level (DRBD replicated) StorageClass. What changed: a node group leaving storageClass empty previously picked the management-cluster default StorageClass; it now falls back to the application storageClass, because linstor-csi v1.11.2 rejects ReadWriteMany volumes on a non-DRBD class and worker VMs need RWX to live-migrate. Who is affected: tenant clusters relying on the empty-storageClass default. What to do: nothing, but the rendered worker template changes, so each tenant's worker MachineDeployment rolls once on the first reconcile after upgrade (@lexfrei in [#3129], [#2987]).

  • authentication.oidc.keycloakInternalUrl now defaults to the in-cluster Keycloak Service. What changed: the default is http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy instead of empty, so OIDC backend calls no longer leave the cluster. Who is affected: clusters that never set the value explicitly. What to do: nothing, but the dashboard and linstor-gui gatekeepers roll once. Set the value explicitly for a non-standard Keycloak placement, or to an empty string to restore external OIDC discovery (@lexfrei in [#3289]).

  • Air-gapped tenant workers: registries.mirrors passthrough is gone. What changed: the Helm-rendered *-patch-containerd Secret that plumbed the platform-wide registries.mirrors config to tenant workers has no consumer in the Talos machineconfig and was removed. Who is affected: tenants relying on registries.mirrors for in-guest container-image pulls. What to do: the Talos OS image and installer are overridable — set talos.imageFactoryURL and talos.installerRepository to your mirror — but in-guest registry mirroring is a Phase 2 follow-up (#2931, [#3244]).

  • SecurityGroup is served with a membership-group shape. The new sdn.cozystack.io/v1alpha1 SecurityGroup API landed inside this release with spec.targetRef replaced by a spec.attachments[] membership model before release, so there is no upgrade path to worry about — but anyone tracking main between 2026-06-30 and 2026-07-16 must rewrite their objects (@lexfrei in [#2922]).

  • The network.cozystack.io API group (ExposureClass / ServiceExposure) was introduced and removed within this cycle. It never shipped in a release and is not part of v1.6.0; native Service type: LoadBalancer + loadBalancerClass covers the same ground, exposed as publishing.loadBalancerClass. Only clusters tracking main that explicitly set publishing.exposureClass need to provision their own MetalLB/Cilium address pool and switch to publishing.loadBalancerClass; any orphaned cozystack-<class> pool can be deleted by hand (@lexfrei in [#3081], removed by @kvaps in [#3218]).

Manual actions required

  • BEFORE the upgrade — recover SeaweedFS tenants classified S or MIXED. The chart refuses to render for them, which blocks the tenant's upgrade until an operator resolves it. Run hack/seaweedfs-naming-audit.sh and follow docs/operations/seaweedfs-431-rename-recovery.md. Do not guess which generation holds the data — the runbook exists because a duplicate that briefly served writes is indistinguishable from one that never scheduled (#3339).

  • BEFORE the upgrade — bump GitOps-managed tenant Kubernetes CRs off v1.30. Migration 46 only patches live objects; Git wins the next reconcile and the HelmRelease then fails the version guard (#2931).

  • BEFORE the upgrade — delete colliding hand-made tenant StorageClasses (see pre-upgrade check 4) (#2872).

  • DURING the upgrade, only if migration 50 halts — if the cluster intentionally has no backup storage (backups disabled, external S3 without staged credentials, or SeaweedFS not yet ready) and you accept adopting live etcd without a safety snapshot, take the documented escape hatch:

sh kubectl edit package.cozystack.io cozystack.cozystack-platform yaml spec: components: platform: values: migrations: etcdAdoptSkipBackup: true

Flux re-renders the platform chart and re-runs the hook, adopting every legacy etcd without a snapshot. A botched adoption on a tenant control-plane etcd is unrecoverable without a snapshot. Set the flag back to false once the upgrade completes, or the next etcd migration will skip its snapshot too (#2859, [#3335]).

  • AFTER the upgrade — review worker MachineHealthCheck tolerance. Remediation is now on by default at 50%; set nodeHealthCheck.maxUnhealthy: "0%" (cluster-wide) or per node group if you want the old behaviour while the fleet settles on Talos (#2931, [#3053]).

  • AFTER the upgrade — re-check metrics scrape targets on ports 10250. cert-manager's webhook, external-secrets' webhook and metrics-server's listener all moved off port 10250, which the kubelet also serves (#3359, [#3360], [#3361]).

  • No action, but expect one rollout each: tenant worker pools (Talos rollover and the storageClass fallback), the dashboard and linstor-gui gatekeepers (OIDC internal URL default), the VPA admission controller (cert-manager-issued TLS), and the linstor-scheduler admission Deployment (migration 52 deletes it so the new immutable selector can be created cleanly).

Deprecations

  • Legacy per-app S3 backup fields. The etcd module's backup.* block is now removed (above); the equivalent legacy fields on Postgres and ClickHouse remain deprecated in favour of the platform-managed BackupClass flow introduced in v1.5.0.
  • Classic per-host ACME certificates for platform services. Still the default, but publishing.certificates.wildcard (DNS-01) and publishing.certificates.wildcardSecretName (operator-provided) are now the recommended path at scale, and the operator-provided certificate propagates to per-tenant termination points automatically (#2988, [#2990]).
  • auto-release.yaml cron patch releases are gone — stable releases are cut only by explicit rc→stable promotion (#3017).

Feature Highlights

Talos Linux Tenant Kubernetes Workers

Tenant Kubernetes worker nodes no longer boot Ubuntu and bootstrap through kubeadm. Phase 1 of the Kubernetes-app split design replaces that path with Talos Linux, driven by cluster-api-bootstrap-provider-talos (CABPT v0.6.12) registered as a second bootstrap provider alongside kubeadm, and a clastix/talos-csr-signer sidecar embedded in the Kamaji control-plane pod.

Talos PKI (an Ed25519 CA plus trustd TLS) is generated through cert-manager with stable, lookup-and-reuse random Talos secrets; the Kamaji control-plane provider is bumped with an upstream-bound patch that exposes KamajiControlPlane.spec.network.additionalServicePorts so trustd (50001/TCP) can be published on the apiserver Service. Workers boot the Talos openstack raw image via a CDI DataVolume streamed from the image factory, with the system disk exposed as virtio-blk using blockSize.custom: logical=512, physical=4096 so 4Ki-native backends such as LINSTOR/DRBD behave under QEMU's O_DIRECT writes while SeaBIOS still boots. The separate kubelet disk is gone — Talos lays out EPHEMERAL itself on the single system disk that diskSize now sizes.

Existing tenants roll over automatically: old machines are replaced by Talos workers without manual intervention, and platform migration 45 pins the outgoing KubeadmConfigTemplate objects with helm.sh/resource-policy: keep so Helm cannot prune them out from under the still-live kubeadm MachineSet mid-rollover.

Two behaviour changes are worth planning for: the tenant kubernetes HelmRelease now reports Ready as soon as Helm completes (DisableWait) and no longer blocks on worker or addon readiness — worker-rollout health is tracked by WorkloadMonitor — and worker MachineHealthCheck remediation is enabled by default at maxUnhealthy: "50%". See the Breaking Changes section above for the full action list (@myasnikovdaniil in [#2931]).

Follow-ups shipped in the same release make the rollover practical: per-node-group maxUnhealthy / nodeStartupTimeout overrides so a stateful group can sit at 0% while a stateless one tolerates 50% (#3053), overridable talos.imageFactoryURL and talos.installerRepository for air-gapped, mirrored or rate-limited environments (#3244), the TalosConfigTemplate moved out of a post-install hook into the main install phase so worker MachineDeployments scale reliably (#3145), and MachineDeployment.spec.replicas handed entirely to the cluster-autoscaler so platform bumps stop draining workers (#3233).

etcd-operator v1alpha2 with In-Place Adoption

Cozystack's etcd stack moves to the donated etcd-operator.cozystack.io/v1alpha2 API — a Membership-API lifecycle replacing the StatefulSet model — served by a Cozystack-authored chart at appVersion v0.5.2, with CRDs split into their own etcd-operator-crds package so they can be installed ahead of the controller.

The interesting part is the upgrade. Platform migration 50 runs as a pre-upgrade hook, while the legacy operator is still the running one, and uses the etcd-migrate tool baked into the migrations image to rewrite ownership, labels and CRs so the new operator takes over the live data plane on its first reconcile — no data move, no pod restart. Before touching anything it takes a mandatory snapshot of every adopted cluster to the platform-managed cozy-backups bucket, under a system key prefix with tenant-invisible credentials, and it fails loudly rather than adopt without one. It also re-issues each cluster's server and peer certificates with the new operator's native member wildcard SAN first, because in secretRef TLS mode the operator never mints certs — an adopted cluster whose certs lack the native domain would silently fail TLS the first time a member is replaced, while still reporting Available=True.

The in-cluster 1.5 → 1.6 path needed three more fixes to actually work, all folded into [#3270]: a chart-managed transitional etcd-headless Service so adopted members stay resolvable at their legacy per-pod DNS names until they roll onto the native subdomain; a pre-upgrade hook that deletes the pre-1.6 operator Deployment, whose spec.selector is immutable and would otherwise fail the whole HelmRelease with field is immutable; and a raised 256Mi memory floor so the operator does not OOM before the VPA admission webhook can scale it. Backup strategies also derive their S3 endpoint, scheme and TLS mode from the provisioned bucket Secret instead of requiring it hand-set.

The tenant etcd module's deprecated backup.* values are removed in the same change — use a BackupClass bound to the Etcd strategy instead (@androndo in [#2859], @myasnikovdaniil in [#3270], [#3335]).

OIDC Single Sign-On for Tenant Kubernetes and Grafana

Two symmetric Phase 1 features let a tenant opt an individual workload into the platform's Keycloak cozy realm through one flat selector, with no platform-level configuration.

Tenant kube-apiserver. Each Kubernetes CR gains spec.oidc.mode: System | CustomConfig | None (default None). System trusts the cozy realm via a per-cluster public client and audience binding; CustomConfig accepts a tenant-supplied structured AuthenticationConfiguration. spec.oidc.users[] drives one ClusterRoleBinding per user inside the tenant cluster (admincluster-admin, viewview). In System mode a <release>-oidc-kubeconfig Secret carrying a ready-to-use kubectl oidc-login exec block is surfaced through the dashboard, so a tenant user goes from "cluster exists" to "kubectl works with my SSO identity" without an operator in the loop (@IvanHunters in [#3044]). The underlying passthrough — controlPlane.apiServer.extraArgs, extraVolumes and extraVolumeMounts on KamajiControlPlane — is also exposed directly for anyone wanting to hand-roll authentication (@lexfrei in [#3123]).

Grafana. Each Monitoring CR gains the same spec.oidc.mode selector. System wires a per-instance confidential Keycloak client with audience binding; authorization is app-side, with spec.oidc.users: [{email, role: Admin|Editor|Viewer}] reconciled into Grafana's Main Org. by a chart-owned post-install/post-upgrade Job that pre-provisions users, adds them to the org, PATCHes their role and prunes stale members. CustomConfig accepts a tenant-supplied [auth.generic_oauth] payload, either inline as a map (the chart merges the two contract-critical settings on top) or as a Secret with an auth.ini key — in which case spec.oidc.users is unsupported and the chart fails the render on that combination rather than silently ignoring it. The admin_user/admin_password Secret remains a documented break-glass path in every mode (@IvanHunters in [#3176]).

Both are documented in-repo at docs/oidc-tenant.md and docs/oidc-grafana.md.

SecurityGroup: Tenant-Managed Network Policy

A new tenant-facing, namespace-scoped firewall resource — sdn.cozystack.io/v1alpha1 SecurityGroup — lets tenants manage their applications' network policy without any access to the cilium.io API group.

A SecurityGroup is a membership group: it owns a membership label (securitygroup.sdn.cozystack.io/<name>) that a new securitygroup-controller stamps onto the pods of every managed application listed in spec.attachments, and removes when an attachment is dropped. The group projects 1:1 onto a CiliumNetworkPolicy in the same namespace whose endpointSelector is exactly that membership label — so one SecurityGroup can cover several applications at once, and fromSG/toSG peers resolve group-to-group references live in the Cilium dataplane. A finalizer guards the backing policy so member labels are always stripped before it is removed, re-asserted by the aggregated-API REST storage on every write so a full-replace PUT cannot orphan them.

Ingress and egress rules only ever add allowed traffic; an empty list does not isolate the member pods, because effective connectivity remains the union of every policy selecting a pod including the platform's blanket-allow baseline. Default-deny enforcement is tracked separately as future work (@lexfrei in [#2922]).

Hierarchical Tenant Resource Quotas

tenant.spec.resourceQuotas previously became a plain per-namespace ResourceQuota that ignored the tenant tree entirely — a super-admin inside a quota'd tenant could create a sub-tenant with a larger quota, or no quota at all, and consume more than was allocated. A tenant's declared quota is now the budget for its whole sub-tree.

Enforcement is two cooperating layers, mirroring OpenShift's ClusterResourceQuota split. A declaration-time gate in the aggregated apiserver deterministically rejects a sub-tenant whose declared quota exceeds the parent's remaining budget. A runtime enforcer in cozystack-controller maintains a per-namespace tenant-quota-allocated ResourceQuota that clamps each pool member to its share of the parent budget, aggregating sub-tree usage so members collectively stay within it — Kubernetes enforces the most restrictive ResourceQuota in a namespace, so this binds without fighting Flux over the chart-rendered tenant-quota.

A parent quota lowered below the sum of its children emits a QuotaOvercommitted event, and --tenant-quota-buffer-percent on cozystack-controller temporarily inflates pool budgets during rollout so workloads already over a freshly-introduced quota keep running. No API types and no tenant chart changes — the feature works with the existing field (@kvaps in [#3086]).

Keycloak: Encryption Proxy, Admin Hostname, and Backups

Keycloak gets four independent additions this release, all opt-in.

An optional KMS-encrypting database proxy provides column-level encryption for PII at rest, backed by either a static KEK or Vault Transit, and off by default (@kvaps in [#3003]). A follow-up adds Vault Kubernetes and AppRole auth for the proxy, a dedicated ServiceAccount, and a shared DEK set covering both migration and HA (@sircthulhu in [#3239]).

An opt-in ingress.adminHost serves the admin console and Administration REST API on a separate hostname and route, with publishing.ingressNameAdmin (_cluster.expose-ingress-admin) attaching that route to a separate — for example private — Gateway or ingressClass. Disabled by default; existing deployments are unchanged (@sircthulhu in [#2954]).

S3 backups for the Keycloak CNPG database land via Barman, configurable under backup.* and disabled by default (@kvaps in [#3174]). Finally, the realm login theme is now selectable and its theme image shipped through platform branding values (branding.loginTheme, branding.loginThemeImage) (@lexfrei in [#3124]), and the database host, port and URL properties are overridable so the connection can be routed through an external proxy (@sircthulhu in [#2907]).

Wildcard Certificates, End to End

v1.5.0 let operators point platform services at a pre-existing wildcard TLS Secret through publishing.certificates.wildcardSecretName, but only for the root tenant. That certificate is now replicated by the platform controller into every tenant namespace that terminates TLS, so per-tenant ingress controllers and Gateways serve it automatically — no extra operator input, and no cross-namespace Secret read (@lexfrei in [#2990]).

For operators who would rather have the platform mint the wildcard, a new opt-in publishing.certificates.wildcard (default false) issues a single *.<root-host> certificate via a DNS-01 solver on the default ingress-nginx path, instead of a per-host ACME certificate — the practical fix for hitting Let's Encrypt rate limits at scale (@lexfrei in [#2988]).

Immutable Tags and rc→stable Promotion

A stable release is now a renamed release candidate: the bytes shipped as vX.Y.Z are bit-for-bit the bytes built and end-to-end tested as vX.Y.Z-rc.N. No tag is ever force-moved and stable is never rebuilt — it is promoted by retagging the rc's existing images by digest.

The enabler was decoupling the operator's self-reported version from its image: it now reads COZYSTACK_VERSION from the environment (threaded through cozystackOperator.platformVersion), falling back to the build-time value, so the same bits can report any release name. All five force-retag sites in the release workflows are removed and replaced with write-once or fast-forward-only semantics, and the cron-driven auto-release.yaml patch-tag workflow is deleted outright — stable tags now exist only through explicit promotion. A promote-rc.yaml workflow validates the rc, skopeo copys every digest-pinned image reference to the stable tag, restamps the version-stamped assets, and opens a release-X.Y.Z PR whose merge cuts the write-once stable tag (@myasnikovdaniil in [#3017]). A companion Cut Pre-release Tag workflow covers the rc side of the same flow (#3253).

Application Deletion Reclaims Its Storage

Deleting a managed application used to leave its volumes and operator-generated Secrets behind. A ten-PR sweep across the catalog closes that, adding post-delete cleanup hooks and PVC retention policies so a deleted application actually releases what it held: ClickHouse keeper and data/log PVCs, Qdrant's data PVC, OpenBAO's data PVC, the tenant monitoring module's VictoriaMetrics/VictoriaLogs storage and TLS Secrets, the tenant seaweedfs module's volume PVCs and cert/db Secrets, the tenant etcd module's data-etcd-* PVCs and five TLS Secrets, Harbor's jobservice and trivy PVCs plus ACME solver resources, MariaDB's operator-generated password Secrets, Bucket's ACME solver resources, Gateway's ACME account Secret, and the Postgres init Job.

Two migrations backfill the release label onto storage provisioned before this change — 48 for ClickHouse keeper PVCs, 51 for monitoring storage PVCs — because the respective operators only stamp volumeClaimTemplates labels onto PVCs they create after the change and never re-label existing ones. Both are best-effort by design: the worst case if a relabel is skipped is the pre-existing leak, which is no worse than the status quo.

This makes deletion destructive. See the Breaking Changes section (@scooby87 in [#3069], [#3071], [#3072], [#3073], [#3075], [#3076], [#3077], [#3078], [#3093], [#3094], [#3095], [#3170]).

Platform Components

  • Talos Linux: v1.13.0 → v1.13.6 (management-cluster image and matchbox assets), with tenant workers on the same v1.13.6. Closes the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes. (release notes) (@lexfrei in [#3008], [#3240], [#3269])
  • Kernel 6.18.29 → 6.18.38, carrying the KVM shadow-paging use-after-free and nested-guest escape fixes. Nested virtualization remains available.
  • containerd 2.2.5, runc 1.4.3, etcd 3.6.11, Kubernetes 1.36.1 in the bundled components.
  • Reliability fixes across the range: DNS server and listeners recreated on host-DNS runner restart, correct int marshalling of kube-scheduler config, relaxed LUKS header and hostname validation, more resources marked sensitive, higher etcd open-file limit, FailurePauseTimeout honoured before reboot.
  • System extensions refreshed to the 20260622 set, including DRBD 9.3.2 and ZFS 2.4.3. Applying this rolls each tenant worker pool once, as the worker template is renamed to carry the new image.

  • etcd-operator: v0.4.5 (vendored upstream chart) → v0.5.2 (Cozystack-authored chart, etcd-operator.cozystack.io/v1alpha2). CRDs split into a new etcd-operator-crds package. See Feature Highlights and Breaking Changes for the adoption path. (sources) (@androndo in [#2859], @myasnikovdaniil in [#3270])

  • Cilium: 1.19.3 → 1.19.5. (release notes) (@lexfrei in [#2972])

  • Fixes CiliumNetworkPolicy fromNodes/toNodes when policy-default-local-cluster is on (the 1.19+ default), and a wildcard-namespace bypass for selectorless ipBlock rules.
  • Gateway API: TLS-passthrough routes no longer fail silently on gateways with mixed HTTP/HTTPS/TLS listeners, weighted backend splitting is fixed for TLSRoute passthrough, stale route parent statuses are pruned, and a controller panic on a malformed parametersRef is prevented.
  • Datapath: fixes node connectivity disruption when ClusterIP/LoadBalancer VIPs overlap node-local addresses, host-proxy packet routing to pods, a corrupted NamespaceSelector on CiliumEgressGatewayPolicy, and an endpoint-ID reuse race that removed a live endpoint's BPF state.
  • Removed: the loadBalancer.standalone Helm option.

  • KubeVirt: updated to v1.8.4. Fixes VMIs stuck in Scheduled on Kubernetes 1.36 (strict CRD numeric-format validation of VMI checksum status fields). (release notes) (@lexfrei in [#2940])

  • Also fixes a gRPC connection leak in virt-handler's GetLauncherClient that caused unbounded memory growth, socket accumulation and goroutine leaks, and closes CVE-2026-35469 via a moby/spdystream bump.
  • Adds missing metrics, recording rules and alerts for the virt components.

  • Velero: 1.17.0 → 1.18.1 (chart 11.0.0 → 12.0.3), with velero plugins moved to velero-1.18-compatible releases and kubevirt-velero-plugin v0.8.0 → v0.9.0. (release notes) (@lexfrei in [#2974])

  • Concurrent backup processing — multiple backups now run simultaneously without interfering, a direct win on multi-tenant clusters.
  • Cache volumes for data movers on restore (CSI snapshot data movement and fs-backup), so data-mover pods no longer fail on limited node ephemeral disk, and restore throughput improves.
  • Incremental backup size is now observable for data-mover backups.
  • v1.18.1 fixes list-scoping bugs that made restores, PodVolumeRestores and CSI PVC backups list across all namespaces instead of the install namespace, and migrates VolumeGroupSnapshot to v1beta2 for Kubernetes 1.34+.

  • Vertical Pod Autoscaler: 1.3.0 → 1.5.0 (chart 10.0.0 → 11.1.1). (release notes) (@lexfrei in [#2975])

  • In-place pod resizing is promoted to Beta, with fallback to eviction when an in-place update fails.
  • Deprecation: UpdateMode: Auto is deprecated upstream, as is the humanized-memory flag.
  • New metrics: failed eviction counters, update_mode labels on updater metrics, VPA name/namespace on resource-update counters; Prometheus bearer-auth support.
  • Dependencies bumped for Kubernetes 1.34; several race conditions in the recommender and updater fixed.

  • Harbor: 2.14.2 → 2.15.1 (chart 1.18.2 → 1.19.1), plus PodDisruptionBudgets for core, portal, jobservice, nginx, registry, trivy and exporter. (release notes) (@scooby87 in [#2966])

  • New: tag-deletion option for garbage collection, per-project upstream-connection limits (max_upstream_conn) with UI, and per-endpoint CA certificate support for registry endpoints.
  • Audit logging to the database can now be disabled at initialization.
  • 2.15.1 fixes proxy-cache serve-local on remote-not-found, prevents background polling from renewing session TTL, redacts redis_url_reg from GC attributes, and refreshes photon base packages for CVEs.

  • Keycloak: 26.5.2 → 26.6.3. (release notes) (@lexfrei in [#2964])

  • JWT Authorization Grant for external-to-internal token exchange using externally signed JWT assertions.
  • Federated client authentication, removing the need to manage individual client secrets in Keycloak.
  • Workflows for automating realm administrative tasks such as user and client lifecycle management.
  • Zero-downtime patch releases — rolling updates within a minor stream without service downtime.

  • LINSTOR: linstor-server 1.33.2 → 1.33.3, linstor-csi v1.10.6 → v1.11.2. (sources) (@lexfrei in [#2987])

  • Upgrade impact: linstor-csi v1.11.2 rejects ReadWriteMany volumes on a non-DRBD storage class, which is why an empty tenant worker nodeGroups[*].storageClass now falls back to the application-level replicated class. See Breaking Changes.

  • linstor-scheduler: v0.3.2 → v0.3.6 (chart 0.2.3 → 0.3.1), re-vendored to upstream with the native admission webhook. (sources) (@lexfrei in [#3201])

  • The admission webhook no longer strips unknown Pod fields — the image volume source and native-sidecar restartPolicy — which had broken KubeVirt containerDisk VMs and native sidecars.
  • The webhook now reloads its TLS certificate after rotation instead of silently failing.
  • Migration 52 deletes the old linstor-scheduler-admission Deployment so the new, differently-selected one can be created (a Deployment's spec.selector is immutable); failurePolicy: Ignore means the brief gap does not block Pod creation.

  • FoundationDB operator: v2.13.0 → v2.30.0 (chart 0.2.0 → 0.3.0). (release notes) (@lexfrei in [#2992])

  • Seventeen upstream releases of operator fixes and reconciliation improvements; the v2.30.0 release itself is a Go 1.26 toolchain and modernization pass.

  • HAMi: 2.8.1 → 2.9.0. (release notes) (@lexfrei in [#2993])

  • HAMi-DRA (NVIDIA) is ready for use; HAMi-core performance optimized and a HAMi-core mode added for Ascend devices.
  • Volcano vGPU device plugin synced to 0.19 with CDI support; new vGPU-monitor --metrics-bind-address flag and Prometheus ServiceMonitor support in the chart.
  • Resource-quota checking in the webhook, plus configurable webhook namespaceSelector/objectSelector.

  • Percona Server for MongoDB operator: 1.21.1 → 1.22.0, with the managed CR moved to crVersion: 1.22.0. (release notes) (@lexfrei in [#2968])

  • Sets the Kubernetes appProtocol: mongo on all services, so service meshes (Istio and friends) recognize MongoDB traffic and mTLS/cluster formation work without workarounds.
  • Physical and logical restore into a cluster with different replica-set names, via replsetRemapping on PerconaServerMongoDBRestore.

  • OpenBao: v2.5.0 → v2.5.1 (chart 0.25.3 → 0.25.7). (release notes) (@lexfrei in [#2991])

  • Security: closes CVE-2025-68121 (Go 1.25.7) and CVE-2026-24051 (OpenTelemetry SDK 1.40.0).
  • Fixes Auto Unseal failing when upgrading to v2.5.0 across AliCloud KMS, AWS KMS, Azure Key Vault, GCP Cloud KMS and OCI KMS.
  • PKI no longer returns 500 until reload on a cancelled cache-invalidation context; irrevocable-lease revocation now decrements the lease count.
  • Note: pre-v2.5.0 corrupt namespace identity groups are removed during unseal and must be recreated by an admin — look for deleting corrupt group in startup logs.

  • CoreDNS: chart 1.43.2 → 1.46.0 (upstream appVersion 1.12.3 → 1.13.1; the shipped image is deliberately held at v1.12.4 for kube-dns alignment). (release notes) (@lexfrei in [#3004])

  • Upstream fixes a Corefile infinite loop on unclosed braces and an import-cycle issue, normalizes panics on invalid origins, and prevents reload deadlocks.

  • csi-driver-nfs: 4.11.0 → 4.13.3. (release notes) (@lexfrei in [#2967])

  • Security: closes CVE-2025-52881 and CVE-2025-58181; CSI sidecar images upgraded.

  • OpenCost: 1.111.0 → 1.120.3 (chart 1.41.0 → 2.5.23). (sources) (@lexfrei in [#3000])

  • NATS: 2.11.8 → 2.11.10 (chart 1.3.13 → 1.3.16). (sources) (@lexfrei in [#2976])

  • ouroboros: 0.8.0 → 0.8.1. The controller rollout strategy is now rendered identically in every controller.mode, so switching a live release into external-dns mode no longer fails server-side apply with spec.strategy.rollingUpdate: Forbidden (sources) (@lexfrei in [#3323]).

  • Managed Kubernetes patch versions refreshed to v1.35.6, v1.34.9, v1.33.13 and v1.32.13; v1.30 removed from the support matrix (@lexfrei in [#3014]).

  • Grafana 11.6.15, victorialogs-datasource v0.14.1 → v0.28.0, and alerta-web 9.1.0 in the monitoring stack (@lexfrei in [#3011]).

  • Kamaji rebuilt on 26.3.6-edge with the datastore unused-deletion deadlock fix and Go stdlib/module CVEs closed (#3033, [#3146]).

  • New system packages: etcd-operator-crds (v1alpha2 CRDs, installed ahead of the controller) and securitygroup-controller (backs the new SecurityGroup API).

Major Features and Improvements

  • [kubernetes] Bootstrap tenant workers with Talos via CABPT: See Feature Highlights — Ubuntu + kubeadm worker bootstrap is replaced by Talos Linux driven by cluster-api-bootstrap-provider-talos and a talos-csr-signer sidecar in the Kamaji control-plane pod; existing clusters roll over automatically (@myasnikovdaniil in [#2931]).

  • [platform] Migrate to etcd-operator v1alpha2: See Feature Highlights — new etcd-operator.cozystack.io/v1alpha2 Membership API, Cozystack-authored operator chart at v0.5.2, snapshot/BackupClass-driven backup flow, and in-place adoption of legacy clusters via migration 50 (@androndo in [#2859]).

  • [etcd] Complete the v1alpha2 transition for in-cluster 1.5→1.6 upgrades: Keeps the legacy etcd-headless Service alive so adopted members stay resolvable, deletes the pre-1.6 operator Deployment via a pre-upgrade hook to get past the immutable selector, raises the operator's memory floor so it does not OOM before the VPA scales it, and makes the adoption migration robust in-cluster (@myasnikovdaniil in [#3270]).

  • [api] Add the SecurityGroup network policy resource (sdn.cozystack.io): See Feature Highlights — a tenant-facing, namespace-scoped firewall that attaches to managed applications by reference and projects 1:1 onto a CiliumNetworkPolicy, letting tenants manage network policy without cilium.io access (@lexfrei in [#2922]).

  • [tenant] Recursive (hierarchical) resource quotas for sub-tenants: See Feature Highlights — a sub-tenant's quota is carved out of its parent's remaining budget and can no longer exceed it; sub-tenants without a quota share the parent's pool (@kvaps in [#3086]).

  • [apps/kubernetes] Per-cluster OIDC selector for the tenant kube-apiserver (Phase 1): See Feature Highlights — spec.oidc.mode: System | CustomConfig | None plus spec.oidc.users[], with a ready-to-use oidc-kubeconfig Secret surfaced through the dashboard (@IvanHunters in [#3044]).

  • [monitoring] Per-instance OIDC selector for the Grafana instance (Phase 1): See Feature Highlights — the same flat spec.oidc.mode selector for Grafana, with app-side role reconciliation into Grafana's Main Org (@IvanHunters in [#3176]).

  • [kubernetes] Expose tenant apiserver authentication in app values: Adds controlPlane.apiServer.extraArgs, extraVolumes and extraVolumeMounts passthrough to KamajiControlPlane, so operators can enable OIDC or a structured AuthenticationConfiguration on tenant clusters by hand. Volume sources are restricted to ConfigMap and Secret, since the control-plane pod runs on the management cluster (@lexfrei in [#3123]).

  • [kubernetes] Propagate remote-accessible LINSTOR StorageClasses to tenant clusters: Infra-cluster LINSTOR StorageClasses whose allowRemoteVolumeAccess is not "false" are auto-created inside each tenant under the same name, with the class named by storageClass (default replicated) becoming the tenant default and the legacy kubevirt class kept as an alias so existing PVCs are unaffected. Node-local classes are not propagated. Delete any hand-made colliding tenant class before upgrading (@myasnikovdaniil in [#2872]).

  • [platform, ingress] Propagate the operator wildcard certificate to per-tenant termination points: See Feature Highlights — publishing.certificates.wildcardSecretName is now replicated into every tenant namespace that terminates TLS, with no extra operator input and no cross-namespace Secret read (@lexfrei in [#2990]).

  • [platform] ACME wildcard certificate on the default ingress-nginx path: Opt-in publishing.certificates.wildcard (default false) issues one *.<root-host> certificate via a DNS-01 solver for system services instead of a per-host ACME certificate, avoiding Let's Encrypt rate limits at scale (@lexfrei in [#2988]).

  • [keycloak] Optional KMS-encrypting database proxy: Column-level PII encryption at rest with a static KEK or Vault Transit; off by default (@kvaps in [#3003]). Extended with Vault Kubernetes and AppRole auth, a dedicated ServiceAccount, and a shared DEK set covering migration and HA (@sircthulhu in [#3239]).

  • [keycloak] Opt-in separate hostname and route for the admin console: ingress.adminHost serves the admin console and Administration REST API on their own hostname, and publishing.ingressNameAdmin attaches that route to a separate (e.g. private) Gateway or ingressClass. Disabled by default (@sircthulhu in [#2954]).

  • [keycloak] S3 backup for the CNPG database: Optional Barman-based S3 backups of the Keycloak database under backup.*, disabled by default (@kvaps in [#3174]).

  • [keycloak] Configurable login theme via platform branding: Select the realm login theme and ship its theme image through branding.loginTheme / branding.loginThemeImage (@lexfrei in [#3124]).

  • [keycloak] Allow overriding the database host/port/properties: db.host, db.port and db.urlProperties plus extraVolumes/extraVolumeMounts, so the database connection can be routed through an external proxy (@sircthulhu in [#2907]).

  • [keycloak] Enable the UPDATE_EMAIL required action in the cozy realm: Users can now change their own email address from the account portal (@kvaps in [#3175]).

  • [release] Immutable tags and rc→stable promotion: See Feature Highlights — a stable release is the byte-identical promotion of its release candidate, with every force-retag site removed and the cron patch-release workflow deleted (@myasnikovdaniil in [#3017]). A companion Cut Pre-release Tag workflow covers the rc side (#3253).

  • [operator] Support private OCI registry credentials for the platform source: New --platform-source-secret flag (and cozystackOperator.platformSourceSecret value) authenticates the platform package source against private OCI registries (kubernetes.io/dockerconfigjson) and private Git repositories (basic auth, bearer token, or SSH key). An empty value preserves the existing public-registry behaviour (@IvanHunters in [#2472]).

  • [apps/kubernetes] Overridable Talos image factory URL and installer repository: talos.imageFactoryURL and talos.installerRepository (both defaulting to the public factory) let air-gapped, mirrored or rate-limited environments serve the worker OS image and installer from their own infrastructure (@myasnikovdaniil in [#3244]).

  • [kubernetes] Per-node-group nodeHealthCheck overrides: Optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout, falling back to the cluster-wide nodeHealthCheck.* when unset — so a stateful group can sit at 0% while a stateless one tolerates 50%. An invalid override fails the render naming the offending group (@myasnikovdaniil in [#3053]).

  • [kamaji] Default to 2 replicas and drop the telemetry webhook: The Kamaji controller now runs two replicas with soft pod anti-affinity, and the telemetry handler and webhook entry are removed — cutting apiserver admission p99 for TenantControlPlane mutating/validating webhooks by roughly 70% on multi-tenant clusters (@mattia-eleuteri in [#2671]).

  • [kubevirt-cdi] Make cloneStrategyOverride configurable: CDI's clone strategy is now settable from values (default csi-clone), so operators on storage backends without efficient CSI cloning can select a different strategy (@mattia-eleuteri in [#2926]).

  • [cozy-lib] CA-only TLS trust-anchor helper for tenants: A new cozy-lib.tls.caCertSecret helper renders a ca.crt-only Opaque Secret surfaced through the tenant-secret API, so a tenant can verify a per-app TLS endpoint without being granted read access to any object that also carries a private key — the foundation the per-app TLS series builds on (@lexfrei in [#2989]).

  • [dashboard] Vendor the cozystack-ui console into the monorepo: The console SPA is now built from in-tree source under packages/system/dashboard/images/console rather than a separate repository. The shipped image and digest are unchanged for cluster operators (@myasnikovdaniil in [#2963]).

Improvements (minor)

  • [api] Return typed apierrors from TenantNamespace registry helpers: The TenantNamespace API now returns 401 Unauthorized for requests without a user identity and a structured 500 InternalError for RBAC lookup failures, instead of a generic 500 (@lexfrei in [#2870]).

  • [platform] Factor the cozystack-version stamp into a shared migration helper: Every migration now stamps cozystack-version through one labeled helper, so the platform.cozystack.io/no-delete label can no longer be silently dropped by a label-less apply — which would have removed the ConfigMap from the deletion-protection policy (@lexfrei in [#2980], @myasnikovdaniil in [#3118]).

  • [dashboard] Show the LoadBalancer external IP on the app Services tab: Externally published services now display their assigned IP in the console instead of leaving the operator to look it up with kubectl (@scooby87 in [#3172]).

  • [dashboard] Render error and unknown-type states instead of an infinite spinner: The console now shows a not-found message on a failed application GET and an unknown-type message for an unrecognized resource (@myasnikovdaniil in [#3098]).

  • [objectstorage-controller] Converge BucketClaim readiness and speed up COSI failover: BucketClaim readiness now converges to the backend Bucket, and single-replica COSI provisioner failover is faster (@lexfrei in [#3034]).

  • [etcd] Pin tenant etcd to v3.5.31: Restores consistent watches by letting kube-apiserver enable RequestWatchProgress on Kubernetes 1.35 tenants (@lexfrei in [#3122]).

Bug fixes

  • [seaweedfs] Close the 4.31 rename fallout on the 1.5.x→1.6 upgrade path: The 1.6 upgrade no longer renames a SeaweedFS instance away from its data. The naming guard now runs in the chart a platform upgrade actually re-renders and refuses when both pre- and post-4.31 naming generations exist; hack/seaweedfs-naming-audit.sh and docs/operations/seaweedfs-431-rename-recovery.md guide recovery, and the refusal is expected for tenants that passed through 1.5.x. Cluster-scoped COSI RBAC is named per namespace again (the 4.31 release-based names collided across tenants), and the seaweedfs-db hand-over now runs for every instance name — previously an instance not named seaweedfs had its filer metadata database pruned on upgrade. New migration 53 repairs clusters that already ran the old hand-over (@myasnikovdaniil in [#3339]).

  • [seaweedfs] Adopt legacy workloads in place across the 4.31 rename: Pins fullnameOverride: seaweedfs so upgrading past the vendored 4.31 chart bump adopts the existing workloads and volumes instead of standing up a second, empty cluster beside them (@myasnikovdaniil in [#3282]).

  • [seaweedfs] Point S3 consumers at the reachable S3 service: The S3 ingress backend, the iceberg ingress backend and the in-cluster COSI provisioner endpoint all resolved a stale name or the wrong port, so the external S3 endpoint returned 503 through ingress-nginx and in-cluster COSI paths hit port 443 or the filer service. All consumers now resolve seaweedfs-s3 on port 8333 (shipped in v1.5.2) (@lexfrei in [#3119]).

  • [seaweedfs] Configure the postgres2 connection pool for the filer: Removes roughly 2s of per-request S3 latency caused by PostgreSQL connection churn (shipped in v1.5.3) (@mattia-eleuteri in [#2906]).

  • [seaweedfs] Raise master probe timeouts above the cluster-status backoff: A master with a transiently empty raft view is no longer killed into a permanent CrashLoop, which could time out the whole SeaweedFS install (@lexfrei in [#3068]).

  • [seaweedfs] Soften master anti-affinity for small clusters: The hard master anti-affinity becomes a preference, so the S3 stack installs on clusters with fewer than three nodes (@lexfrei in [#3197]).

  • [kubernetes] Make the default md0 node group removable: The built-in md0 node group is applied only when no node groups are configured, so it can finally be removed or replaced; migration 47 pins md0 explicitly on existing clusters to preserve their topology (@myasnikovdaniil in [#2936]).

  • [apps/kubernetes] Honour nodeGroup minReplicas in MachineDeployment.spec.replicas: The chart stops managing spec.replicas — the cluster-autoscaler owns it, seeded by CAPI's defaulting webhook from the min-size annotation and preserved through SSA field-manager ownership. helm upgrade no longer drains workers back to a hardcoded replicas: 2 on every platform bump (@IvanHunters in [#3233]).

  • [kubernetes] Run the Talos reconcile Job in the main install phase: Creating the worker TalosConfigTemplate from a main-phase Job instead of a post-install hook makes worker MachineDeployments scale reliably during install (@lexfrei in [#3145]).

  • [kubernetes] Default worker disk storageClass to the application storageClass: A node group leaving storageClass empty now falls back to the application-level (DRBD replicated) class rather than the management-cluster default, so live migration of worker VMs gets the ReadWriteMany volume it requires (@lexfrei in [#3129]).

  • [kubernetes] Make node group resources optional when instanceType is set: A node group sized purely by instanceType no longer fails schema validation with "missing property 'resources'"; the unsupported combination of instanceType plus explicit resources is rejected at render time instead (shipped in v1.5.2) (@lexfrei in [#3121]).

  • [kubernetes] Clear the Kamaji datastore-secret finalizer to unblock namespace deletion: The leftover finalizer.kamaji.clastix.io/datastore-secret on the datastore-config Secret is now cleared during tenant teardown, so deleting a Kubernetes app or tenant no longer leaves the namespace stuck Terminating (@scooby87 in [#3078]).

  • [kubernetes] Gate the vmop cert-manager-crds dependency on certManager.enabled: Clusters running monitoringAgents.enabled=true with certManager.enabled=false had victoria-metrics-operator permanently blocked on a HelmRelease that would never exist, cascading to monitoring-agents and vertical-pod-autoscaler never becoming ready (shipped in v1.5.2) (@myasnikovdaniil in [#3130]).

  • [apps/kubernetes] Bound the pre-delete hook's tenant HelmRelease wait: Tenant Kubernetes teardown no longer hangs when the cluster has no working nodes — the pre-delete hook bounds its wait for the in-tenant HelmReleases and force-clears their Flux finalizers on timeout (@myasnikovdaniil in [#2826]).

  • [kamaji] Backport the datastore unused-deletion deadlock fix: Deleting a Tenant whose etcd DataStore had no more referencing TenantControlPlanes left the DataStore stuck Terminating, hanging the etcd Helm uninstall and wedging the tenant namespace. Backports clastix/kamaji#1122 as a source patch on the pinned build (shipped in v1.5.2) (@myasnikovdaniil in [#3146]).

  • [tenant] Inherit the full ancestor label chain from the parent namespace: Tenant namespaces now carry the complete tenant.cozystack.io/<ancestor> chain (including tenant-root) at any nesting depth, restoring root-ingress reachability to nested tenants; migration 49 backfills the labels on existing clusters (@IvanHunters in [#2912]).

  • [tenant] Grant the watch verb to the cleanup Role: Tenant deletion no longer hangs on the pre-delete hook, which needed watch to wait; the hook also moves to a maintained kubectl image (@lexfrei in [#2923]).

  • [tenant] Gate VM-resource HelmReleases on victoria-metrics-operator readiness: The tenant etcd, ingress and monitoring releases create VictoriaMetrics-operator CRs behind a failurePolicy: Fail webhook; on cold installs they could reconcile before the webhook was serving and fail with "connection refused". They now dependsOn the operator (shipped in v1.5.2) (@lexfrei in [#3141]).

  • [monitoring] Surface missing VictoriaLogs and gate the dead-end vlinsert ExternalName: A missing platform VictoriaLogs is now reported as not-ready instead of being masked, and the dangling vlinsert-generic ExternalName no longer silently drops logs. Adds a monitoring.rootEnabled platform value that operators must set to match their actual Tenant/root spec.monitoring (@scooby87 in [#3215]).

  • [mariadb] Re-guard replication so single-replica passes the operator webhook: Replication is enabled only for replicas > 1, matching the mariadb-operator validating webhook, so single-replica instances can be created again; the dashboard, RBAC and backups now target the bare service for single-replica setups (shipped in v1.5.2) (@scooby87 in [#3180]).

  • [mariadb] Derive my.cnf sizing from the instance's own resources: my.cnf is now sized from the instance's own CPU, memory and volume instead of hardcoded values assuming a large dedicated host — small presets previously advertised max_connections=4096 they could not serve; the limit now scales with memory from the server default of 151 (@lexfrei in [#3345]).

  • [mariadb] Widen the startup probe budget so bootstrap cannot be killed: A slow first boot can no longer leave the datadir permanently unusable (@lexfrei in [#3344]).

  • [mariadb] Bind the metrics exporter to IPv4: The exporter's readiness probe is reachable again and the metrics endpoint populates reliably (@lexfrei in [#3064]).

  • [kafka] Set entity-operator resources to avoid OOM under the tenant LimitRange: The topic- and user-operator containers carried no requests/limits, so the 128Mi tenant LimitRange default OOMKilled them on startup — leaving KafkaTopic/KafkaUser objects unreconciled and topics stuck on the topic-operator finalizer during teardown (shipped in v1.5.2) (@myasnikovdaniil in [#2934]).

  • [harbor] Preserve the jobservice and trivy storageClass on upgrade: The wrapper never forwarded storageClass to the jobservice jobLog PVC, so it picked up the cluster default via admission — and because PVC storageClassName is immutable, that produced rendered-vs-live drift on every upgrade. Both now preserve an existing PVC's class via lookup (shipped in v1.5.2) (@myasnikovdaniil in [#2930]).

  • [postgres] Purge a stale recovery cluster on repeat in-place restore: A repeat in-place restore no longer silently succeeds without restoring — a recovery cluster left over from an earlier completed restore is purged so the restore re-bootstraps from the backup (@IvanHunters in [#3318]).

  • [keycloak] Point HTTPRoute backends at the service port: Fixes Gateway API routing to Keycloak, which targeted the wrong port (@lllamnyp in [#3235]).

  • [kubevirt-instancetypes] Restore persistent EFI/TPM state: v1.5.1 stripped persistent EFI/TPM from the windows.11, windows.2k22 and windows.2k25 preferences because the RWO backend-storage PVC pinned VMs to a node and blocked live migration. RWO-Filesystem backend storage already live-migrates on KubeVirt v1.4+, so persistence is restored (dropped in v1.5.1 via [#3006], restored in v1.5.3) (@kvaps in [#3154]).

  • [gateway] Pin identical allowedRoutes.kinds on port 443 listeners: All port-443 listeners now carry identical allowedRoutes.kinds=[HTTPRoute,TLSRoute], preventing Cilium (cilium#45559) from collapsing the HTTPS-terminate and TLS-passthrough listeners and silently dropping HTTPRoutes (@shreyaabaranwal in [#3084]).

  • [platform] Route OIDC backend calls through the in-cluster Keycloak by default: authentication.oidc.keycloakInternalUrl now defaults to the in-cluster Keycloak Service, so enabling OIDC on a fresh install no longer leaves the dashboard down while the root ingress is not yet serving (@lexfrei in [#3289]).

  • [platform] Order backupstrategy-controller after cozystack-basics: On a fresh install the cozy-backups Bucket could race tenant-root namespace creation and fail, which then raced the install-wait deadline and failed unrelated fresh installs (shipped in v1.5.2) (@lexfrei in [#3037]).

  • [platform] Make *-rd sources depend on cozystack-engine: Fixes an install-ordering race where resource-definition sources reconciled before the engine that serves them (@lexfrei in [#3041]).

  • [cozystack-basics] Validate legacy Ingress hostnames against the tenant apex: Legacy Ingress hostnames in tenant namespaces are now constrained to the tenant's apex domain, closing a cross-tenant hostname-claim gap (@lexfrei in [#3200]).

  • [operator] Force source-watcher drift on a stuck ArtifactGenerator status: cozystack-platform could time out at 15m on fresh install when a PackageSource's derived ArtifactGenerator stalled in Ready: Unknown despite its artifacts being produced — an upstream patch.Helper race (fluxcd/pkg#934) that loses the Ready-condition write under etcd load. The reconciler now detects the stuck signature and forces source-watcher onto its drifted-branch reconcile (@IvanHunters in [#3182]).

  • [fluxcd] Omit empty distribution.artifact in FluxInstance for guest clusters: Guest Kubernetes clusters with the fluxcd addon enabled no longer fail to install; an empty distribution artifact is omitted so the operator uses its embedded manifests instead of rendering an invalid null (@IvanHunters in [#3284]).

  • [flux] Tune source-watcher concurrency and CPU: Prevents the platform install from timing out under load (@lexfrei in [#3163]).

  • [vertical-pod-autoscaler] Migrate admission-controller TLS to cert-manager: The in-chart genCA/genSignedCert templates are not idempotent, so every Flux reconcile minted a fresh certificate, flipped a checksum annotation and materialised a new ReplicaSet — producing 10+ ReplicaSets in minutes under load and livelocking the admission controller on its own election lease. A namespace-scoped cert-manager Issuer and Certificate replace them, with pod rotation keyed off the actual tls.crt content (@IvanHunters in [#3178]).

  • [cert-manager] Raise the cainjector memory limit: CA injection into webhook caBundles no longer fails under memory pressure, which previously surfaced as intermittent x509: certificate signed by unknown authority admission errors (@IvanHunters in [#3199]).

  • [cert-manager] Move the webhook off the kubelet's port: The admission webhook no longer listens on port 10250, which the kubelet also serves — a connection misrouted to a node answered with the kubelet's certificate, failing every cert-manager admission call cluster-wide with a misleading x509 error (@lexfrei in [#3359]).

  • [external-secrets] Move the webhook off the kubelet's port: Same class of collision; the Service also targets the port by name so the move rolls out without an admission gap (@lexfrei in [#3360]).

  • [metrics-server] Move the listener off the kubelet's port: The default made hostNetwork mode unable to start at all; the new port also keeps node-port collisions out of the metrics path (@lexfrei in [#3361]).

  • [cilium] Pass IPv6 to the kernel stack when the IPv6 datapath is disabled: Node IPv6 (ICMPv6 Neighbor Discovery, BGP unnumbered over link-local) now works with the host firewall enabled — Cilium host policies apply to IPv4 only and node IPv6 is not filtered (@lexfrei in [#2871]).

  • [kube-ovn] Raise the ovs-ovn CPU limit and request: The co-located ovn-controller, ovsdb-server and ovs-vswitchd daemons get CPU headroom (limit 4 cores, request 100m) under burst installs (@lexfrei in [#3050]).

  • [linstor] Set controller resource requests and relax the liveness probe: The linstor-controller no longer crash-loops under node CPU contention, which previously left its Service with no endpoints and blocked RWX volume provisioning (@lexfrei in [#3043]).

  • [opensearch-operator] Gate leader election on replica count: Leader election is disabled on the single-replica operator Deployment, so a transient apiserver or etcd latency spike no longer crashloops the manager at install time; it re-enables automatically past one replica (@lexfrei in [#3040]).

  • [monitoring-agents] Relax fluent-bit probes for loaded installs: The DaemonSet no longer crash-loops on a loaded install when its :2020 HTTP server binds late behind a busy kube-API (@lexfrei in [#3045]).

  • [velero] Add a startupProbe so slow startup does not crashloop the install gate: Velero's slow first start no longer trips the liveness probe and fail the platform install (@lexfrei in [#3138]).

  • [kubevirt-cdi] Order behind cozystack-basics: The cdi-clone-dv RoleBinding now finds cozy-public, fixing a cold-install ordering failure (@lexfrei in [#3137]).

  • [vm-default-images] Refresh stale Alpine and openSUSE 16.0 image URLs: Alpine moves 3.21.6 → 3.21.7 and the openSUSE Leap 16.0 URL is corrected, so default-image imports stop 404ing (@myasnikovdaniil in [#3063]).

  • [apps/vpn] Remove the invalid foo field from the urls Secret: A stray field broke Helm upgrades under server-side apply (@IvanHunters in [#3281]).

  • [dashboard] Unbreak CORS on an expired session for Kubernetes API calls: The dashboard SPA no longer fails with CORS errors after the kc-access cookie expires — oauth2-proxy returns 401 (not 302) for /api and /apis, and the KeycloakClient declares webOrigins: ["+"] (@IvanHunters in [#2788]).

  • [dashboard] Join pod phase on (namespace, pod) instead of uid: Fixes the Capacity Planning and Namespaces dashboards failing with "duplicate time series" errors when static pods are present (@myasnikovdaniil in [#2782]).

  • [cozy-lib, harbor, mariadb] Digest-pin the cleanup-hook kubectl image and route it through cozy-lib.images-registry: The post-delete cleanup hook image is now digest-pinned and resolved through the cluster images registry via a new cozy-lib.image helper, so mirrored and air-gapped installs can resolve it and uninstall no longer depends on a moving Docker Hub tag (@scooby87 in [#3171]).

  • [migrations] Derive the etcd-adoption snapshot target from the projected bucket credentials: Migration 50 resolves its S3 endpoint and scheme from the claimed bucket's projected Secret rather than requiring them hand-set, and makes the etcdAdoptSkipBackup escape hatch actually reachable from the Package CR (@myasnikovdaniil in [#3335]).

Security

  • [talos] Bump Talos to v1.13.6 (CVE-2026-53359): Ships Linux 6.18.38 with the kernel fixes for the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes (@lexfrei in [#3240]).

  • [kubernetes] Bump tenant worker Talos to v1.13.6: The same kernel fixes reach tenant worker VMs — the KVM shadow-paging use-after-free and nested-guest escape into the worker VM. Nested virtualization remains available; applying this rolls each tenant worker pool once (@lexfrei in [#3269]).

  • [seaweedfs] Enforce read-only bucket access by bumping cosi-driver to v0.3.1: The bundled COSI driver hardcoded read-write S3 permissions for every BucketAccess and ignored the -readonly BucketAccessClass, so credentials meant to be read-only could still upload and delete objects. The driver now honours accessPolicy: readonly and issues Read/List-only credentials (shipped in v1.5.2) (@lexfrei in [#3136]).

  • [platform] Raise the Go toolchain to 1.26 across first-party images: Clears standard-library security advisories in every Cozystack-built binary (@lexfrei in [#2941]).

  • [deps] Bump golang.org/x/crypto to v0.52.0: Closes 13 known OSV advisories (GO-2026-5005 through GO-2026-5033) (@myasnikovdaniil in [#3019]).

  • [kamaji] Close Go stdlib and module CVEs in the shipped binary: Rebuilds the pinned Kamaji binary on a patched toolchain and module set (@lexfrei in [#3033]).

  • [build] Refresh first-party Alpine runtime bases to 3.24: Clears the musl and zlib advisories across Cozystack-built images (@lexfrei in [#3009]).

  • [kilo] Rebuild the kilo image on current upstream and Alpine 3.24: Clears the libpng advisories (@lexfrei in [#3007]).

  • [ci] Harden GitHub Actions workflows for OpenSSF Scorecard: Pins every action by commit SHA, scopes per-job GITHUB_TOKEN permissions to least privilege, and adds a zizmor gate (pre-commit plus PR workflow) that blocks future regressions (@myasnikovdaniil in [#3223], [#2849], @kvaps in [#3227]).

  • [ci] Enable Renovate digest pinning and OSV vulnerability alerts: Base images and GitHub Actions are pinned to digests, indirect Go dependencies are bumped, and OSV alerts are enabled — improving the Scorecard Pinned-Dependencies and Vulnerabilities scores (@myasnikovdaniil in [#3025], [#3026]).

Dependencies & version updates

See Platform Components above for the components with user-visible upstream changes. The remaining version work in this release:

  • [linstor] Bump linstor-server to 1.33.3 and linstor-csi to v1.11.2: Carries the RWX-on-non-DRBD rejection that drives the worker storageClass fallback (see Breaking Changes) (@lexfrei in [#2987]).
  • [images] Bump opencost, multus and clickhouse-backup: OpenCost to 2.5.23, multus-cni to v4.3.0 and clickhouse-backup to 2.7.2 (@lexfrei in [#3000]), with clickhouse-backup later refreshed to v2.7.4 (@app/renovate in [#3226]).
  • [images] Refresh stale utility image references: kube-rbac-proxy v0.22.0, matchbox v0.11.0 and alpine/k8s 1.36.2 (@lexfrei in [#2969]).
  • [kubernetes] Re-pin the deleted centos:stream9 digest for kubevirt-csi-driver: Upstream deleted the pinned base-image digest, breaking the kubevirt-csi-driver build; re-pinned twice as the upstream tag moved again (@IvanHunters in [#3303], @androndo in [#3386]).
  • [deps] Renovate updates: kubevirt/csi-driver digest to 27b52aa (#3066), actions/checkout to v6.0.3 (#3133), and a batch of container image and action digests (#3232) (@app/renovate).

System Configuration

A sweep of platform components that previously ran without resource requests or limits — and therefore inherited namespace LimitRange defaults or ran unbounded — now declare them explicitly:

  • [reloader] Set resources requests/limits on the reloader container (@matthieu-robin in [#2631]).
  • [metrics-server] Set resources requests/limits on the metrics-server container (@matthieu-robin in [#2630]).
  • [external-dns] Set resources requests/limits on the external-dns container (@matthieu-robin in [#2629]).
  • [bootbox] Add resources requests/limits to the matchbox container (@matthieu-robin in [#2625]).
  • [bucket] Add resources requests/limits to the s3manager container (@matthieu-robin in [#2624]).
  • [goldpinger] Set resources requests/limits on the goldpinger DaemonSet (@matthieu-robin in [#2623]).
  • [cert-manager] Set resources requests/limits on cert-manager pods (@matthieu-robin in [#2616]).
  • [kubernetes] Add CPU/memory limits to the csi-resizer sidecar (@matthieu-robin in [#2614]).

Development, Testing, and CI/CD

  • [e2e] Migrate the app suite from BATS to Chainsaw and wire it into CI: The managed-application end-to-end suite moves to Chainsaw, giving declarative, per-resource assertions in place of shell scripting (@myasnikovdaniil in [#2826]).

  • [e2e] Diff-driven install selector select-install.sh: Computes the minimal package install set for a diff-scoped E2E run by walking the forward PackageSource dependency closure, with a --validate graph-consistency mode (@myasnikovdaniil in [#3279]).

  • [ci] Require an API-owner review for sizeable API changes: A PR that introduces a new API group, a new resource, or a breaking change to an existing API schema now requires a review from an API owner. The detector discovers CRDs and APIService registrations by content rather than directory name, and builds from the base-branch tip when the merge base predates the gate (@lllamnyp in [#3167], [#3184], @lexfrei in [#3193]).

  • [ci] Guard migrations.targetVersion against a silent skip: A CI check now catches a migration added without bumping targetVersion, which would otherwise ship a migration that never runs (@IvanHunters in [#2771]).

  • [build] Parallel matrix image builds on ephemeral runners: Image builds fan out across an ephemeral-runner matrix instead of serializing on one host (@myasnikovdaniil in [#2983]).

  • [ci] Move remaining workflows off the persistent self-hosted runner: All CI jobs now run on ephemeral or GitHub-hosted runners; release tag builds warm-start from the shared registry build cache (@kvaps in [#3268]).

  • [ci] Overlay current-main images for packages a PR did not rebuild: PR e2e and the installer artifact now run current-main images for untouched packages instead of the last release's, so a PR is tested as current-main plus its own changes (@myasnikovdaniil in [#3148]).

  • [ci] Move the install gate to a 32 vCPU runner: Ends the CPU oversubscription that produced most of the install-phase flakes (@lexfrei in [#3251]).

  • [e2e] Stop the install pipeline flaking: A combined pass over VPA OOM/churn and wait budgets across the install pipeline (@kvaps in [#2957]).

  • [e2e] Make the in-sandbox Talos image cache reachable by tenant workers: Fixes deterministic kubernetes-* E2E failures where worker CDI importers could not reach the sandbox image cache (@lexfrei in [#3254]).

  • [e2e] Escalate the cilium leak healer to an agent restart on leaked-IP reuse: The only reliable recovery from a Cilium endpoint-manager IP leak under pod churn (@lexfrei in [#3051]), with the reserved:ingress IP variant skipped (#3142).

  • [testing] Refresh the e2e-sandbox tools and base: Ubuntu 24.04, kubectl 1.33.13, talosctl 1.13.5, Helm 3.21.2 (@lexfrei in [#3010]).

  • [tests] Make bats assertions and hooks actually run under cozytest: Negative assertions now fail as intended and dead setup/teardown hooks execute, closing a class of silently-passing tests (@lexfrei in [#3132], [#3134], [#3131]).

  • Additional E2E and CI work: capture host→pod CNI data-plane on failure (#3047), capture LINSTOR satellite ErrorReports in cozyreport (#3209), wait for LINSTOR pool free capacity between tenant Kubernetes tests (#3210), gate the LINSTOR node-list probe on a ready controller endpoint (#3250), drain tenant VMs/PVCs between kubernetes tests (#3159), widen tenant-root monitoring bring-up waits (#3160), widen etcd convergence and FoundationDB status budgets (#3036, [#3039]), raise the kafka/zookeeper preset off the 128Mi OOM threshold (#3052), bump the sandbox tenant storage quota to 200Gi (#3206), make the hairpin DNS check tolerate a tenant node recycle (#3241), size the e2e tenant apiserver to the chart default (#3351), fix the etcd metrics-probe attach race (#3350), drop the serviceexposure Chainsaw suite left after the feature removal (#3336), drop duplicate package test targets (#3352), route PR base-image pulls through mirror.gcr.io (#3042), remove the redundant Release E2E workflow (#2978), prune stale and redundant workflows (#2982), fix the backport workflow's conflict_resolution nesting (#3155), drop the pr-labeler edited trigger (#3211), map dashboard/console and cert-manager/external-secrets/metrics-server scopes in the labeler (#3117, [#3362]), add needs-triage on new issues and new area labels (#3165), force chore(deps) commits and an area/dependencies label from Renovate (#3015), resolve the nightly packages digest without image-specific skopeo operations (#3332), mirror and retag images whose digest is embedded in the values tag or whose registry host lives outside repository — ten images across eight packages were being silently skipped, so nightly published references that 404 at install and promoted releases did not carry the :<version> tags they claimed (#3375), pin the console image builder to pnpm 11.1.2 (#3099), and block hard-wrapped prose in markdown and GitHub bodies (#3297).

Documentation

  • [docs/security] Add a design-level threat model: New docs/security/threat-model.md describing trust boundaries and attacker models for the platform (@lexfrei in [#3217]).

  • [docs/security] Add a security self-assessment: New docs/security/self-assessment.md prepared for the CNCF incubation due diligence (@tym83 in [#3224]), alongside an alignment pass on the existing security docs and a new incident-response document (#3225).

  • [docs] Route bugs, features, and proposals to the right repository: Issue templates and contribution docs now direct reporters to the correct Cozystack repository (@myasnikovdaniil in [#2854]).

  • [vm-instance] Document attaching a VM to an external VLAN: New docs/vm-external-vlan.md covering the Linux-bridge and bridge-type NetworkAttachmentDefinition path (@lexfrei in [#3196]).

  • [harbor] Fix the SeaweedFS discovery key described as an annotation: It is the namespace.cozystack.io/seaweedfs namespace label (@lexfrei in [#3293]).

  • [readme] Add the OpenSSF Scorecard badge (@tym83 in [#3237]).

  • [docs/agents] Add a downstream repository checklist to the PR template (@lexfrei in [#3298]).

  • [docs/release] Publish the patch-release changelogs: The generated changelogs for v1.4.5, v1.5.1 and v1.5.2 land in docs/changelogs/ so the release history on main is complete (@app/cozystack-ci in [#3083], [#3024], [#3162]).

  • [website] Multi-language proof-of-concept (ru, de, zh-cn, hi): Wires four additional languages into the site — localized landing page, getting-started guide, support page and navigation — with key-parity and translation-freshness lint, non-ratified localizations excluded from search indexing, and SEO blocks per language (@tym83 in cozystack/website#593).

  • [website] Tenant network isolation post (Cilium eBPF + Kube-OVN VPC): Explains how Cozystack isolates tenant traffic across the two data planes (@tym83 in cozystack/website#614, cover image in cozystack/website#620).

  • [website] CVE-2026-53359 (Januscape) advisory and Talos fix guide: A security advisory for the KVM guest-to-host escape plus a step-by-step Talos remediation guide (@tym83 in cozystack/website#603, cozystack/website#604, cozystack/website#612).

  • [website] CVE-2026-43499 (GhostLock) exposure assessment: Documents whether and how Cozystack clusters are exposed (@tym83 in cozystack/website#613).

  • [website] Call for independent Cozystack adopters for the CNCF Incubation review (@tym83 in cozystack/website#624).

  • [website] Simplify the backup guides around the default BackupClass: Corrects the admin override path and the platform-package backupStorage documentation for both next and v1.5 (@androndo in cozystack/website#622).

  • [website] Document host firewall and node IPv6 behavior: Explains that Cilium host policies apply to IPv4 only and node IPv6 is passed to the kernel stack (@lexfrei in cozystack/website#574).

  • [website] Document the existingSecret certificate mode and wildcardSecretName: Corrects the publishing reference and documents the third certificate mode (@lexfrei in cozystack/website#615, cozystack/website#619).

  • [website] Drop the manual KubeVirt patch step for GPU passthrough: The platform now auto-wires permittedHostDevices, so the manual patch documented since v1.4 is obsolete; includes pre-upgrade migration steps for hand-edited entries (@lexfrei in cozystack/website#556).

  • [website] Explain why packages live in apps vs extra (@kvaps in cozystack/website#594).

  • [website] cozyvalues-gen in the developer guide; sync the external apps guide with the example repo (@myasnikovdaniil in cozystack/website#610).

  • [website] talm: encrypted user values and dry-run secret redaction (@lexfrei in cozystack/website#590).

  • [website] Refresh the telemetry snapshot to June 2026 and fix per-period tenants (@tym83 in cozystack/website#598).

The following documentation shipped in the v1.5.x patch releases and is included for completeness:

Other Repositories

talm v0.33.0 / v0.33.1

  • [talm] Detect vendored chart and preset-template drift: talm now warns when the vendored charts in a project drift from the talm binary's embedded library, and detects preset-template drift via a new .talm-preset.lock file (commit it). talm init --update becomes an exact re-sync that names the drifted paths, and strictCharts: true / --strict-charts escalates warnings to hard failures — blocking on unverifiable or missing drift baselines rather than passing silently. CRLF line endings are treated as checkout artifacts, not drift (@lexfrei in cozystack/talm#216).
  • [talm] Disable and then restore KVM nested virtualization in the Talos presets: v0.33.0 disabled nested virtualization in the cozystack preset as an immediate mitigation for CVE-2026-53359, pinning grubUseUKICmdline=false so the kernel args actually apply on Talos v1.12+. v0.33.1 reverts that mitigation now that the Talos v1.13.6 kernel carries the upstream fix — so nested virtualization is available again on a patched fleet (@lexfrei in cozystack/talm#224, cozystack/talm#225).

talm v0.32.0 (shipped in v1.5.2)

  • [talm] First-class encrypted user values, honored at template and apply: A value file named *.encrypted.yaml referenced from templateOptions.valueFiles is age-decrypted in memory at both talm template and talm apply, with talm init --encrypt/--decrypt covering values-secret.yamlvalues-secret.encrypted.yaml. Secret-bearing fields are redacted by default from rendered node files, stdout and the apply drift preview, with --show-secrets/--show-secrets-in-drift to reveal them (@lexfrei in cozystack/talm#222).

ansible-cozystack v1.5.1 / v1.5.2 / v1.5.3

Contributors

We'd like to thank all contributors who made this release possible:

New Contributors

We're excited to welcome our first-time contributor:

Full Changelog: https://github.com/cozystack/cozystack/compare/v1.5.0...v1.6.0

Download cozystack

Source: README.md, updated 2026-07-22