v1.6.4 (2026-09-28)

A patch release with backported fixes for VM live migration during eviction and scheduling, Kafka release deletion, KubeVirt-related billing labels, ingress client-IP trust, PostgreSQL-adjacent LINSTOR/CDI/ZFS reliability, MongoDB and RabbitMQ defaults, and a batch of e2e hardening, plus a new talm documentation site, an NVIDIA vGPU restore role for ansible-cozystack, and a large Marketplace documentation section on the website.

Fixes

• fix(cluster-api): backport live migration before host eviction: cluster-api-provider-kubevirt now attempts live migration before falling back to drain and delete when a guest node is evicted, uses the guest node's own name for the drain fallback instead of the wrong identifier, and uncordons a recreated guest — so an eviction no longer forces an unnecessary VM restart when a live migration would have sufficed (@yankawai in [#4291], backport [#4474]).

• fix(kubeovn): retry VM migration setup after scheduling: Retries the VM migration network setup when the target Pod or source/target scheduling information is not yet available, instead of failing outright, and folds the Kube-OVN image into the regular in-tree build matrix — so a migration that starts before Kube-OVN has finished scheduling both endpoints no longer aborts (@yankawai in [#4253], backport [#4328]).

• feat(kubevirt): expose migration configuration through platform values: Cluster-wide KubeVirt migration settings (bandwidth per migration, parallel migrations per cluster, parallel outbound migrations per node) can now be set through the platform value kubevirt.migrations instead of hand-patching the generated Package, which the platform undoes on every render; leaving the value unset preserves existing defaults (@yankawai in [#4254], backport [#4402]).

• fix(linstor): add opt-in graceful satellite shutdown on Talos: Adds an opt-in Talos satellite preStop hook that releases unused Secondary DRBD resources, ZFS zvols included, before the satellite stops, without forcing, demoting or calling the Kubernetes API. Enable it with talos.gracefulShutdown.enabled for sequential node maintenance; existing installations keep today's behavior by default, and the hook also runs on ordinary pod restarts so it should be enabled only where node restarts are serialized (@yankawai in [#4292], backport [#4401]).

• fix(workloadmonitor): populate VM instance-profile label from the KubeVirt preference annotation: WorkloadMonitor read a kubevirt.io/cluster-instanceprofile-name annotation that KubeVirt never writes; the real annotation is kubevirt.io/cluster-preference-name. Reading the correct key means a VM's instance profile — a Windows guest, for example — now reaches the billing metric pipeline instead of being silently dropped (@IvanHunters in [#4448], backport [#4454]).

• fix(opensearch-operator): name the component reconciler a stall sits in: The OpenSearch cluster controller now logs a watchdog report naming which of its nine component reconcilers has been running for over a minute, and keeps reporting on an interval while it stays stuck, instead of leaving a stalled reconcile with no trace in the operator log at all (@IvanHunters in [#4333], backport [#4377]).

• fix(manifestutil): report one message when the CRD wait is cancelled: A CRD wait cut short by a cancelled context now always reports one error naming the CRD it was waiting on, instead of sometimes including the name and sometimes not, depending on which of two exit paths the scheduler happened to pick (@lexfrei in [#3742], backport [#4475]).

• fix(kafka): delete release topics before the topic operator is removed: Deletes release-owned KafkaTopics with a pre-delete hook before the Strimzi topic operator is removed, and fixes the hook's own --request-timeout flag, which made it skip in-cluster credentials and fail every deletion outright. Together these mean a Kafka release with the strimzi.io/topic-operator finalizer no longer leaves stray topics behind on reinstall, and release deletion itself no longer wedges (@yankawai, @IvanHunters in [#3938], [#4280], backport [#4456]).

• fix(ingress): trust CF-Connecting-IP only from Cloudflare ranges: With cloudflareProxy enabled, CF-Connecting-IP is now trusted only from Cloudflare's published address ranges. Previously the ranges were passed under a key ingress-nginx ignores, so any client that reached the load balancer directly could set the header itself and bypass the ingress IP whitelist (@lexfrei in [#4444], backport [#4445]).

• fix(apps): let the cleanup hooks watch the objects they delete: The Harbor, ClickHouse, MariaDB and Qdrant cleanup hooks now wait for the objects they delete to actually disappear, with the wait bounded so a stuck claim cannot hang the whole uninstall (@yankawai in [#4135], backport [#4369]).

• fix(registry): preserve write options in aggregated storage: The application, tenant-secret and security-group APIs now preserve Kubernetes write options (dry-run, field manager, field validation, delete propagation, grace periods, preconditions) when delegating to controller-runtime, instead of silently dropping them; a rejected write also keeps its machine-readable status, so a quota or admission failure now answers 403/409 with a reason instead of a generic 500 (@yankawai in [#3937], backport [#4370]).

• fix(dashboard): let dashboard users read backup CRD schemas: Grants dashboard users read access to the backup CRD schemas, fixing the backup create/edit forms that previously returned 403 (@androndo in [#4236], backport [#4307]).

• fix(backups): give the barman-cloud sidecar its own resources: The barman-cloud sidecar previously had no resource requests or limits, so in tenant namespaces it inherited a 128Mi LimitRange default and was OOMKilled during backups even while the surrounding ObjectStore and Cluster stayed healthy. It now requests 100m/256Mi and is capped at 1Gi on both the chart and the platform's Go rendering path (@yankawai in [#4220], backport [#4336]).

• fix(linstor): reconnect only the stalled peer in the satellite plunger: The satellite plunger disconnected a stalled DRBD peer but then tried to reconnect every peer of the resource at once, which failed with "Device has a net-config" for peers that were never disconnected and left the stalled one permanently unreconnected. It now reconnects only the peer pair it disconnected, so a stalled resync actually recovers (@yankawai in [#4184], backport [#4319]).

• fix(foundationdb): let tenant roles read the connection ConfigMap: Tenants previously had no way to obtain their FoundationDB connection string: the cluster file lives in a ConfigMap tenant subjects could not read and the dashboard resource map did not expose. The ConfigMap is now in the dashboard resource map with a Role/RoleBinding granting the tenant use access level get/list/watch on it (@yankawai in [#4148], backport [#4315]).

• fix(opensearch): remove the data PVCs when the application is deleted: Adds a post-delete hook that reclaims an OpenSearch application's data PVCs when it is removed, instead of leaving them orphaned (@yankawai in [#4136], backport [#4368]).

• fix(nats): merge config.merge.accounts with the generated accounts map: config.merge.accounts is now merged into the generated NATS accounts map instead of being rendered as a second, conflicting copy of the same key (@yankawai in [#4134], backport [#4321]).

• fix(clickhouse): keep the Keeper name inside the StatefulSet volume-name limit: Shortens the generated Keeper name for application names longer than 15 characters, fixing the Keeper StatefulSet failing to create for those names (@yankawai in [#4133], backport [#4314]).

• fix(linstor): wait for temporary probe devices: Backports a LINSTOR upstream fix that waits for a ZFS probe device to appear before reading its I/O properties, instead of racing udev; without the wait, a pool could be left with incorrect block sizes and failed replica placement on 4K pools (@yankawai in [#4100], backport [#4335]).

• fix(mongodb): fill the dashboard credentials on a first install: The dashboard credentials Secret is now filled with the database password and connection URI on first install; previously it shipped empty until someone ran helm upgrade by hand, because the lookup it relied on could only see users the Percona operator had not yet created (@yankawai in [#4014], backport [#4330]).

• fix(rabbitmq): right-size the default resources preset to s1.nano: Raises the default resourcesPreset from t1.nano (128Mi) to s1.nano (250m CPU, 512Mi), because a default-sized RabbitMQ 4.2 broker was OOMKilled on t1.nano and never became Ready. Brokers with no stored preset are restarted on upgrade and gain a larger guaranteed memory reservation — check node headroom, and tenants near a resourceQuotas memory ceiling, before upgrading; brokers with an explicit resourcesPreset already set keep their existing size (@yankawai in [#3936], backport [#4393]).

• fix(kafka): make topics[].config optional: Kafka topics can now omit the optional Strimzi config map entirely, instead of the chart requiring one even when there is nothing to set (@yankawai in [#3935], backport [#4334]).

• fix(kubernetes): pin KubeVirt CSI sidecars to SIG Storage releases: Pins the KubeVirt CSI sidecars to versioned SIG Storage images, fixing sidecars that previously would not run on pre-x86-64-v3 CPUs (@yankawai in [#3934], backport [#4313]).

• fix(cdi): update CDI to v1.66.1: Updates CDI to v1.66.1, fixing HTTP imports that block volumes on 4Kn devices (DRBD/LINSTOR) via a repaired qemu-img, and a resourceVersion update storm on prime PVCs during WaitForFirstConsumer imports (@lexfrei in [#3920], backport [#4156]).

• feat(etcd-operator): bump to v0.5.5 and vendor EtcdDefragPolicy CRDs: Bumps etcd-operator to v0.5.5, adding the EtcdDefrag and EtcdDefragPolicy APIs for on-demand and scheduled etcd defragmentation (@androndo in [#3968], backport [#4151]).

Development, Testing, and CI/CD

• test(e2e): run the merge-gating lanes on Talos containers: Moves the merge-gating E2E lanes onto Talos containers, fixing tenant node-join failures that were happening in 8 of the last 11 runs because tenant workers sat one virtualization level too deep on the shared runners and had their CSR signed after the readiness window closed (@myasnikovdaniil in [#4020], backport [#4437]).

• chore(e2e): ignore the artifacts the sandbox bringup writes to the repo root: Adds .gitignore entries for the Talos secrets bundle, rendered machine configs, kubeconfigs, boot image and per-VM directories a local e2e sandbox bringup writes to the repo root, closing a path where a broad git add could have staged cluster private keys (@lexfrei in [#3891], backport [#4438]).

• fix(e2e): budget tenant scheduling separately from workload readiness: Waits for a tenant node to actually accept a Pod, on its own budget, before creating the workload used by the kubernetes suites, instead of sharing one 300s deadline between node scheduling and workload readiness — fixing intermittent timeouts where scheduling alone consumed most of the shared window (@lexfrei in [#3579], backport [#4435]).

• fix(e2e): budget each linstor wait separately in post-install-prep: Gives each of the two LINSTOR waits in the post-install-prep script its own deadline, started when the previous one finishes, instead of sharing a single 15-minute window anchored at the script's start — fixing failures where a slow HelmRelease left the following Deployment wait too little time (@lexfrei in [#3715], backport [#4434]).

• fix(reloader): add startupProbe to survive slow first start under load: Adds a startup probe to cozy-reloader so a slow first start is no longer killed by the liveness probe and left crash-looping, which also unblocks the LINSTOR stack that depends on it starting cleanly (@lexfrei in [#3470], backport [#4433]).

• fix(e2e): stop the sandbox cluster CIDR overlapping the pod range: Moves the e2e sandbox's Kubernetes cluster CIDR off the range Kube-OVN uses for pod IPAM, fixing intermittent Cilium/Kube-OVN IP collisions that either refused a pod outright with "IP already in use" or silently broke host-to-pod traffic (@lexfrei in [#3750], backport [#4432]).

• fix(e2e): disable guest fsync on ephemeral CI VM disks: Boots e2e QEMU guests with cache=unsafe so ephemeral CI VM fsyncs no longer stall the management-cluster etcd, removing cross-operator leader-election flakiness that was failing a different chainsaw suite on every run (@kvaps in [#3455], backport [#4431]).

Documentation

• [website] feat(compliance): surface AI Conformance page, mark submission accepted: Publishes the AI Conformance results page and marks the platform's submission as accepted (@tym83 in cozystack/website#706).

• [website] fix(docs): refresh trunk version pins to Cozystack v1.6.3: Updates the documentation's trunk version references to Cozystack v1.6.3 (@lexfrei in cozystack/website#618).

• [website] docs(platform-package): sync the publishing.* table with the platform values: Keeps the platform values reference's publishing.* table in sync with the actual platform values (@lexfrei in cozystack/website#679).

• [website] docs(networking): GatewayClass selection, edge TLS mode, and the Cloudflare Tunnel package: Documents GatewayClass selection, the edge TLS certificate mode, and the Cloudflare Tunnel Gateway API package (@lexfrei in cozystack/website#655).

• [website] docs(platform-package): document KubeVirt migration settings: Documents the new kubevirt.migrations platform value from the corresponding fix above (@yankawai in cozystack/website#697).

• [website] fix(demo): keep the screen when the demo is reloaded: Fixes the live console demo losing its current screen on a browser reload (@kvaps in cozystack/website#701).

• [website] docs(platform-package): document kubevirt.disabledFeatureGates: Documents the kubevirt.disabledFeatureGates platform value (@lexfrei in cozystack/website#678).

• [website] chore(demo): refresh the console demo from the cozystack monorepo: Refreshes the live demo's bundled cozystack-ui console from the monorepo (@kvaps in cozystack/website#698).

• [website] fix(site): point the Certified Kubernetes badge at the live landscape entry: Fixes the Certified Kubernetes badge linking to a stale landscape entry (@tym83 in cozystack/website#699).

• [website] docs(compliance): add AI Conformance page, update Kubernetes Conformance: Adds the initial AI Conformance page and updates the Kubernetes Conformance page (@tym83 in cozystack/website#693).

• [website] docs(blog): route independent-adopter replies to the project: Redirects independent-adopter reply channels on the blog to the project instead of an individual (@tym83 in cozystack/website#696).

• [website] fix(site): resolve /roadmap and move the vendor logo off a project path: Fixes the /roadmap link and moves a vendor logo off a project-owned path (@tym83 in cozystack/website#695).

• [website] feat(home): add a "where Cozystack fits" section: Adds a "where Cozystack fits" section to the home page (@tym83 in cozystack/website#694).

• [website] fix(site): give the live-demo band its own layout: Gives the live-demo band on the home page its own layout instead of sharing one with unrelated content (@tym83 in cozystack/website#692).

• [website] feat(site): verify the @cozystack.io Bluesky handle over HTTPS: Serves the Bluesky handle DID at /.well-known/atproto-did to verify the @cozystack.io Bluesky handle over HTTPS (@kvaps in cozystack/website#691).

• [website] docs(marketplace): add Application Marketplace section: Adds a large new Application Marketplace documentation section covering the CLI reference, package taps, subscriptions, trust model, and catalog behavior (@lexfrei, @IvanHunters in cozystack/website#672).

• [website] fix(site): replace vendor social accounts in the footer with project channels: Replaces vendor-owned social accounts in the site footer with the project's own channels (@tym83 in cozystack/website#690).

• [website] fix(networking): correct what disabledPackages does to ouroboros: Corrects the documented effect of disabledPackages on the ouroboros package (@lexfrei in cozystack/website#683).

• [website] docs(virtualization): add a VMware vSphere migration guide: Adds a guide for migrating VMware vSphere VMs into Cozystack, including the VDDK image build (@kvaps in cozystack/website#673).

• [website] docs(kubernetes): add OIDC authentication guide: Adds a guide for configuring OIDC authentication on managed Kubernetes clusters (@IvanHunters in cozystack/website#596).

• [website] ci(telemetry): put the monthly refresh back on a schedule: Restores the scheduled monthly refresh of the telemetry dashboard data (@tym83 in cozystack/website#645).

• [website] docs: update managed apps reference for v1.6.3: Refreshes the managed applications reference for the v1.6.3 release (@app/cozystack-ci in cozystack/website#681).

• [talm] docs: publish the manual as a documentation site: Publishes talm's manual as a documentation site at talm.cozystack.io, with a CLI reference generated from the Cobra command tree (@lexfrei in cozystack/talm#235).

• [talm] ci: unbreak the lint job under Go 1.27: Fixes the talm lint job under Go 1.27 and requires Go 1.27 for the build (@lexfrei in cozystack/talm#237).

• [talm] chore(deps): bump Talos to v1.14.0: Bumps the Talos dependency to v1.14.0, along with fixes so applycheck and meta --insecure keep working against it and a rendered config keeps every document instead of dropping some (@sircthulhu in cozystack/talm#223).

• [ansible-cozystack] feat(roles): restore NVIDIA vGPU host state across reboots: Adds a role that restores NVIDIA vGPU host state after a reboot, instead of requiring it to be reconfigured by hand (@lexfrei in cozystack/ansible-cozystack#70).

• [ansible-cozystack] chore(deps): track cozystack installer v1.6.3: Renovate dependency bump of the cozy-installer Docker image tag to follow the v1.6.3 Cozystack release (@app/renovate in cozystack/ansible-cozystack#71).

• [ansible-cozystack] chore(deps): update k3s to v1.37.0+k3s1: Renovate dependency bump of the k3s Kubernetes distribution used by the management-cluster bootstrap role (@app/renovate in cozystack/ansible-cozystack#72).

Contributors

Thanks to everyone who contributed to this patch release:

• @androndo
• @IvanHunters
• @kvaps
• @lexfrei
• @myasnikovdaniil
• @sircthulhu
• @tym83
• @yankawai

New Contributors

We're excited to welcome our first-time contributor:

• @yankawai - First contribution!

Full Changelog: https://github.com/cozystack/cozystack/compare/v1.6.3...v1.6.4