Seldon Core Files

Overview

Core 2.9.1 is a patch release focused on bugfixes and security. We also introduce a number of important features related to cyclic pipelines, usability and cost-effectiveness:

Bug fixes

• Allow Core 2 to work reliably with pod disruption budgets (PDBs) (#6560). Previously, terminating/draining pods remained Ready: True, which meant that they were still considered non-disrupted from a PDB perspective.
• Make model-gateway request timeouts configurable (previously 10 minutes, new default 2 minutes) (#6522). This is a fix for particular cases when inference requests overload the available inference server replicas, leading to significant latency increases. Previously in such cases, a large backlog of unprocessed entries gathered in kafka model input topics, leading to new requests always timing out (enough requests ahead of them in the queue for timeout at LB-level). The new default is likely too high for most usecases, but is set conservatively so as to not prevent inferences for slow models. However, timeouts can be set matching your own workloads at the model-gateway pod level, via the MODELGATEWAY_WORKER_TIMEOUT_MS environment variable.
• Fix dataflow-engine edge cases for communicating to the scheduler (#6506). In the case of multiple dataflow-engine replicas, or a single restarting replica, the status of a Pipeline was set based on the last received message. Sometimes, due to the way locking was done, the status update from a terminating dataflow-engine replica was processed after updates from the replica started by k8s to replace it. This meant that pipelines would transition into PipelineTerminated states in a non-deterministic way.

New features

• Allow pipelines to have cycles, with a bounded number of iterations (#6413, [#6480], docs). This feature is enabled on a pipeline-by-pipeline basis via a newly added spec.allowCycles field in the CR.
• Allow the Core 2 operators to be installed within their own namespaces but manage Core 2 CRs in a list of other namespaces (#6434, docs)
• Allow end-users to delete the kafka topics associated with a model or pipeline when deleting the model/pipeline (#6353, [#6383], docs). Care needs to be taken in using this feature because it implies loosing observability with respect to historic inference requests/responses, potentially targeted at models within a pipeline.

Docs improvements

• Expanded documentation on Autoscaling and Performance Tuning
• Quickstart guide

Core 2 release images

• Core 2 images published on docker hub now have embedded SBOM attestations.
• At the moment of the release, security scans show zero CVEs within those images.

CRD Updates

All CRD changes in this release maintain backward compatibility, so clusters with existing CRs can be migrated seamlessly. Please see specific feature docs for changes.

Kudos:

We recognise the significant contributions to this release from @RobertSamoilescu

With contributions from @RobertSamoilescu, @lc525, @Rajakavitha1, @paulb-seldon

Changelog

Dates are displayed in UTC. Generated by auto-changelog.

v2.9.1

9 July 2025

• build(deps): fix k6 Dockerfile after version update #6603
• build(deps): update internal package references, k6 #6602
• build(deps): bump grafana/grafana from 11.5.2 to 12.0.2 in /scheduler #6543
• build(deps): bump ubi9/ubi-minimal from 9.5 to 9.6 in /scheduler #6596
• Update README.md #6598
• Update core-autoscaling-servers.md #6599
• fix(ci): kustomize version environment var #6581
• fix(build): scheduler Makefile for building docker images with SBOM #6580
• fix(github): Update image build action to support SBOM attestations #6579
• feat(Agent): add readiness probes #6560
• fix(operator/lint): fix deprecated golangci settings #5887
• fix(o11s): Filter gpu metrics on exported_namespace #6377
• Re-generate license info #6575
• Bump envoyproxy/envoy from v1.33.0 to v1.34.1 in /scheduler #6444
• build(deps): update Core 2 dependencies #6574
• Update README.md - fix link #6572
• Replaced time.sleep with asyncio.sleep #6569
• add quickstart demo #6475
• feat(modelgateway): Added support for custom worker timeout #6522
• Autoscaling Docs #6500
• Update concepts #6437
• Update SUMMARY.md #6539
• Revert "Update autoscaling.md (#6534)" #6536
• Update autoscaling.md #6534
• update readme initial #6523
• Update istio.md #6517
• Update faqs.md #6515
• docs: update FAQs #6511
• docs: Update faqs.md #6510
• fix: dataflow message reconcile #6506
• Updated docs with maxStepRevisits #6507
• feat: detect and stop cycles #6480
• Perf-tuning docs #6491
• docs: added page descriptions #6492
• Comment out broken links #6478
• Update Scaling Docs #6476
• Updated docs for installing the operator with watchNamspaces flag. #6455
• docs(pipelines): Docs for cyclic pipelines (counter) #6440
• docs(Kafka): Included docs for managing Kafka topics #6438
• Update 2.8 to 2.9 upgrade notes #6450
• Link to new docs across our gh repo #6448
• (docs) Update the dependencies table #6446
• docs: updating docs to support kubectl and seldon-cli #6219
• feat: operator watch namespaces #6434
• feat: custom kafka stream join window #6419
• feat: allow removal of kafka pipeline topics #6383
• docs: Update the concepts section #6417
• feat: allow removal of kafka model topics #6353
• feat: cyclic pipelines #6413
• docs added a banner #6390
• docs(pipelines): Fix "Chain" example in the "Pipeline" doc #6370
• docs: add the release notes #6384
• Update SUMMARY.md #6385
• docs fix: added a banner #6376
• feat(o11y): GPU Grafana dashboard #6374
• Generating changelog for v2.9.1 8111172
• Setting version for helm charts 2d91dee
• Setting version for yaml manifests 0cdcb04