Major Features 🔥

• Rust Dataplane is the default Dataplane (Golang will be deprecated in next release). To switch to Golang Runtime, set NUMAFLOW_RUNTIME

  - name: vtx-name
    containerTemplate:
      env:
        - name: NUMAFLOW_RUNTIME
          value: "golang"
  

Recommend SDK

• Please use 0.10 and above. Refer Compatibility Matrix
• Contextual debugging only available for MonoVertex

🚨 Breaking Change 🚨

• If you have a Reduce vertex with persistence, pipeline should be recreated. The WAL format has changed.

What's Changed

• doc: accumulator doc by @vigith in https://github.com/numaproj/numaflow/pull/2609
• feat: enable pod template support for serving deployment by @whynowy in https://github.com/numaproj/numaflow/pull/2620
• feat: exponential backoff retry strategy for sink by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2614
• fix: potential incorrect k8s minor version by @whynowy in https://github.com/numaproj/numaflow/pull/2647
• feat: Kafka source - Rust implementation by @BulkBeing in https://github.com/numaproj/numaflow/pull/2636
• feat: Aligned Window for Reduce Async Data Movement by @yhl25 in https://github.com/numaproj/numaflow/pull/2618
• feat: Rust pipeline metrics by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2666
• fix: conditional forwarding in Async Dataplane by @vigith in https://github.com/numaproj/numaflow/pull/2668
• fix: return 400 if serving ID header contains '.' by @vigith in https://github.com/numaproj/numaflow/pull/2669
• fix: retry attempts calculation and logs in retry strategy by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2653
• feat(doc): update quick start and UI docs by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2622
• feat: Kafka sink implementation in Rust by @BulkBeing in https://github.com/numaproj/numaflow/pull/2672
• feat: extension for http source by @vigith in https://github.com/numaproj/numaflow/pull/2687
• feat: optionally enable Wire compression to optimize network and ISB stability by @vigith in https://github.com/numaproj/numaflow/pull/2709
• feat: add serving http server by @cosmic-chichu in https://github.com/numaproj/numaflow/pull/2633
• feat: http source should honor response status codes sent to clients by @yhl25 in https://github.com/numaproj/numaflow/pull/2732
• feat: insert id into generator payload by @vigith in https://github.com/numaproj/numaflow/pull/2735
• feat: Pulsar Sink implementation by @BulkBeing in https://github.com/numaproj/numaflow/pull/2745
• docs: enhance user-defined sinks docs with message-response contract by @sapkota-aayush in https://github.com/numaproj/numaflow/pull/2747
• test(ui): partially fix UI test by @tmenjo in https://github.com/numaproj/numaflow/pull/2726
• fix(UI): pending and ackPending buffer info on Edges by @yogesh1801 in https://github.com/numaproj/numaflow/pull/2746
• feat: Unaligned Window for Async Data Movement by @yhl25 in https://github.com/numaproj/numaflow/pull/2699
• docs: Add user guide for GPU annotation and resource requests in vertices by @sapkota-aayush in https://github.com/numaproj/numaflow/pull/2753
• fix: use kafka timestamp as event time by @yhl25 in https://github.com/numaproj/numaflow/pull/2781
• fix: jetstream source should use published timestamp by @yhl25 in https://github.com/numaproj/numaflow/pull/2778
• doc: Example PDB and Anti-Affinity config for ISB by @vigith in https://github.com/numaproj/numaflow/pull/2787
• feat: e2e tests with rust runtime by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2693
• fix: kafka writes messages to same partition when setKey is not set by @yhl25 in https://github.com/numaproj/numaflow/pull/2797
• feat: Daemon server to use http based watermark fetcher by @yhl25 in https://github.com/numaproj/numaflow/pull/2798
• fix(UI): health filter & support for mono-vertex health on pipeline listing page by @adarsh0728 in https://github.com/numaproj/numaflow/pull/2761
• doc: User documentation for Jetstream source by @BulkBeing in https://github.com/numaproj/numaflow/pull/2826
• fix: Detect tokio worker threads by @BulkBeing in https://github.com/numaproj/numaflow/pull/2831
• fix: handle stream already existing when making consumer by @saty9 in https://github.com/numaproj/numaflow/pull/2832
• fix: add serving pipeline transformer config by @cosmic-chichu in https://github.com/numaproj/numaflow/pull/2835
• feat: Filter subjects and deliver policy as user options by @BulkBeing in https://github.com/numaproj/numaflow/pull/2837
• feat: enable rust runtime by default by @yhl25 in https://github.com/numaproj/numaflow/pull/2833
• feat: unpause pipelines vertices and mvtx from min replicas by @kohlisid in https://github.com/numaproj/numaflow/pull/2840
• fix: graceful shutdown during reduce udf panic by @yhl25 in https://github.com/numaproj/numaflow/pull/2845
• fix: requeue reconciliation after seeing container restart by @whynowy in https://github.com/numaproj/numaflow/pull/2855

New Contributors

• @yogesh1801 made their first contribution in https://github.com/numaproj/numaflow/pull/2729
• @sapkota-aayush made their first contribution in https://github.com/numaproj/numaflow/pull/2744
• @saty9 made their first contribution in https://github.com/numaproj/numaflow/pull/2832

Full Changelog: https://github.com/numaproj/numaflow/compare/v1.5.0...v1.6.0