#1 Fill Operations Feature

[Anonymous]

Originally created by: hawkfish
Originally owned by: SouravRoy-ETL

A common pair of data prep operations are fill down and fill up. Coming in DuckDB 2.0, you can write window functions in extensions, which would let you add these operations in an extension. A simple version of fill down is already included in the demo extension code (it could use a default argument.)

[Anonymous]

• assigned_to: SouravRoy-ETL

[Anonymous]

Originally posted by: SouravRoy-ETL

Working on it!

[Anonymous]

Ticket changed by: SouravRoy-ETL

• status: open --> closed

[Anonymous]

Originally posted by: SouravRoy-ETL

Done, shipped both directions today.

xf.fill_forward already existed, so I just added the sibling xf.fill_backward next to it - together they cover the fill down / fill up pair you mentioned.

I went with a plain window function over the system DuckDB CLI rather than the 2.0 extension route you suggested:

SELECT * REPLACE (
  first_value(col IGNORE NULLS) OVER (
    PARTITION BY ... ORDER BY ts
    ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
  ) AS col
) FROM upstream

Reasoning: works on every DuckDB version Duckle currently ships against, single pass, no extra LOAD step on the user's machine, no extension binary to package. The 2.0 window-function extension is on my radar though - if it ends up materially faster in practice I can swap the implementation behind the same component schema without anything breaking.

Both live in the Rows group of the Transforms palette. Required props: column, orderBy. Optional: partitionBy (fills each group independently). Round-trip test against real DuckDB at crates/duckdb-engine/tests/execution.rs::fill_backward_propagates_next_non_null.

Goes out with the next release. Appreciate the nudge.

[Anonymous]

Ticket changed by: SouravRoy-ETL

• status: closed --> open

[Anonymous]

Originally posted by: SouravRoy-ETL

@all-contributors please add @hawkfish for infrastructure, tests and code

[Anonymous]

Originally posted by: allcontributors[bot]

@SouravRoy-ETL

I've put up a pull request to add @hawkfish! 🎉