#82 Bulk Column renaming via external map file

[Anonymous]

Originally created by: gmacc00
Originally owned by: SouravRoy-ETL

It would be useful to provide users with the ability to perform bulk column renaming—particularly for datasets or files containing a large number of columns ( 70+ ) —by supplying a simple mapping file (e.g., JSON, YAML, CSV, etc.). Such a mapping file could be generated externally, for example by a third-party application, and then used to automatically apply the desired column name changes.

Currently, this can be achieved in one of two ways:

1. Manually configuring a series of Rename transformation steps to rename each column.
2. Writing an inline SQL query and explicitly defining column aliases for the desired column names.

Both approaches can be quite exhaustive and difficult to maintain when working with a large number of columns.

[Anonymous]

Originally posted by: SouravRoy-ETL

Nice one. I will look into this.

[Anonymous]

Originally posted by: DarekDan

Interesting idea. Thank you for that/

[Anonymous]

• assigned_to: SouravRoy-ETL

[Anonymous]

Originally posted by: SouravRoy-ETL

Shipped in v0.5.0. xf.rename now takes an external mapping file (JSON / CSV / YAML), so you can bulk-rename columns from a map instead of one at a time.

Please download - https://github.com/slothflowlabs/duckle/releases/tag/v0.5.0 and let me know!

[Anonymous]

Originally posted by: gmacc00

Created a 70-column CSV mapping file in "old,new" format and integrated it into the pipeline after the JSON source, enabling successful data loading into Oracle. This approach proved particularly useful for handling lengthy JSON field names when transferring data to legacy Oracle instances with a 32-character identifier limit.