#88 Add Qlik qvd files as source and destination

[Anonymous]

Originally created by: vrejany
Originally owned by: SouravRoy-ETL

The problem

Qlik Sense is widely used for analytics workflows that rely on QVD (Qlik Data) files as a core storage format. Currently, Duckle does not support reading from or writing to QVD files directly. This limitation requires converting datasets into intermediate formats such as CSV or Parquet before processing them in Duckle. These additional steps disrupt native Qlik workflows, remove QVD-specific optimizations, and introduce unnecessary overhead in data pipelines.

Proposed solution

Introduce native connectors in Duckle that support QVD files as both a data source and a destination. The connector should enable direct reading and writing of QVD binaries. This could be implemented using existing libraries (such as Python-based QVD readers/writers) or by leveraging Qlik-compatible APIs, allowing Duckle to interact with QVD files without requiring a full Qlik Sense environment on execution nodes.

Alternatives / workarounds

The current workaround involves exporting QVD data into intermediate formats (e.g., CSV or Parquet), processing the data in Duckle, and then reloading the results back into QVD files through Qlik scripts. This approach introduces redundant data transformations, increases I/O overhead, and complicates pipeline orchestration.

Additional context

Supporting QVD natively would simplify end-to-end data workflows, reduce pipeline complexity, and improve performance by eliminating unnecessary format conversions.

Python library reference for QVD: https://pypi.org/project/qvd/

[Anonymous]

• assigned_to: SouravRoy-ETL

[Anonymous]

Originally posted by: SouravRoy-ETL

Sure! Amazing addition will add it in the next release!