#146 Five dbt seeds still overwrite konsol's write-through tables (same relation, two writers)

[Anonymous]

Originally created by: grynn-in

Five dbt seeds write to the same ClickHouse relation that a konsol doctype
TRUNCATE+INSERTs, so dbt seed and bench migrate overwrite each other's data.

Seeds materialise into epm_gold (dbt_project/dbt_project.yml, seeds: +schema: gold),
so seeds/<name>.csv becomes epm_gold.<name> — exactly the tables these
controllers declare as CH_TABLE:

relation	konsol doctype	doctype rows (live)	seed rows
epm_gold.allocation_rules	Allocation Rule	3	3
epm_gold.consolidation_adjustments	Consolidation Adjustment	0	8
epm_gold.ic_elimination_rules	IC Elimination Rule	3	5
epm_gold.scenario_definitions	Scenario	17	4
epm_gold.spread_profiles	Spread Profile	24	24

epm_gold.consolidation_groups was the sixth and is fixed in grynn-in/konsol#116
/ grynn-in/konsolidat#145 (F2), where the collision was demonstrated: one
dbt seed --select consolidation_groups reverted a published ownership change,
deleted a whole consolidation group, and made two entities appear under two
groups at once. dbt ls --select +tag:domain:consolidation --resource-type seed
still lists allocation_rules, consolidation_adjustments and
ic_elimination_rules, so a governed consolidation build re-seeds all three.

Why this was not fixed alongside the sixth: the row counts disagree. Deleting
consolidation_adjustments.csv would empty a table that the doctype cannot
refill (0 rows, no fixture file); ic_elimination_rules would lose 2 rows and
scenario_definitions is richer in the doctype than in the CSV. Each one needs
someone to decide which side is the truth and reconcile the data before the CSV
goes — that is a data question, not a mechanical one.

Suggested fix

Per table, in the order above:

1. Decide which side is authoritative; reconcile the rows into the doctype
   (fixtures where the demo data should ship).
2. Delete the seed CSV and its seeds: block in dbt_project.yml.
3. Declare epm_gold.<name> as a source in
   dbt_project/models/staging/_staging__sources.yml and re-point every
   ref('<name>').
4. Add the DDL to clickhouse/init-db.sql and to
   konsol/clickhouse.py::_REFERENCE_TABLE_DDL (the idempotent upgrade path for
   volumes that predate it), keeping the two byte-identical.

This is the same shape as F3 (grynn-in/konsol#112) and F2 — the pattern is
established, only the data reconciliation is new.

[Anonymous]

Ticket changed by: grynn-in

• status: open --> closed