#166 Retire generate_demo_data.py

[Anonymous]

Originally created by: grynn-in

Part of [#158] (the "retire generate_demo_data.py" item). The other items in [#158] stay open.

What was removed and why

The Contoso/Alpine demo is already gone: [#156] replaced clickhouse/demo-data.sql with clickhouse/raw-schema.sql, and konsol [#127] made fixtures reference-only. Nothing in the deploy scripts runs scripts/generate_demo_data.py. It still emitted Historical Equity Rate INSERTs, the last trace of the dual-writer bug from konsol#82 (closed). konsol is now the only regular writer of historical_equity_rates, through the Historical Equity Rate doctype on submit.

• Deleted scripts/generate_demo_data.py (1,583 lines). No helper modules, fixtures, Makefile targets or CI steps used it, and nothing imported from it.
• .gitignore: removed the clickhouse/demo-data.sql entry and its comment. Only the script produced that file.
• docs/admin-guide/deployment-guide.md:
• The "Demo Data" section (the fictional Alpine group, its entities and IC flows, and the pointer to the script) is now a "Loading Data" section. It says there is no demo data. Data comes from connectors or trial balance uploads: a single trial balance through the Trial Balance Submission form in Desk, and bulk uploads (one file for many entities and periods) through konsol-exec at /konsol-exec/uploads, which only the Close Lead (EPM Admin) can use.
• The deploy diagram is renumbered to match deploy.sh's five steps, with dbt build as step 5. The prerequisite check and the final URL printout are unnumbered, as they are in deploy.sh.
• "Connecting Real ERP Data" no longer talks about replacing demo data. Its last step points at the Run Pipeline button on a new Pipeline Run; EPM Settings has no such button.
• docs/getting-started/setup-guide.md, docs/user-guide/initial-setup-guide.md: removed the "demo mode" and "seed/demo data" wording. Airbyte is optional when entities upload trial balances, and the single and bulk upload paths are described the same way as above.
• docs/d365-consolidation-readiness.md: one sentence added to the existing "partly superseded" note to say the demo data has since been removed. The rest of the dated snapshot is unchanged.
• tests/integration/test_dbt_build.py: test_gold_trial_balance_has_rows assumed seed data, so it failed on a fresh stack. It now skips, with a reason, when no source GL data exists: the D365 and ERPNext connector tables in epm_raw are empty or missing, and no trial balance submission batch is claimed (the same inner join bronze_trial_balance_submissions uses). It uses the file's existing ch fixture. With source data it asserts rows exactly as before. A docstring that mentioned demo data is reworded.
• Comments only, no logic change:
• dbt_project/tests/assert_equity_rate_coverage.sql: the [#120] background is marked as history (those fixes lived in the demo data and its generator, both removed), and the demo entity names are gone. It now says the test needs a rate for every entity node in consolidation_groups, parent included.
• dbt_project/dbt_project.yml no longer mentions "shipped demo data".
• docker/configurator/init.sh no longer says it "loads demo data", which it did not do.

Reference sweep (git grep -n <pattern> | wc -l)

Pattern	Before	After
generate_demo_data	2	0
demo-data.sql	9	4
demo-data	13	6
demo data	23	19

The demo data count includes new lines that say there is none (the setup guides and the test's skip reason) or mark it as history (the equity-test note).

The remaining hits are history or unrelated, and were left on purpose:

• docs/prd/**: kept as history.
• DBT-HANDOFF.md, DBT-PRDS.md: the log of a finished dbt agent loop (every item checked off).
• clickhouse/raw-schema.sql header: a provenance note that says where the CREATE TABLEs came from (#156).
• dbt_project/tests/assert_d365_gl_vouchers_balance.sql, assert_trial_balance_balances.sql: comments about D365 sandbox data, not the generator.
• deploy.sh (two "tolerated for demo data" comments): it does not reference the generator, and it is being changed in [#152].

docker-compose*, the Makefile and .github/workflows/ had no references.

Verification

• bash -n docker/configurator/init.sh: exit 0
• python3 -m py_compile tests/integration/test_dbt_build.py: exit 0
• The new skip helper, run against a stubbed ch: 0 rows on a fresh stack and on empty tables, 12 rows for a claimed upload, and the test skips before running dbt when there is no data. It was not run against a live ClickHouse.
• uvx --with-requirements docs/requirements.txt mkdocs build --strict -d <scratch>: exit 0 (baseline on main also 0)
• dbt parse (dbt-clickhouse via uvx, offline, target and log paths in a scratch dir): exit 0
• No * 2.* iCloud duplicates in the tree.
• Not run: deploy.sh, image builds and running containers.

🤖 Generated with Claude Code

https://claude.ai/code/session_013WewQKFQgG7o2M3mUDPRR5

[Anonymous]

Ticket changed by: grynn-in

• status: open --> closed