#6 feat: one-click deployment, integration tests, pipeline automation

[Anonymous]

Originally created by: grynn-in

Summary

• One-click deployment: Full docker-compose.yml with 11 services (ClickHouse, MariaDB, Redis x2, Frappe backend/worker/scheduler, Cube.js, Caddy reverse proxy, configurator, dbt init). Single ./deploy.sh command bootstraps everything with auto-generated passwords, healthchecks, site creation, and demo data seeding.
• Integration test suite: 4 pytest files covering staging table verification, dbt build validation, end-to-end pipeline chain (staging insert → dbt build → gold output), and Frappe sync pattern testing (NULL handling, SQL escaping, TRUNCATE+INSERT).
• Pipeline automation (konsol app, separate repo): Auto-triggers debounced dbt build after any consolidation/allocation doctype save via doc_events hooks on 9 doctypes.
• Validation & monitoring: Source freshness checks on all epm_staging tables (warn 72h, error 168h), ClickHouse sync failure tracking with realtime error events, check_health() function, upgraded /api/method/konsol.api.health endpoint.
• Backup/restore: ./deploy.sh backup dumps MariaDB + ClickHouse + Frappe files with 7-day rotation and optional S3 upload. ./deploy.sh restore --from <path> restores everything.

Files changed

Area	Files
Deployment	deploy.sh, docker-compose.yml, docker/ (5 files), .env.example, cube/cube.js
Tests	tests/integration/ (4 test files + conftest.py)
dbt	_staging__sources.yml (freshness)
Docs	deployment-guide.md (rewritten)
Git	.gitignore (backups, credentials)

Companion changes (konsol repo — not in this PR)

• tasks.py: run_dbt_build_async(), on_consolidation_doc_update() hook
• hooks.py: doc_events for 9 consolidation/allocation doctypes
• clickhouse.py: sync failure tracking, check_health(), realtime error events
• api.py: upgraded health endpoint with ClickHouse status
• install.py: Docker configurator EPM Settings setup

Test plan

• [ ] docker compose config validates without errors
• [ ] ./deploy.sh runs successfully on a fresh Ubuntu 22.04 server
• [ ] Frappe accessible at :8069, Cube.js at :4000, ClickHouse at :8123
• [ ] pytest tests/integration/ -v passes with ClickHouse running
• [ ] ./deploy.sh backup creates backup in ./backups/
• [ ] ./deploy.sh --domain example.com configures Caddy with auto-SSL

🤖 Generated with Claude Code

[Anonymous]

Ticket changed by: grynn-in

• status: open --> closed