#2 [HOLD] feat(observability): end-to-end OTLP tracing via OpenObserve

[Anonymous]

Originally created by: Akarsh-Hegde

Summary

End-to-end observability across all three Meridian runtimes (Rust daemon, Python agents, Node MCP/UI), backed by a local OpenObserve instance and stitched together via W3C trace context.

Filter by session_id in OpenObserve and you see the full journey of a screenpipe frame: ETL block close → Python rule firing → stage2/stage3 LLM decision → Jira sync — all in a single trace tree.

What changed

• Rust daemon (src/observability.rs): layered tracing-subscriber (stderr pretty + ~/.meridian/logs/*.jsonl daily-rotated + OTLP/HTTP exporter). #[tracing::instrument] on run_etl, close_block, extract_block_context, upsert_open_block, cleanup_incomplete_runs, categorize, settle_*, three classifier backends, jira provider. close_block persists the active span's traceparent into app_sessions.traceparent so downstream Python agents inherit the trace.
• Python agents (services/agents/observability.py): one setup(agent_name) call wires OTel TracerProvider, OTLPSpanExporter, LoggingInstrumentor, and python-json-logger. Spans on tagger.run_once, tagger.session, tagger.rules.fire, tagger_daemon.tick, stage2.match, stage3.decide, jira_keeper.sync, jira.api.update, embeddings.encode_batch. _tag_session reads traceparent from the DB row and continues the Rust trace.
• MCP server (packages/meridian-mcp/): pino → stderr (stdout reserved for JSON-RPC; verified clean), OTel SDK exports spans, every tool dispatch wrapped in mcp.tool.<name>.
• Next.js UI (ui/): instrumentation.ts register hook + lib/observability.ts. Each /api/* route handler wrapped in api.<name> span, console.error replaced with logger.error.
• Migration 010_traceparent.sql: adds traceparent TEXT to app_sessions, agent_runs, ticket_links.

How to use it

# Start OpenObserve (one-off install — single binary at ~/.openobserve/, launchd-managed)
launchctl load ~/Library/LaunchAgents/ai.openobserve.plist

# Run the daemon with OTLP enabled
export MERIDIAN_OO_AUTH=$(cat ~/.openobserve/.auth_token)
./target/release/meridian

# Browse logs + traces
open http://localhost:5080

Useful queries:

• SELECT service_name, operation_name FROM "default" WHERE session_id = '<id>' — full cross-process journey
• WHERE trace_id = '<id>' — single trace tree

Test plan

• [x] cargo build --bin meridian clean
• [x] cargo clippy --bin meridian -- -D warnings clean
• [x] cargo test — all suites pass
• [x] cd packages/meridian-mcp && npm run build clean
• [x] cd ui && npm run build clean
• [x] MCP stdio smoke: stdout has only JSON-RPC, all pino output on stderr
• [x] End-to-end smoke: Rust daemon closed session 1868, full 7-span trace in OpenObserve
• [x] app_sessions.traceparent populated with valid W3C string (00-{trace}-{span}-01)
• [x] Python extract_parent_context unit-verified: child span inherits Rust trace_id 755c4549160e7ae61eb865531bca6dee exactly
• [x] OpenObserve cross-service query for session_id returns spans from both meridian-rust and meridian-tagger

Caveats

• Full Rust→Python trace tree (via OpenObserve's reference_parent_trace_id link in the UI) wasn't observed in the live smoke because both sessions with traceparent populated so far are too short (5-7s) and get filtered as trivial-overhead before reaching _tag_session. The extraction code path is unit-verified; it will fire naturally as soon as a longer session goes through.
• Pre-existing ImportError in services/agents/jira_keeper.py:20 (importing JIRA_STATE_FILE, CURRENT_CONTEXT_FILE, JIRA_DIR from agents.config) is not introduced by this PR — confirmed present on main before this branch. Worth a separate fix.
• github.rs and linear.rs provider stubs are uninstrumented because they have no outbound calls yet.

🤖 Generated with Claude Code

[Anonymous]

Ticket changed by: Akarsh-Hegde

• status: open --> closed