Menu

#256 feat(observability): hot-reload log level + SIGHUP restart for OTel config

closed
nobody
released (243)
2026-06-11
2026-06-11
Anonymous
No

Originally created by: Akarsh-Hegde

Summary

Eliminates the need to manually restart the daemon when changing observability settings.

Two tiers of reload, matching how Prometheus/Grafana handle this:

  • Log level → hot-reloaded in-process via tracing_subscriber::reload::Handle. Takes effect within one poll tick (≤60 s). No restart, no downtime.
  • OTel endpoint / credentials / toggle → daemon restarts gracefully via SIGHUP. launchd brings it back in ~2 s with fresh settings. ETL cursor is preserved (no data loss).

Changes

Rust daemon (src/)

  • observability.rs: EnvFilter now installed via reload::Layer; global FILTER_HANDLE (OnceLock) stores the handle. New reload_log_level() public fn + build_default_filter() helper. SIGHUP added alongside SIGTERM/SIGINT in the poll loop — same clean shutdown path, launchd restarts.
  • main.rs: poll loop tracks last_log_level, calls observability::reload_log_level() when changed. wait_for_shutdown now takes &mut sighup.

Next.js UI (ui/)

  • app/api/daemon/reload/route.ts (new): POST reads PID from ~/.meridian/daemon.sock, sends SIGHUP.
  • app/api/daemon/status/route.ts (new): GET liveness probe, used for polling.
  • SettingsView.tsx: Observability footer replaced — single Apply button that saves settings → sends SIGHUP → polls /api/daemon/status every 500 ms → shows Saving → Reloading → Active.

Test plan

  • [ ] Change Log Level in Settings → Save → within 60 s daemon logs "log level hot-reloaded" with old/new level fields — no restart
  • [ ] Change OTel email/password → Apply → UI shows Reloading… → Active within ~3 s — OO receives traces with new auth
  • [ ] Toggle OpenObserve Export off → Apply → daemon restarts → OO receives no new spans
  • [ ] Kill daemon manually while Apply is in flight → UI shows Failed after 15 s timeout
  • [ ] Set RUST_LOG=meridian=warn in env → change Log Level in UI → daemon logs skip the reload (env-var wins)

🤖 Generated with Claude Code

Related

Tickets: #194
Tickets: #251

Discussion

  • Anonymous

    Anonymous - 2026-06-11

    Originally posted by: Akarsh-Hegde

    Conflicts resolved — rebased onto current main.

    What conflicted:

    • ui/app/api/worklogs/route.ts + WorklogsView.tsx — duplicate of the worklog task-title commit from PR [#252] that's already in main; skipped the older copy.
    • ui/app/(dashboard)/queue/page.tsx, ui/app/api/queue-review/route.ts, ui/components/views/QueueView.tsx — modify/delete conflicts; all three were deleted in main when the Queue view was removed. Accepted the deletions.

    Branch is now 4 commits ahead of main (feat(observability): configure OpenObserve OTLP export, feat(observability): hot-reload log level, feat(daemon): SIGHUP-triggered restart + UI Apply button, chore: file header catchup). No content changes — conflict resolution only.

     

    Related

    Tickets: #252

  • Anonymous

    Anonymous - 2026-06-11

    Ticket changed by: Akarsh-Hegde

    • status: open --> closed
     
  • Anonymous

    Anonymous - 2026-06-11

    Originally posted by: adityaharishch

    🎉 This PR is included in version 1.48.0 🎉

    The release is available on:

    Your semantic-release bot 📦🚀

     

Log in to post a comment.