Phoenix Tracing

Phoenix is the agent-tracing dashboard. OptimalOS optionally exports OpenTelemetry traces of agent runs (Claude Code, OpenClaw, etc.) to a self-hosted Phoenix instance running on the Pi.

Topology

Concern	Where
Phoenix server	Docker container on the Pi. (Hetzner CCX13 has a dormant phoenix.service on :6006/:4317 with the Pi snapshot pre-staged — awaiting Cloudflare CNAME flip per kanban f997057e.)
HTTP UI	:6006 (also published at phoenix.optimal.miami via Cloudflare Tunnel).
OTLP gRPC ingest	:4317.
Storage	SQLite local to the container. Not synced to Supabase.

The container is named phoenix and runs alongside the Pi's other services. OS-level integration is via Docker (requires sudo docker on the Pi).

Enabling tracing in OptimalOS

Tracing is opt-in. Set phoenix.enabled = true in ~/.optimalos/config.json:

{
  "phoenix": {
    "enabled": true,
    "otlp_url": "http://localhost:4317"
  }
}

Or via env var:

PHOENIX_ENABLED=true
PHOENIX_OTLP_URL=http://localhost:4317/v1/traces

The OpenTelemetry exporter lives at optimalOS/src/orchestration/tracing.ts. When phoenix.enabled is true, agent lifecycle events and Loom step runs emit OTEL spans pointing at the configured endpoint.

What gets traced

• Agent session start / stop (with agent type, command, tmux session id).
• Tool-use events (when Claude Code's hook receives a tool_use payload).
• Loom step boundaries (start, finish, status).

Traces are joined on the OptimalOS runId / sessionId so a single agent invocation appears as one trace tree across multiple spans.

Viewing traces

Open https://phoenix.optimal.miami in a browser. Phoenix's UI groups traces by service, and the search bar filters by attributes (e.g. optimalos.workflow_id = "priority-triage" to see only triage runs).

Why self-host

Phoenix's hosted tier exists, but:

• The Pi already has the compute headroom.
• Trace data includes prompt content for Claude Code calls — keeping it on-prem matters.
• SQLite-backed Phoenix is easier to back up alongside the rest of ~/.optimalos/.

Stuck-run reaper

A periodic Loom workflow walks Phoenix's traces looking for runs marked running for >24 hours and marks them cancelled with a "stuck — reaped by phoenix-cleanup" reason. This avoids the dashboard accruing perpetually-running placeholders when an OptimalOS process is killed mid-run.

To verify: reaper workflow id

Confirm the exact workflow id in optimalOS/workflows/. If you can't find it, the reaper may live in optimalOS/src/orchestration/auto-claim.ts or a dedicated cleanup module.