Observability Is Not Optional

The minimum bar for production AI:

1. Log every prompt, response, and tool call to a structured store (Postgres, Datadog, Langfuse).
2. Track tokens in/out and cost per request. Aggregate by user, endpoint, and feature.
3. Sample 1–10% of production traffic into an eval queue for human review.
4. Set up alerts on cost-per-user spikes — they catch infinite loops fast.

Tools to know: Langfuse, LangSmith, Helicone, Phoenix (Arize). All have generous free tiers.