Best LLM Monitoring Tools in 2026

LLM monitoring is software that charts cost, latency, errors, and quality on live model traffic, then pages someone when a threshold breaks. If it cannot fire without a human opening a trace, it is a log, not a monitor.

The chart is the product, and the prompt text is evidence you may not be allowed to keep. Langfuse sells custom dashboards on every cloud plan and then meters alerts: 2 on Hobby, 20 on Core, 50 on Pro, 100 on Enterprise, so the page is capped by plan. Datadog prices Agent Observability by LLM span and, in its SDK docs, keeps token and cost metrics on all instrumented traffic even when you sample traces. A cheaper sample does not hide the spend chart. LangSmith's monitoring tab keeps working after a base trace's 14-day retention ends, because it reads metadata kept for more than 30 days. Helicone's alert docs cover error rate, cost, latency, token counts, and request count, with email or Slack.

A gateway monitor sees the request you were willing to proxy. Portkey and Helicone sit on that path, which is fast to turn on and blind to a tool call that never left your process. Skip the proxy when that failure lives inside the agent loop. An SDK monitor sees the steps you instrumented, which is what you want for an agent and what you will not get from a base-URL change.

Quality monitors are a third shape, for a team that pages on a wrong answer rather than a slow one. Galileo wants eval scores to become guardrails, Braintrust wants custom charts of scores, and Fiddler sells a per-trace meter next to a free guardrail. None of those is a substitute for a latency page if the score never runs.

The wrong adjacent purchase is a generic ingest bill with no LLM span. New Relic prices data at $0.40/GB after 100 GB a month, which can hold model telemetry and still does not name an LLM span. The head-to-head on the rest of the stack is Datadog vs New Relic, and prompt-injection blocking belongs in LLM security.

The expensive mistake is paying for 14 days of prompt text and assuming the page will still explain last month's bill. LangSmith base traces are kept 14 days, and extended traces are 180 days for new traces after the September 14, 2026 change in LangSmith's billing docs. The monitoring tab is the part that stays when the review is a chart rather than the prompt. If your on-call playbook says "open the prompt," you needed the longer tier, and that upgrade is a second charge on the same trace.

Langfuse makes the same split with a clearer clock. Hobby keeps 30 days of data and 2 alerts. Core keeps 90 days and 20 alerts, enough to cover a monthly review. Pro keeps 3 years, 50 alerts, and the SOC 2 and ISO 27001 reports, at $199/mo, with the same 100,000 included units as Core. You are paying for history and the page, not for more included traffic. Enterprise is $2,499/mo and raises the alert cap to 100.

Portkey splits the clock inside one plan. Developer keeps logs for 3 days and metrics for 30. Production keeps logs for 30 days and metrics for 90. The chart answers what spiked, and the log answers what the model said. Buying the short log and expecting a postmortem is how teams discover the gap during the incident. Helicone's Hobby plan keeps 7 days and Pro keeps 1 month, so a review the following week needs the paid retention. Team, at $799/mo, keeps 3 months and is the first tier that lists SOC 2 and HIPAA.

Datadog's default on the first span block is 15-day trace retention. Longer trace retention is a higher rate per 10,000 spans, up to $7.50 per 10,000 on an annual bill for 90-day traces. Sample the traces if you must, and read the SDK note first. Token and cost metrics stay on 100% of instrumented traffic, while the bill follows the spans you send, and the default sample rate is 1.0. A sample that was meant to save money does not shrink the cost chart.