Claude Code usage tracking by LangWatch

LangWatch is a platform designed for LLM evaluations and AI agent testing. It helps teams test, simulate, evaluate, and monitor LLM-powered agents throughout their lifecycle, both before release and in production. The platform is built for organizations that require robust regression testing, realistic simulations, and production observability without the need to develop custom tooling. LangWatch enables end-to-end agent simulations, allowing users to run scenarios against their full stack, including tools, state, user simulators, and judges, to identify where and why agents fail. It integrates evaluation, observability, and prompt optimization into a single workflow, facilitating a cycle of tracing, dataset creation, evaluation, prompt/model optimization, and re-testing. The platform is OpenTelemetry/OTLP-native and agnostic to specific frameworks and LLM providers, promoting open standards and avoiding vendor lock-in. It also includes an AI Gateway for governance and cost control, offering features like virtual keys, hierarchical budgets, inline guardrails, and automatic fallback across providers.