Single and multi-agent systems
Record runs, turns, tools, hand-offs to sub-agents, tests, retries, feedback and explicit business outcomes with a framework-neutral Python SDK.
The gap
Agents rarely crash. They quietly do the wrong thing.
A trace is not a verdict. Tracing tools record every call. Someone still has to decide whether the agent got worse, which runs are affected and whether the fix works.
Thresholds guess. A fixed alert line on a noisy judge score either fires constantly or misses the change that matters.
“Done” is a claim. An agent's final answer says the refund was issued or the tests pass. The recorded tool results and test runs say whether that is true.
Five problems, five short videos
What Verdict does for a team running agents.
Each card links to a recorded walkthrough of the released beta on a wholly synthetic, multi-agent support system and two synthetic coding-agent sessions.
Production agents
A release quietly broke a workflow.
A prompt or model change shipped. Nothing crashed, but refunds stopped completing. Verdict links recorded failed outcomes, tool errors and thumbs-down feedback to the runs and events that recorded them, then shows where the failures concentrate.
Watch the flow →
Model-call drift
The model started behaving differently.
Thresholds and averages either page you constantly or miss the change. Verdict compares a frozen reference cohort with current traffic using significance tests, corrects for testing many signals at once, and reports a change only when it is statistically significant and larger than a minimum effect size.
Watch the flow →
Release check
The fix might break something else.
Run the live version and the candidate on the same reviewed cases before shipping. Verdict reports what was fixed, what newly regressed, and what stayed the same.
Watch the flow →
Claude Code and Codex
The coding agent said it was done.
Verdict reads local Claude Code and Codex history without instrumentation or an API key. It keeps the agent's final answer next to the recorded test results and token growth, so “all tests pass” can be checked against what actually ran.
Watch the flow →
Your existing stack
You already have a tracing tool.
Keep it. Verdict imports Langfuse, LangSmith, Phoenix, Opik, MLflow, Datadog and OpenTelemetry GenAI data and adds the verification layer on top: evidence-linked findings, statistical change detection and release checks.
Watch the flow →
Recurring checks
Nobody is watching overnight.
Verdict suggests checks that fit the evidence you capture, runs them once for review, then hands them to a supervised worker. It shows whether the worker actually ran, not just whether a schedule was saved.
How monitoring works →Why it is different
Statistics and evidence, not dashboards and guesses.
Verdict is not another place to store traces. It is the check that runs on top of them.
Significance-tested change detection. Fisher's exact test per signal, Benjamini–Hochberg correction across signals, a minimum effect size, and alpha spending across repeated looks. Results report which signals had enough data to test; “no alert” covers only the tested signals. Statistical alerts apply to provider model calls; agent-run comparisons are labeled descriptive.
Findings open the evidence. Each finding links to its affected runs and, where one was recorded, the failing event. Missing steps and capture gaps are shown, not hidden.
Honest labels. Recorded failures, descriptive splits, judge opinions and statistical alerts are labeled separately, so nobody mistakes one for another.
Your environment, your keys. SQLite locally or a dedicated PostgreSQL deployment. Content is redacted before storage. Judge calls are optional and use your provider key: Evaluator Lab evaluations need an evidence preview, approval and a list-price spend limit, dashboard Inspect asks you to confirm before it sends sampled conversations, and the evidence you approve is sent to that provider.
Works with your stack
Keep Langfuse, LangSmith or Phoenix. Add the verification layer.
Verdict captures supported provider SDK calls and agent work directly, or imports the data your current tools already collect. Imported calls join the same investigation, drift and evaluation workflow.
See supported sources →- Langfuse
- LangSmith
- Arize Phoenix
- Comet Opik
- MLflow
- Datadog LLM Observability
- OpenTelemetry GenAI
- OpenInference
- Anthropic SDK
- OpenAI SDK
- Google GenAI SDK
- Claude Code
- Codex
- Voice transcripts
Where it fits
One verification layer across every kind of agent.
Chat and voice
Group turns into sessions, import text-only voice transcripts, and grade conversations with your own rubric when you choose to.
Claude Code and Codex
Import local history read-only to see sessions, tool use, test results and reported tokens. No instrumentation and no API key.
Start with one agent
Bring one workflow and one question. We will show you the evidence.
Install locally in minutes, or run a scoped proof of concept on a production agent with your data staying in your environment.