72-second product walkthrough

See the quality shift your uptime chart misses.

Your LLM application can be up, fast, and within budget while its answers quietly become less useful. Verdict adds workload-specific quality evidence to the operational signals you already watch.

Watch the walkthrough ↓ Install without cloning
Illustrative dashboard · synthetic demo data · public alpha
Watch Verdict in action

From captured calls to inspectable drift evidence.

Follow the path from trace capture to rubric calibration, like-for-like comparison, and a persisted signal a reviewer can investigate.

Illustrative dashboard with synthetic demo data. The traces and results shown here are a bounded product demonstration, not customer traffic or a caught-in-the-wild regression.

1 minute 12 seconds · captions included · no autoplay

The workflow

Measure the behavior that matters for one workload.

Verdict focuses on the LLM-call layer: start with trace evidence, define an explicit rubric, test the judge against human labels, then compare recent behavior with a reference window. This demo illustrates Verdict's supported capture path.

  1. 01

    Start with trace evidence

    Use an existing Verdict-compatible store, or keep your normal Anthropic, OpenAI, or Google SDK path and let Verdict record operational metadata locally without putting a proxy in front of your traffic.

  2. 02

    Define what “good” means

    Use PASS/FAIL dimensions that belong to your workload, then label a representative sample before asking a model to judge it.

  3. 03

    Calibrate before trusting

    Measure whether the chosen judge agrees with your experts. A generic benchmark or an impressive point estimate is not treated as proof for your application.

  4. 04

    Compare like with like

    Group comparable prompts and emit a drift signal only when configured statistical and practical effect-size gates are met.

Inside the dashboard

Quality, cost, latency, and errors from the same evidence.

Each view stays tied to captured traces and persisted evaluation evidence, so a signal can lead to an investigation instead of another disconnected chart.

Verdict overview with synthetic traffic, cost, drift, and judge-readiness summaries
Overview. Start with traffic, estimated cost, latency, judge readiness, and the latest persisted drift state.
Verdict Trace Explorer displaying synthetic calls and rubric outcomes
Trace Explorer. Inspect the calls behind an aggregate and see when approved content is available—or explicitly unavailable.
Verdict judge-scores view showing synthetic pass rates and calibration evidence
Judge scores. Read pass rates beside evaluator identity and independent agreement evidence instead of treating a model score as ground truth.
Verdict drift detail with synthetic direction, effect size, adjusted p-value, and sample counts
Drift evidence. Review direction, effect size, adjusted p-value, sample counts, and evidence traces before acting.
Verdict comparison view for three models on the same synthetic workload
Model comparison. Compare candidate models on the same bounded workload; do not turn unrelated traffic into a universal leaderboard.
Honest scope

Evidence for LLM calls—not a claim about complete agent success.

What the demo shows

A packaged, read-only dashboard over synthetic traces; workload-specific rubrics; evaluator evidence; and persisted drift details that connect back to trace inspection.

What it does not prove

A real customer regression, calibration for your workload, full agent-run reconstruction, automated root cause, outbound alerting, or general production readiness.

Start with one workload

Install locally. Calibrate carefully. Let real traces guide the decision.

Verdict is a public alpha under Apache 2.0. Install the latest compatible packages directly from PyPI; if you need capture, add the maintained integration skill afterward. Neither step requires cloning the repository.

See installation options → Open GitHub Talk to us