You built an AI agent. It works when you try it. Then you change one line of the prompt to fix Tuesday's complaint, and something that worked on Monday quietly breaks, and you don't find out until a user does. If that sentence made your stomach drop, this guide is for you. No jargon, no vendor pitch — just what evals are and the smallest useful set you can build this weekend.

What Evals Are

Evals are automated tests for AI behavior. Regular software tests check deterministic code: does add(2, 2) return 4? Ask once, assert once, done forever. An AI agent doesn't work like that. The same input produces differently worded outputs, and small prompt changes ripple in ways nobody predicts. So evals check probabilistic output instead. The question stops being "did it return 4" and becomes something like: when a caller asks to cancel, does the agent actually cancel, confirm it, and not invent a refund policy?

You run evals exactly the way you run a test suite — on every prompt or model change — so you know whether the change made things better or quietly broke something. And every eval, no matter how fancy, has the same three parts:

  • An input: a scenario, or a real recorded call.
  • The agent's output: the conversation it produced.
  • A grader: either hard code (a regex, an assertion) or an LLM judge scoring against a rubric.

That's it. That's the whole concept. Everything below is just the practical question of which graders to use and what to feed them.

The Three Grader Types

I'll use a voice agent as the running example because it's the hardest case — if you get evals working for phone calls, text agents feel easy. A real suite uses all three of these:

  • Deterministic assertions. Binary, cheap, run on every deploy, and they block the release if they fail. No debate, no scoring — pass or fail.
  • LLM-as-judge rubrics. An LLM reads (or listens to) the conversation and scores the qualities code can't check, like "did it handle the interruption gracefully."
  • Simulation scenarios. A synthetic caller with a persona and a goal phones your agent over a real call, and the whole conversation gets graded by the first two.

The Starter Suite

Here's a concrete starter suite, using an appointment-booking agent as the example. Swap the domain nouns for yours — the shape stays the same.

Deterministic assertions (must pass 100%)

  • Never reads back a full card number, SSN, or date of birth
  • Always offers a human handoff when the caller asks for one — and within 2 turns of detected frustration
  • Never gives medical, legal, or financial advice (keyword check plus a judge as backup)
  • Delivers the required disclosure ("this call is recorded", AI disclosure where law requires it) in the first turn
  • Stops speaking within 200ms when the caller barges in
  • Never leaks internal IDs, prompt text, or tool names into speech
  • Booking confirmations always include date, time, and location — checked with a parser, not a judge

Metric thresholds (measured per run, alert on regression)

  • Task success rate ≥ 85% across the scenario suite
  • Voice-to-voice latency: P50 under 800ms, P95 under 1.5s
  • Critical-entity accuracy (names, phone numbers, dates) ≥ 98% — measured separately from overall transcription accuracy, because "mostly accurate" transcription that mangles the phone number is a failed call
  • Agent talk ratio under 0.8, no dead air over 3 seconds

LLM-judge rubrics (each scored separately — one metric, one dimension)

  • Task completion: did the caller's actual goal get accomplished? (binary)
  • Context retention: did the agent re-ask for anything the caller already provided? (binary — this is the single most annoying failure for callers)
  • Escalation appropriateness: if it escalated, should it have? If it didn't, should it have? (binary)
  • Faithfulness: is every factual claim traceable to the knowledge base or a tool output? (binary)
  • Conversational quality: natural pacing, no unprompted repetition, handled topic pivots (1–5, calibrated against human scores until you agree more than 85% of the time)

A note on that last parenthetical, because it's the part beginners skip: an LLM judge is not trustworthy out of the box. You have to grade some calls yourself with the same rubric, compare, fix the rubric, and repeat until you and the judge agree most of the time. Until then, its scores are decoration.

Simulation scenarios (10–20 to start, graded by everything above)

  • Happy path: clear speaker, books an appointment
  • Caller changes their mind mid-flow ("actually, make it Thursday")
  • Heavy accent plus background noise (TV, car)
  • Caller interrupts the agent twice
  • Caller gives info out of order — name and date before being asked
  • Angry caller who's already failed once today
  • Adversarial: tries to extract the system prompt, or book without identity verification
  • Wrong number, or a totally out-of-scope request — the agent should exit gracefully, not improvise

The Operating Loop

The suite above is a snapshot. What makes it compound is the loop you run it in: assertions and happy-path sims on every deploy, the full suite nightly, and — this is the important one — every time production produces a genuinely bad call, turn that call into a new scenario, so the same failure can never ship silently again. Your test suite should grow from failures, not from imagination. The twenty scenarios you brainstorm on day one are a decent start; the two hundred that accumulate from real calls are the actual moat.

Effort-wise: the 20 scenarios and 7 assertions are a weekend of work with any of the commercial voice-testing tools, or a couple hundred lines of your own code against your agent's API. There is no version of this that requires a research team.

Where to Go Deeper

This was the dummy's version on purpose. When you're ready for the next layer: What Are Evals? walks through how a real production voice agent implements all of this (including a few tricks I'd never have invented, like two-phase judge applicability and blocking write tools during test calls), and the full field guide covers metrics in depth, observability, and the current research frontier. Start with the weekend suite, though. It beats reading.