What it is
Agent evaluations are repeatable tests that score an AI agent's answers and actions for accuracy, safety and task completion before and after release. They compare responses to a 'golden' expected answer or rubric and catch regressions, hallucinations, and tone drift whenever a prompt, model, or data source changes. Evals can be automated (LLM-as-judge), rule-based, or paired with human reviewers for nuanced cases.
Why it matters
Mid-market teams often ship agents fast but lack a safety net when prompts or underlying models change. Evals turn 'it seemed fine in testing' into measurable quality scores leadership can trust.
Key components
- golden datasets
- scoring rubrics
- LLM-as-judge
- regression testing
How it connects
Evals feed directly into tools like Agentforce Testing Center and the broader ADLC, gating releases before agents reach customers.
Good to know
Start with 20-30 real customer questions as your golden dataset before investing in fancy eval tooling.
Related terms
Agent Development Life Cycle (ADLC)
A structured methodology for planning, building, testing, deploying, and maintaining AI agents in enterprise environments, adapting SDLC practices for agent-specific concerns.
Agentforce Testing Center
Salesforce's built-in tool for testing Agentforce agents at scale against generated and real test cases before they go live.
Guardrails
The boundaries you set on AI agents to control what they can discuss, what actions they can take, and when they must escalate to a human.
Hallucination
When an AI confidently generates information that is factually wrong — the primary risk in any AI deployment without proper grounding.
