Testing Tool Calls, Handoffs and Agent Workflows

Testing an AI agent means testing a complete workflow, not just asking whether its prose sounds good. An agent can produce a reasonable sentence while selecting the wrong source, calling the wrong tool, exposing data, creating a duplicate record, or failing to hand off an exception.

Build a representative task set

Collect real requests from the workflow and label the expected outcome. Include common questions, ambiguous wording, missing fields, conflicting records, out-of-scope requests, and cases that should be refused or escalated. Keep acceptable alternatives where more than one answer or route is valid.

Separate the test set from the prompt examples used during development. A hidden or carefully controlled evaluation set helps reveal whether an apparent improvement generalises beyond the examples the system has already seen.

Test each layer

• Intent and field extraction: did the agent understand the request?
• Retrieval: did it select current, authorised sources?
• Planning: did it choose a sensible next step?
• Tool calls: were the tool, arguments, and permissions correct?
• Handoffs: did it provide a useful and complete context?
• Response: was the result accurate, clear, and appropriately cautious?

Layered tests make failures actionable. A wrong answer caused by stale retrieval needs a different fix from a correct answer followed by an unauthorised tool call.

Use realistic tool doubles

Do not test every run against production systems. Create safe test doubles that return realistic successes, validation failures, timeouts, partial data, duplicate responses, and permission errors. Verify that the agent reacts correctly to each result and does not retry a non-idempotent action blindly.

Record the tool name, arguments, authorisation outcome, and result class. Scrub personal information and secrets from test traces. The goal is to understand behaviour without turning the evaluation environment into another source of risk.

Test handoffs and recovery

Force the agent to stop when information is missing, a policy is unclear, or the request exceeds its authority. Check that the human receives the objective, evidence, completed work, unresolved questions, and proposed next action. Test what happens when a reviewer rejects the proposal or an external system becomes unavailable halfway through.

Run regressions deliberately

Run the evaluation set after changes to prompts, models, retrieval settings, source documents, tool schemas, permissions, and routing. Store the version of each dependency with the results. Investigate not only the average score but also new severe failures and changes in cost or latency.

Use permission and secret controls as part of the tests, not as a separate checklist. Include indirect instructions in documents, hostile tool results, and attempts to access another user or tenant.

Set a release gate

Define minimum acceptable behaviour for critical paths and identify failures that always require human review. Assign an owner for approving release and for reviewing production traces afterwards. An agent is ready when its important behaviours are repeatable, observable, and bounded—not merely when a demo looks impressive.