Check the Records Before You Trust an AI Agent

An AI agent investigated a delayed delivery, checked the refund policy, created a support ticket and marked it resolved. According to a joint Microsoft and Hugging Face article about ThinkingBox, the delivery problem was still outstanding. The ticket should have remained on hold, and the customer had not received a proper answer.

The example comes from a benchmark, rather than a reported customer incident. But it captures the problem the researchers set out to measure: an agent can perform apparently sensible steps and still leave the business record in the wrong state.

Microsoft’s ThinkingBox evaluates the records and other changes agents leave behind. The authors report testing 507 business workflows, with each task repeated 20 times from an identical clean starting point. They assess the final result, rather than accepting the agent’s closing message as proof.

In a separate analysis covering 121,680 valid trials across 12 models, the authors found that 67.24% of failed attempts ended cleanly, used a tool that changes records and reported no final tool error. A smooth finish did not establish that the work was correct.

Your definition of done needs a record check

For a business of five to fifty people, the practical lesson from ThinkingBox is to make completion something you can inspect. Before delegating a task, specify which record should change, what its final value should be and what must remain untouched.

That distinction matters in the article’s delivery example. The agent did create and update a ticket. Checking only that an update happened would have missed the mistake. The required evidence was more specific: the ticket needed the correct status for an unresolved delivery.

The authors also found wrong field values, unintended extra changes and missing required changes among failed attempts. Their findings suggest checking both sides of a task: whether the intended work happened and whether anything else changed unexpectedly.

Keep the limits of the research in view. ThinkingBox reports benchmark results, not performance measurements for your business. Treat its findings as a reason to test your own workflow, rather than a prediction of your error rate.

Put evidence into your next agent trial

Based on the benchmark’s approach, use this checklist for the next workflow you trial:

• Write the finish line first. Name the record, required status or value, and any conditions that must still be satisfied before the task counts as complete.
• Read the saved result. Check the business system after the agent finishes. Compare the actual record with your requirements instead of relying on its summary.
• Check for extra changes. Look for records or fields altered beyond the agreed task. ThinkingBox explicitly checks unintended effects as well as missing work.
• Repeat the same test. Use a clean starting point each time. Record every outcome, including failures, so one successful demonstration does not become your entire assessment.
• Keep exceptions open. If the saved result fails your checks, require review before accepting completion.

ThinkingBox distinguishes succeeding once from succeeding across all 20 recorded attempts. The authors show that broad task coverage and consistency can differ substantially. For your trial, ask how often the agent reaches the correct final state, not simply whether it has ever managed the task.

Your acceptance rule can be straightforward: the agent reports what it did; the saved records provide the evidence. Make that check part of the workflow before handing over routine work.