One of the most useful questions I can ask an AI assistant is also one of the simplest: “How will we know this is finished?”
A confident answer is pleasant. A result I can use is what matters. If I ask for an article to appear in two places, I need both copies to be there, readable, and saying the same thing.
Start with the result someone needs
Hamel Husain and Shreya Shankar offer a useful starting point in their guide to evaluating agent workflows: first assess whether the whole task achieved its goal. Investigate individual steps when a failure needs explaining.
My practical interpretation is to write a small finish line before starting. For a publishing task: the right article, in the right places, with matching content and working source links. That keeps the review connected to the original request.
Technical translation Explore the detail
An end-to-end check examines the completed outcome. A diagnostic check looks inside the process to explain a particular failure. They answer different questions; we can choose the one that helps the current decision.
Check what happened after the action
For this series, our publication review included opening the public website and Substack pages, comparing their article text, and checking the link back to the original. Those observations support a narrow conclusion: those editions were present and matched when checked.
They do not show that the article attracted readers, helped someone make a better decision, or improved the business. Those would need different evidence.
A small-business example: if you ask an assistant to prepare a meeting agenda, a useful finish line might be an accessible document with the decisions, background and questions the meeting needs. Counting how many steps the assistant took would tell you much less about whether you can use it.
Technical translation Explore the detail
A tool response is evidence about one action. A readback checks the resulting state: what was actually saved, changed or made available. For objective facts, a simple comparison can be more useful than another opinion about whether the work looks complete.
Teach the reviewer what good means
Antaripa Saha and Hamel Husain studied automated review using one set of 100 apartment-leasing conversations. The reviewers identified useful issues, but missed some failures that depended on the product’s goals and customer experience. The authors also found false alarms. That is evidence from one setting, not a universal score for AI reviewers.
The lesson I take is to give an assistant concrete examples of useful and disappointing work, along with the reasons. “Make this excellent” leaves a lot open. “The reader should understand this idea and leave with one action they can try” gives the review a purpose.
For our radar, that means keeping the explanation approachable, making the source easy to find, and being clear about what we have actually tried. A technically accurate article can still be difficult to read or offer very little help.
Keep the check small enough to use
Before your next AI task, finish these three sentences:
- The result I need is…
- I will know it worked when…
- The evidence I will check is…
Then review that evidence. If the outcome falls short, identify the first meaningful gap and address it. Repeating a check that already passed should have a reason, such as a relevant change or a new concern.
We are applying this thinking to the radar’s publishing workflow. Its usefulness over repeated unattended runs remains to be tested. We have not established that these checks save time or prevent every mistake.
Sources and acknowledgments
- Hamel Husain and Shreya Shankar, How do I evaluate agentic workflows?, published June 29, 2025; modified September 1, 2026; reviewed November 12, 2026.
- Antaripa Saha and Hamel Husain, Do Automated Evals Work?, published July 11, 2026; reviewed November 12, 2026. A single-dataset study; no vendor ranking adopted here.
The business examples and suggested questions are our interpretation. The publication observations concern this series only. Prepared with AI writing assistance; no client information is included.