Before adding AI agents, build the ability to judge their work

Before giving an AI system more responsibility, understand how the people using it assess its output. Can they explain its limits? Can they recognise a poor result? Do they know which tasks are appropriate?

Identify the gaps first, then focus learning on relevant tasks. A team preparing reports needs practice checking summaries and sources. A team exploring automation needs to understand the steps, decisions and exceptions in its process.

Use simple examples, discuss mistakes and put the useful guidance where people work. Familiarity grows through application and feedback, rather than a single presentation.

Consider a manager using AI to summarise a project update. A fluent answer might say delivery is on track even though a source note says a key dependency is unresolved. A capable reviewer checks the source, spots the conflict and changes the conclusion. Being able to write a polished prompt would not settle that problem.

Build a short practice exercise around a task the team recognises. Give people the source material, an AI draft and the acceptance criteria. Ask what they would send, what they would change and what they would escalate. Discuss the differences. This reveals a more useful learning need than asking whether people feel confident with AI.

For a first workflow, keep the human approval point clear. The person using the tool should be able to explain the source, the intended audience and the checks needed before the output is used. If they cannot, narrow the task and make the checking steps explicit.

As the workflow gains more responsibility, test more than the happy path. What should happen when source data is missing, a request is outside scope or two records disagree? Decide who owns the exception and how work returns to a person. These decisions belong in the workflow before you automate its actions.

A useful internal champion needs more than a label. Give them time to run examples, collect recurring questions and improve the guidance. They also need a route to the people responsible for information security, data and the business process.

Keep a small library of checked examples and known failure cases beside the tools people use. Revisit it when the tool, process or information changes. The aim is a team that can explain its judgement, including when it decides not to use AI.