The starting point
‘We should add AI’ is a conversation starter, not a product brief. A more useful starting point is a specific task and a reason to improve it. What is difficult today, who would use the output, and how would you tell whether an AI-assisted approach was actually helpful?
Keep the first task narrow
Choose a task with clear inputs, a defined output and someone who can judge the result. For example, an internal pilot might draft a summary from approved documents for a team member to review. Treat this as a hypothesis to test, not a promised improvement.
Compare the idea with a simpler approach, such as a better search interface, a fixed rule or a workflow change. Document why AI or machine learning might be worth testing. Do not assume the most complex approach is the best fit for the problem.
Understand the data and its boundaries
List the information the pilot would need, where it comes from and who is allowed to use it. Check whether the examples are representative of the task. Decide what must be excluded before connecting a tool to business data.
Review access, retention and provider terms with the appropriate people in your organisation. Avoid entering sensitive information into an unapproved service. The pilot plan should name an owner for these decisions instead of leaving them as assumptions for the implementation team.
Define what a useful result looks like
Build a small evaluation set with ordinary cases, awkward cases and examples where the system should ask for help. For a document summary, reviewers might check whether important details are missing, statements are supported by the source and the output is useful for the intended task.
Compare results with the current process. Account for the time spent reviewing and correcting the output, not just the time to generate it. Agree in advance what findings would justify continuing, changing direction or stopping the pilot.
Plan review, fallback and ownership
Make the review step part of the workflow. Decide who can approve, edit or reject a suggestion and how the work continues when the system is unavailable or the result is unsuitable. Keep a conventional way to complete the task where the risk calls for one.
Use a risk-management process appropriate to the application. NIST’s AI Risk Management Framework offers voluntary guidance for considering AI risks through design, development, use and evaluation. It is a useful reference for structuring the discussion, not a certification or a substitute for reviewing your particular use case.
A reference for the risk-management discussion above.
NIST: AI Risk Management FrameworkPut the thinking into practice
Ready for your next step?
