An AI feature is valuable not because its answers look impressive, but because it consistently supports a clearly defined task. Start design with the workflow, the user and acceptable error levels, rather than choosing a model first.
This guide helps define the task, prepare evaluation samples, identify where human review is needed and measure results in a low-risk pilot.
1. Start with the task, not AI
Describe in one sentence the decision or manual task the feature would support. Make the goal observable, such as preparing an incoming document's data for review, rather than simply saying “let's use AI”.
- Who starts the task, and with what input?
- What result is expected, and where will it be used?
- Which errors are merely inconvenient, and which could cause business or legal harm?
- What are the measured duration and error rate of the current manual process?
Open image full size 2. Prepare evaluation samples
- Gather anonymised real examples or synthetic samples covering common, rare and invalid inputs.
- Record the acceptable result or evaluation rule for every example.
- Keep development samples separate from the final evaluation set so that you do not optimise only for previously seen cases.
- Measure correctness, omissions, false claims, response time and processing cost separately.
3. Define where human review is needed
| Risk | Recommended behaviour | Example |
|---|---|---|
| Alacsony | Automatic output with a reversible action | Internal tagging or search suggestions |
| Medium | Human approval before execution | A draft reply or suggested data entry |
| Magas | AI organises information without making decisions | An action affecting finance, legal matters or permissions |
4. Launch a limited pilot
- Start with a clearly defined user group and reversible actions.
- Log the input type, result status and human corrections, but do not collect unnecessary personal or confidential content.
- Define stop conditions, such as repeated serious errors or unusual cost increases.
- At the end of the pilot, compare with the original manual workflow, not a theoretical perfect system.
5. Use this evidence to decide the next steps
- The user completes the task faster or with fewer errors.
- Correction and review time does not cancel out the savings.
- Access, data retention and supplier terms are documented.
- It is clear how costs change with increased usage.
- Someone is responsible for tracking quality, security and future retesting.
