Human review

Marion Clegg complained about a letter from her adviser's firm. It said she had agreed to raise her drawdown income to £2,000 a month. She hadn't. On the recording of the meeting, she only asked what that would do to her pot. The AI had drafted the letter from its notes of the meeting. The product's log showed the adviser approved the notes of a 70-minute meeting 40 seconds after they arrived. The notes were usually right, so approving them had become a habit.

The approval put a review on file, but nobody had compared the notes with the recording. Human review means a person checks the AI's work against its sources before the firm relies on it, and fixes or rejects anything that doesn't match. For meeting notes, the source is the recording. For a suitability report, it is the fact-find and statements. Reading the draft and deciding it looks right does not count.

Firms already run a four-eyes check, where a second person checks work before it goes out. Human review applies it to the AI's work. A colleague unsure what Marion had decided would have left a question for the checker. Instead, the AI wrote that she had agreed, a hallucination, and nothing marked it as a guess.