Is it a hallucination, or did you give the AI the wrong source?

Name the wrong claim

Calling every incorrect answer a hallucination can hide the actual cause. Start by identifying the material statement that is wrong and the approved evidence that contradicts or fails to support it. A model may faithfully explain a superseded source. That is still an unacceptable answer for the employee, but changing the wording alone will not fix the source problem.

Use a permitted evaluation record with the question, intended audience, applicable source, retrieved passages and response. Protect private information under your existing rules. If the relevant evidence cannot be inspected, record that limitation rather than guessing which component failed.

Find the first failing stage

Assign repair ownership according to that first failure. A missing source needs source or ingestion work. A wrong audience needs applicability or authorization review. Unsupported generation needs answer evaluation and validation. Several stages can fail together, so preserve evidence rather than forcing every incident into one category.

Illustrative editorial photograph, not a customer result.

Try a counterexample, not another perfect prompt

Ask the question with missing context, a common alternate term and a case the source does not answer. Does the system clarify, retrieve the right material or invent a universal instruction? If it succeeds only with the exact demonstration wording, the workflow may not be ready for ordinary employees.

Keep these cases in the evaluation set. Do not silently change the approved procedure to make the generated answer look correct. The procedure owner resolves policy ambiguity and signs off changes. A model is not the authority that chooses the company rule.

Repair the maintained cause

If employees repeatedly receive an obsolete escalation contact, correct the authoritative guide and verify propagation through retrieval and caches. Editing individual answers without addressing the source leaves the next employee exposed to the same problem. If the right guide is retrieved but the model adds a step, compare claims with passages and test the revised candidate on held-out questions.

Treat a prompt adjustment as a candidate repair, not independent proof. Inspect the actual answer, source and boundary behavior. If security controls failed, use the established security review and incident process rather than accepting a more cautious phrase as resolution.

Report failure types separately

Count wrong-source, wrong-version, wrong-applicability and unsupported-generation cases independently. Track reviewer effort and unresolved governance issues too. This gives leadership an actionable view of what must improve. A single accuracy percentage is much less useful when it hides both missing information and unauthorized disclosure.

Use the citation ledger and the Company AI Source Register. OWASP's RAG security guidance describes the wider retrieval pipeline; it does not certify your repair. Request a failure-case demo with one incorrect answer and its approved source. SimplSolutions can help scope the source and workflow improvement instead of assuming a different model solves every knowledge problem.

Put this to work this week

Select five incorrect answers from permitted evaluation evidence. For each, identify the first failing stage and the material claim it affected. Assign repair to the source, retrieval, applicability, generation or action owner as appropriate. Rerun the original case and a fresh neighboring question after the candidate fix. Keep unresolved governance issues visible. The outcome should be a supported employee instruction, not merely a rewritten paragraph that sounds more cautious.