The recommendation arrives polished. The logic appears clean. The conclusion fits neatly on one slide.
Then someone asks what would have to be true for the recommendation to hold.
The room goes quiet.
AI may have done useful work. It may have compared more material, surfaced better alternatives, and turned scattered input into a clear position. Its involvement is not the problem.
The problem starts when the output is more convincing than the organisation’s understanding of it.
The answer is present. The explanation is not.
Fluency can hide the boundary
In 2002, Leonid Rozenblit and Frank Keil studied how well people believed they understood familiar mechanisms. Detailed explanation often exposed gaps that initial confidence had concealed.
Their research was not about AI or organisations. It does not establish what happens when a company uses a language model. It does offer a precise comparison: familiarity and confidence can feel like understanding before understanding has been tested.
AI can make that boundary harder to notice. It gives partial ideas structure, fills transitions, and produces language that sounds complete.
A complete sounding answer is still not the same as owned knowledge.
AI is useful because it expands the range
AI can challenge a first assumption, compare competing explanations, organise evidence, and help a team explore more options in less time.
That is a genuine gain.
The strongest use is not simply to obtain a final paragraph faster. It is to make the reasoning easier to inspect and the alternatives easier to discuss.
An answer should remain open to correction, context, and professional judgement. Good phrasing should not close a question that the business still needs to examine.
Ownership appears when the case changes
The first case rarely reveals whether the reasoning has taken root. The next case does.
What happens when a source becomes unreliable? When a customer behaves differently? When an exception reaches the process? When an accountable owner disagrees?
Can the business identify which assumption changed and adjust the recommendation?
If only the output can be reproduced, the dependence remains hidden. If the rationale can be reconstructed, AI becomes an extension of organisational capability.
Keep the rationale inside the business
This does not require access to a model’s private internal reasoning. It requires the company’s own decision basis to remain visible.
Five questions provide a practical test:
- Which sources support the answer?
- Which assumptions shape it?
- Where are its limits and exceptions?
- Who can challenge it with subject expertise?
- What change would make the answer invalid?
A company that can answer those questions is not using less AI. It is using AI with greater control.
An answer belongs to the business only when the business can test its rationale and change it.
Patrick Thole, Disruption Dynamics, Zürich
disruption-dynamics.ch