The assistant went live on a Tuesday, indexed the intranet, and was asked by somebody about to resign what notice period applies. It answered three months, quoting a policy page from 2019. The answer has been one month since the 2022 handbook. The pilot review recorded a hallucination. The model had not hallucinated. It had read the intranet.
Nobody had taken the 2019 page down. There was never a reason to. The people reading it were checking a rule that applied to them, and they knew to go past it to the handbook. The ones who did not know asked somebody who did. The page does not link to the handbook, and the handbook does not link to the page.
The correction nobody wrote down
This page is superseded. That report double counts anyone who transferred mid-year. The Friday pack runs a month behind. The regional totals still stop at the pre-merger boundaries. None of it was written down, because none of it looked like work. It was what reading the intranet consisted of.
There was no villain in that. A written correction is a second document describing the errors in the first, which is an odd thing to propose in a meeting and an easy thing to decline. It would also have needed an owner, and the pages had none. They were written by somebody who has since left, in a team reorganised twice.
Somebody raised a ticket about the page in 2021. It was closed as no action required, because the handbook was correct.
Anyone who knew all this got the right answer. Anyone who did not asked a colleague who did, and got the right answer a little later. The documents stayed as they were, wrong in the same places for years, and the wrongness stayed a private arrangement between the staff and the pages.
The page was corrected every time it was read, and never once changed.
Elsewhere, in procurement
By the end of the month of the pilot every renewal in the queue was blocked, three buyers were raising exceptions by hand, and somebody was writing down, for the first time, which purchases the rule had never really covered.
A workflow was found too rigid, and a project opened to make it configurable.
Procurement policy says anything over ten thousand euro needs three quotes. It has said so for years, and it is accurate in the sense that somebody wrote it and somebody signed it. Renewals were never put through it, because there is nothing to get three quotes for when the question is whether to carry on with a supplier already installed. The policy had been incomplete for as long as anyone could remember, and it had made no difference, because nothing had ever applied it exactly.
A tool that cannot object
The pilot review came back with three recommendations: better retrieval, a fine-tuning pass, and a policy on where the assistant may be used.
None of those recommendations are new. An algorithm is one transformation among several in Metier’s account of information processing, and it arrives with a requirement older than any pilot: no data insight is worth taking seriously until it has been tested on new data.
When something goes wrong, an explanation usually goes in one of two places. One is a specific local cause: this operator, this batch, this machine, on this morning. The other is the system, whatever would still be there tomorrow if the person were swapped out. W. Edwards Deming spent his career moving the balance towards the second, away from blaming a person for a mistake. A tool is new, it is external, it has a vendor attached, and it cannot object.
The retrieval was improved. In March somebody about to resign read the 2019 page itself.
The intranet has been wrong since 2022, and it has cost nothing. The assistant was the first to take it at its word.