At an events and project agency, up to twenty supplier quotes came in by email per project. They were saved twice, in the mail folder and on the server, and then typed over line by line into the costing. A large project runs to seventy or well over a hundred lines.
The plan was clear. Mail comes in, system reads the project number out of the mail, links the quote to the right project, done.
The check nobody does
Before we started building, we opened eleven real emails to see how that project number appeared in them.
It was not there. Not in a single one.
Eleven out of eleven. Not "usually not" or "often forgotten" — zero out of eleven. Suppliers send their quote with their own reference, not with yours.
If we had skipped that check, we would have built a link that works fine on the test data and does nothing in practice. That is not rare. That is the standard way automation fails.
Why this keeps happening
When a process is mapped out, everybody describes how it is supposed to go. Nobody describes what actually happens in the field, because you only know that once you look.
The project managers knew perfectly well there was no project number in there. They matched it themselves, on content, without thinking about it. Ask them "is the project number in the email", and they say yes — because in their heads it belongs there.
That is why we no longer ask. We look.
What we built instead
Matching on content rather than on a number. The AI reads the PDF and the mail text, recognises the supplier, the amounts and the descriptions, and proposes the right project. A person confirms.
Of twelve quotes examined, ten were fully processed into a file that can be loaded straight into the project system. For every quote that states a total itself, our sum matches to the cent.
That last bit is the most important sentence: it was tested on their own material, not on an example.
The rule of thumb
Before you automate anything, pull out ten real cases. Not ten someone picked out for you — ten at random from last month's pile.
Is the field you want to match on there in all ten? Then you can build. Is it there in seven out of ten? Then you build a tool that produces manual work in thirty percent of cases, plus the irritation that it "was supposed to be automated".
Is it there zero times? Then you have just saved yourself a week's work today.