Provenance as a product constraint

Requiring every generated claim to resolve to a retrievable source is one sentence to write and an architecture to hold. Here is what it rules out.

There is a version of Recruiting.no that took a week to build. You take a job description, embed it, retrieve some profiles, hand the whole pile to a model, and ask for a ranked shortlist with reasons. It demos well. Every line of the output is fluent and specific, and roughly one line in eight is invented.

The invented ones are not obvious. That is the problem. A fabricated claim in a shortlist does not look like a fabricated claim — it looks like the other eleven, because it was generated by the same process that produced them.

So we wrote down a constraint: every claim about a candidate must resolve to a document we can show you. Not a citation the model produced. A record the system retrieved, stored, and can hand back.

That is one sentence. It cost most of the architecture.

What it rules out

Free-text generation as the output format. If the shortlist is prose, there is no seam between the part that came from a record and the part that came from the language model's sense of how sentences about engineers usually end. The output has to be structured, and each field has to carry a pointer.

Summarisation as a shortcut. The tempting move is to let the model read six sources and write one paragraph. The paragraph is better writing than any of the sources. It is also un-anchorable: you cannot point at a sentence in it and say which of the six it came from. Anything the reader sees has to be traceable to exactly one record, or it has to be visibly marked as our inference.

Silent enrichment. Every plausible-looking gap in a profile is an invitation to fill it in. A missing seniority level, an ambiguous title, a company nobody has heard of. The model will happily resolve all three, and it will be right most of the time, which is worse than being wrong all of the time — a system that is right eighty percent of the time trains you to stop checking.

A single relevance score. A number between zero and one is a claim with no source. It compresses evidence into something that cannot be argued with, which means it cannot be corrected either. We show the evidence and let the reader do the compressing.

What it costs

Recall, mostly. There are real candidates we cannot surface because the thing that makes them a match is not written down anywhere we can retrieve. A system willing to guess would find them. We think a shortlist you can check is worth more than a longer shortlist you cannot, but it is a trade, and pretending otherwise would be its own kind of fabrication.

It also costs latency and storage. Holding the source for every claim means holding the source for every claim — retrieval results are not transient context, they are part of the record, and they have to still be there when somebody opens the shortlist three weeks later.

Why it is the product

The interesting thing about this constraint is that it is not a feature. Nobody asks for provenance. What people ask for is to stop wasting a month on interviews that were never going to work, and provenance is the mechanism that makes the shortlist worth acting on without re-verifying it yourself.

A tool that produces work you have to check is not a tool. It is a first draft with extra steps, and the checking is the part that was expensive to begin with.