The Confident Wrong Answer

Jason Duke, Founder, Kronaxis

Tag: Intelligence

Behind every sanctions screen and every know your customer check sits one dull, decisive question: are these two records the same real world person or company? Miss the join and you miss the risk. Force a wrong one and you flag the innocent. So we benchmarked our own answer to it on a public dataset, tested the hardest cases on purpose and this is what the numbers actually said, including the number that stung.

What we built

The usual way to ask a model whether two records match is to make it write back "yes" or "no". You get a bare verdict and no idea how sure it was. We changed the question. Instead of reading the word the model writes, we read the probability it assigns, in a single pass, to each answer before it commits. A pair does not come back as "match". It comes back as "match, probability 0.98" or "match, probability 0.55". Same model, same one call, but now every decision carries the model's own confidence rather than hiding it.

That one change turns a yes or no into a dial and the dial is what lets you decide, per case, whether the machine may act alone or whether a person should look. It runs on a locally hosted open model, so no record ever leaves our infrastructure to be scored.

The test

We used the public OpenSanctions Pairs dataset, which ships hundreds of thousands of record pairs each labelled by hand as the same entity or not. We did not cherry pick the easy ones. We pulled the hard slice: the pairs where the names half overlap, a shared surname or a partial company name, the band where a cheap matcher cannot tell and a lazy one guesses. Four hundred of those, labelled, 88 per cent of them genuine matches. That base rate matters, because it sets the bar: a system that blindly shouts "match" at everything scores 88 per cent accuracy on this slice. Beat that or you have added nothing.

The number that stung

Taken as a plain yes or no, the model did not beat it. Its verdicts scored 81 per cent accuracy, below the do nothing baseline of 88. It was extremely precise, 98.9 per cent of its "match" calls were right, but it was cautious to a fault, missing about a fifth of the true matches by calling them "no". If we had shipped the bare verdict, we would have shipped something worse than a coin that always says yes. That is the honest result and it is the one most write ups quietly leave out.

The number that mattered

The confidence told a completely different story. Ranked by the probability the model assigned, true matches sat above false ones almost perfectly: an area under the curve of 0.947, where 0.5 is useless and 1.0 is flawless. The model knew which pairs were real. It just could not express that in a single yes or no. The information was in the doubt and the doubt is exactly what the yes or no throws away.

One more thing the confidence exposed, which changed our design. The model is trustworthy when it says yes and untrustworthy when it says no. Look at where its answers fell against the truth:

A confident yes is gold. A confident no, on these hard pairs, is wrong more often than not. So our resolver never auto rejects. It auto accepts the confident matches and sends everything else, including its own confident refusals, to a person, ranked by probability so the likeliest matches surface first.

The operating point we ship

Put together, that is a resolver that clears about seven in ten of the hard pairs on its own, at 98.9 per cent precision, three wrong accepts in nearly three hundred and routes the remaining three in ten to a human who sees them worst first. Not a leaderboard win. Something a regulated buyer values more: the machine acts only where it is provably safe, the uncertain cases have a name and a person on them and every automated decision is one you can show was safe to automate. You set the threshold on your own labelled sample and you know, with a number, what you are trusting the machine to do.

The verdict was the wrong output to measure. The product is the calibrated confidence and the routing it allows.

Why we published the losing number too

Because the discipline that runs through everything we build is provable, not plausible and you cannot claim that while hiding the result that does not flatter you. The bare verdict lost to a coin. The confidence beat it decisively. Both are true, both are in the open and together they are the reason our resolution is something you can put in front of an auditor rather than something you have to take on faith. Entity resolution is where that discipline earns its keep, because it is the join the whole picture hangs from.