How confident is each number, and what confidence actually means

Every usage event in your archive carries a confidence: exact, strong, or derived. It travels with the row into your reports and into the sync payload, so a number that had to be worked out never gets to look like one that was read straight off.

The label is easy to misread, so it is worth being precise about what it claims.

It is about identity, not arithmetic

Confidence does not mean "we are fairly sure this was 4,000 tokens". The token counts come out of the log as written. Confidence answers a different question:

If I read this file again, can I tell that this event is the same event I already have?

That matters because tokenstat scan runs repeatedly, often hourly, over files the tools are still appending to. Every scan re-reads material it has seen before. The only thing standing between you and a total that doubles every hour is being able to recognise a request you have already recorded.

The three levels

exact is the comfortable case and the common one for tools that log a request id.

strong has no provider id to lean on, so identity is assembled from fields the source already writes: timestamps, model, counters, session. Unique by construction rather than by declaration. In practice it behaves like exact.

derived is the honest one. There was nothing stable to key on, so the event is identified by where it sat in a series. Re-scan the same file and it lands in the same place, so ordinary repeated scans are safe. Rewrite that file in a different shape and the identity moves, which can leave a duplicate or a gap.

What to do with it

For almost everything, nothing. Scan on a schedule, read your reports, and the levels take care of themselves.

Two moments where it is worth a look:

A total moved and you cannot see why. derived rows in a source that rewrote its logs are the first thing to check. tokenstat doctor is the command for this, and reconciling the archive is what it is for.

You are comparing against a vendor's billing page. A pile of derived events is one of several reasons two numbers legitimately differ. Why your numbers disagree with a vendor's dashboard covers the rest, most of which have nothing to do with confidence.

The rule underneath all of this

Confidence exists for the same reason as another rule that runs through the whole tool: an unavailable number is reported as unavailable, never as zero.

A zero you cannot distinguish from "no data" is the worst possible output, because it looks like an answer and you will act on it. Labelling how each row was identified is the same instinct applied one level down. A count that had to be inferred is still shown, and it is shown wearing a label.

Related

• Which tools tokenstat reads, and what each one records lists every source and where its numbers live.
• What tokenstat counts, and where the numbers come from covers the counting itself.