An evaluation nobody can choose after the fact.

1 · Roles

• The registrant — an organisation that registers a protocol about a system (its own or another's), supplies the pool of items, names the judges, and pays. It never touches an item after registration.
• The registry — holds the pool sealed, commits and reveals the seed, draws the sample, collects verdicts, computes the bound, signs the report, writes the log, serves the mark. It reads an item only to show it to a judge.
• The judges — people who are not concerned (staff, a panel), the people the answers are about (first person), or a machine judge the protocol names. A judge sees their items and never the system's name, never whether an item is a decoy, never another judge's items.
• The verifier — anyone. With the report, the public key, the log and the recipes below, the verifier needs no word from the registry.

2 · What is committed, and when

1. Registration. The registrant sends the protocol: the system (slug, name, version), the mode (third party or first person), the judges' source, the sample size n, the number of decoys, the bound's target alpha and confidence 1 − delta, and the pool of items { prompt, answer | null, context?, subject? }. The registry seals every item (sealed under a key the base never sees, bound to its protocol and its item), computes the pool digest, draws a 256-bit seed, commits it by its SHA-256, writes the canonical protocol (below), hashes it, anchors the hash on OpenTimestamps, and appends a protocol line to the log. From this moment the protocol's digest, pool digest, seed commitment, sizes and parameters cannot change; a corrected protocol is a new protocol.
2. The draw. Once. The registry reveals the seed, checks it against its commitment, draws the sample and the decoys with the recipe below, fixes the order the judges will see, and appends a draw line carrying the seed. A second draw is refused.
3. Judging. Each judge receives a link that opens on their items. Three verdicts per item: correct, incorrect, correct but should not have been said to this person. An abstention of the system (a null answer) is not judged. A second pass may reopen the same judges later, blind again, on a third of their items.
4. Closing. Once. The score is computed and written, the report signed, a report line appended with the report's digest. A closed protocol never reopens.
5. The mark. Computed every time it is asked for, never stored: measured while the bound holds and the measurement is younger than 90 days under a standing mark; expired, withdrawn or no mark otherwise. A mark line when it stands, a withdrawal line when it stops; never two in a row.

3 · The canonical protocol

The hashed document is the canonical JSON — keys sorted at every depth, compact — of:

{ "version": 1, "protocol": {
    "id", "organisation", "system": { "slug", "name", "version" }, "title",
    "mode": "third_party" | "first_person", "raters_source": "staff" | "panel" | "subjects" | "mixed",
    "n", "decoys", "alpha", "delta",            // alpha and delta as decimal strings
    "pool_count", "pool_digest", "seed_commitment",
    "verdicts": ["correct", "incorrect", "not_to_them"],
    "floor_judged_items": 30,
    "bound": "exact binomial (Clopper–Pearson) upper bound on the share judged wrong among the answers given; Hoeffding beside it",
    "registered_at" } }

Pool digest. pool_digest = SHA-256 of the per-item digests joined by '\n', in pool order; an item's digest = SHA-256 of the canonical JSON of {prompt, answer, context, subject} (keys sorted, compact; nulls kept). The registrant can recompute it from its own items; the registry never shows the pool to anyone.

4 · The draw, reproducible

stream = HMAC-SHA256(seed bytes, 8-byte big-endian counter); uniform(n) by rejection sampling on 4-byte big-endian words; order = Fisher–Yates over 0…pool_count−1 from the top with uniform(i+1); sample = the first n; decoy prompts = the next `decoys`, each shown with the answer of the k-th sampled item (third-party) or the first sampled item of the same subject (first-person); positions = a second Fisher–Yates over sample and decoys from the same stream; ref = first 24 hex characters of SHA-256(protocol_id ':' idx ':' decoy_of or '-').

A verifier who holds the seed (public after the draw) and the pool's size recomputes which indices were drawn and in which order; a registrant who holds its pool recomputes which items. A decoy shows a drawn prompt with another drawn item's answer — in first person, another item of the same person — so that a judge who calls it correct was not reading.

5 · The score

• Judged: drawn items, decoys left out, with at least one first-pass verdict. An item's verdict is the majority for correct; otherwise the fault most named; a tie is a fault.
• Wrong: judged incorrect or not to them, counted apart and never merged into one number.
• Bound: the smallest error rate under which seeing at most wrong errors in judged would be as unlikely as delta — the one-sided Clopper–Pearson upper bound — with Hoeffding's wrong/judged + √(ln(1/delta) / 2·judged) beside it. The guarantee holds when the exact bound is under alpha.
• Abstention: the share of drawn items the system declined to answer, printed beside the bound, never inside it.
• Withheld: fewer than 30 judged items; or the judges caught fewer than 50% of at least five decoys. A withheld measurement publishes no number — not a number with a caveat, no number.
• Agreement: between judges on the same item (percent and Cohen's κ on the first two), and of each judge with themselves across the second pass. Printed when they exist, absent when they do not.

What the registry certifies is that nothing was chosen after the commitment: not the items, not the judges, not the result. What it does not certify is that the pool represents the system's use: the protocol says how the pool was built, and the reader judges that.

6 · The report, the log, the roots

Signature. Remove `signature`, sort every object's keys at every depth, serialise as compact JSON (no spaces), and verify the Ed25519 signature of that UTF-8 string against `signature.key` (SPKI, base64). Any Ed25519 implementation will do; no call to Selione is needed. The key is printed below and on the Standard page.

Log. For each line: digest = SHA-256 of the payload as canonical JSON (keys sorted at every depth, compact); entry_hash = SHA-256 of prev_hash + ':' + digest + ':' + kind + ':' + at (ISO 8601 UTC with milliseconds). The first line's prev_hash is 64 zeros. A day's last entry_hash is the root anchored on OpenTimestamps; verify the proof with the public `ots` tool.

7 · Words

A registry following this specification says measured, withheld, expired, withdrawn. It never says certified, accurate, safe or compliant: it did not test the system, it made sure nobody could choose what was shown about it.

8 · A clause for buyers

A procurement officer, an insurer or a regulator who wants evidence nobody chose can ask for it in one paragraph. This one is free to copy:

“Before producing any evaluation result it intends to rely on, the Supplier shall register the evaluation protocol — the system and version, the pool of items, the judges, the sample size and the bound — with an independent registry implementing the Open Evaluation Register specification, version 1 or later, such that the protocol is hashed and anchored before results exist and the sample is drawn by the registry from the committed pool. The Supplier shall provide the registry's signed report and the public log lines, and shall not rely on any evaluation result that was not so registered.”

9 · Reviewed — the reasoned human decision, and the person's receipt

The same Register, used for a different document: not a measurement of a system, but a decision a human took about a person after a machine recommended it. What the law judges in such a decision — the reviewer's authority, their competence, that all the data was considered including what the person said, and that the review was not a token gesture — is satisfied here by construction, and leaves a proof anyone can verify without the registry. The registry decides nothing: it is the processor of the organisation's own reviewers.

1. The policy, committed first. Which class of decisions, which roles have authority to reverse, the minimum time per case, the share of golden cases, the days the person has to be heard, where the person can go next. Canonical JSON, SHA-256, a policy line in the log, an OpenTimestamps anchor; frozen by the base.
2. The reviewer and their key. Role, authority, a training the organisation's owner attests, an anonymous reference, and a passkey made on the device they review on (ES256 / P-256, user verification required). The public key is published in a reviewer line; a synced passkey is recognised by its flags and marked as such.
3. The person, heard first. Before the decision, the person receives a door. They write the facts, the dates, the documents, what they ask for; a guide may ask up to three clarifying questions in their language and never writes for them. The statement is assembled from their own words by a function without a model, sealed, and its digest written as a statement line.
4. The room. Every section must be opened — the recommendation and its confidence, each input, the statement — and the openings are part of the record. The policy's minimum time runs server-side. The reasons are typed by the reviewer in a template of five elements (what was considered; what the person said and how it was weighed; the rule applied; what would change the outcome; what happens next) plus an answer to each thing the person asked for. No model writes a word. A role without authority cannot close a case.
5. The signature. The canonical record is frozen and hashed; the passkey signs with the digest as its challenge; the registry verifies (type, challenge, origin, relying party, user verification, counter, ECDSA), countersigns with its Ed25519 key, appends a decision line carrying the digest and the key's fingerprint, and the day's root is anchored. Offline verification needs the record, the reviewer's public key and the registry's published key; the recipe is printed in the contract.
6. The receipt. The person's page says: a [role] with authority to reverse reviewed your file on [date], read your statement, spent [n] minutes; here is the outcome, the reasons in plain words, what would change it, and where to go next — with the digest, the two signatures and the log line. Always free. The public verification endpoint returns the proof for a digest and never a word of the record.
7. Oversight, private by default. Volumes, disagreement with the machine with exact binomial bounds, time spent, golden cases caught, delays, reviewers as roles and references. One signed bundle when an authority asks. Never a public statistic: an organisation is not its own notary, and the registry is not its judge.

A clause for buyers of decisions, free to copy: “Where a decision about a natural person is recommended by an automated system, the Supplier shall ensure that it is taken by a person with authority to depart from the recommendation, after that person has considered all relevant information including any statement the data subject chose to make, and shall provide the data subject with a receipt stating the role and authority of the reviewer, whether the statement was considered, the time spent, the outcome and its reasons, and the avenues of recourse — in a form whose authenticity can be verified by any third party without recourse to the Supplier.”