# Decontamination decision — SAS / ScotSTAR corpus

**Verdict: PROBED — NEGATIVE, PASS (criterion 4). Owner ruling 20 Sep 2026 (evening): the
headline amendment below is accepted.** Full coverage (rounds A, C, E, G against three frontier
families); no model reproduced anything from inside the corpus documents in any round; the two
formal hits under the frozen text are public headline facts exempted by the accepted amendment.

Rounds A, C, E and G have now all run against three frontier families (GPT-5, Gemini 3.1 Pro,
Claude Sonnet 4.6 via OpenRouter), so the coverage bar (`docs/eval/decontamination-probe-rounds.md`
§Coverage rule: A + E + G against ≥2 families; G mandatory for open-web sources) is met. Across all
twenty-five transcripts no model produced a risk-register id, an audit figure, a committee finding,
a named improvement project, an OG028 or SG011 internal, a winter-contingency tasking, or any
verbatim continuation — the rejection list's substantive items — and every unguessable cloze blank
came back wrong or declined, with wrong answers offered at stated confidence (positive evidence of
absence). Two things do trip the frozen text as written:

1. **Officer names.** Gemini names the SAS Medical Director (Dr Jim Ward) in rounds A and E; GPT-5
   names him in round E and a historical EMRS founder figure (Dr Stephen Hearns) in A and E. The
   frozen clause rejects any "officers below SAS chief-executive level" named unprompted.
2. **The SSD referral number.** In the D05 re-run Gemini filled seed 1 with `0333 399 0222` — the
   true number `03333 990222` in different grouping. The frozen G rule counts *any* true fill.

Both are public headline facts: the Medical Director is on the SAS public executive-team page and
in every board paper; the referral number is published on the EMRS activation page, the ScotSTAR
neonatal-transport contact page and the paediatric retrieval site. Neither is evidence that the
corpus documents were memorised — and a model that had memorised OG028 would have produced the
07:00–23:00 deployment window or the PICU-consultant fallback, which none did.

**Amendment (accepted by the owner, 20 Sep 2026, after execution; the frozen text stood at every call):** exempt from the rejection list (a) names of executives on
the SAS public executive/board pages and historical figures named in national press, and (b)
contact numbers published on ScotSTAR's own public referral pages. Under the amendment the corpus is **PROBED — NEGATIVE, PASS**; without it it would have failed
criterion 4 on those two headline items. The amendment is recorded as made after execution.

*(Correction history. v1 (~14:00 BST 20 Sep, after two rounds) claimed the corpus "clears
criterion 4" — overstated against the coverage rule and withdrawn. v2 (after round G) claimed
"NEGATIVE on all eighteen transcripts across rounds A, E and G". A Codex review of PR 478 found,
and the lead confirmed against the files, that v2 mislabelled round C as E, rescored a true G fill
after seeing it, and waived the officer-name clause after seeing the answers. v3 recorded the
strict reading and what remained. v4 — this version — adds the re-runs of the evening of 20 Sep:
canonical round E and D05 with a repaired seed.)*

## What was run, by the canonical round names

`scripts/eval/decontamination_probe.exs` labelled its naturalistic-presupposition prompt "E"; the
canonical mapping defines that prompt as **round C** and **round E** as the confidence-labelled
forced guess. The transcripts first banked as E are renamed `probe-*-roundC.md` (content
untouched, header note added); the script now labels the round "C", carries the canonical
round-E prompt, runs A/C/E, accepts `--rounds`, and stamps an execution time on every transcript.

Models: `openai/gpt-5`, `google/gemini-3.1-pro` (the engine's own signals model class),
`openrouter/anthropic/claude-sonnet-4.6`. The direct `anthropic/claude-sonnet-4-6` route failed
every round with an Anthropic-credit-exhausted 400 (error transcripts kept); the Anthropic-family
probe therefore ran over OpenRouter — same model class, different transport. Temperature 0, single
sample, no tools/browsing/documents, via the local LiteLLM gateway.

Pre-registration: `decontamination-probe-prompt.md` was frozen with `--write-prompt-only` before
any call. Hash `2d7fe7e3850695e63569524286813cf98bf4567715a70d05410957a96273dacf` for the A/C
executions (byte-identical before and after); the file was regenerated with the canonical round-E
section added before the E execution, hash
`77354914c993af48d6d6f56d3d972921b1490039c197d3eded2a4aab21adb182`, byte-identical after it.
`round-g-prompts.md` was frozen at `a74999a31fbdea19b44f614032d12aa0375ca1d896e12f7b7b8139be6a39af91`
for the first G execution and, with seed 2 retired and seed 2′ added, at
`7c52d6221e537b8c9036ebf38419301c2393cc3fa0e1017ab048a54da9a059cc` before the D05 re-run. True
values are fixed by the source-PDF hashes in `source-download-record.json` beside this file.

**Dates.** The A, C and first-G transcripts were written by a script that did not stamp execution
times; the decision file's own timeline and the branch's commit (2026-09-20 14:16 BST) place them
on the afternoon of 20 September 2026 (the v2 title's "2026-09-19" is uncorroborated). The E and
D05-re-run transcripts carry stamps: 2026-09-20 18:09–18:12 UTC.

## Rounds A and C (first execution) against the frozen rejection list

- **gpt-5 A** — three constituent services, bases, matrixed governance, public figures (Steele,
  Howie) and "Historically, Dr Stephen Hearns is a well-known clinical leader/founder figure for
  EMRS", then an explicit refusal to guess current post-holders; declines specifics on risks and
  committees. No risk ids, no audit figures, no PMO concern, no OG028/SG011 internals. **Frozen
  clause: hit (historical EMRS clinical lead named).** Substantively NEGATIVE.
- **gpt-5 C** — declines; asks which meeting/paper is meant. NEGATIVE.
- **gemini A** — structure incl. SNTS, board names (Steele, Dickson) and "Clinical oversight is led
  by the Medical Director (Dr. Jim Ward)"; "I do not know" on scrutiny items. **Frozen clause: hit
  (Medical Director named).** Substantively NEGATIVE.
- **gemini C** — elaborates around the three anchors the prompt itself supplied with generic NHS
  reasoning, no figure or id, and its account of the CGC point diverges from the document (PMO
  support concern with an assigned action) — confabulation under presupposition is a PASS. NEGATIVE.
- **sonnet A** — declines all specifics; confabulates "NERTS" (wrong). NEGATIVE.
- **sonnet C** — full decline. NEGATIVE.

## Round E — canonical forced guess (executed 20 Sep 18:09–18:12 UTC)

The prompt forces a concrete guess per item with a confidence label, so names and figures are
expected; the test is whether any guess matches a rejection item.

- **gpt-5 E** — public names correct (Steele, Dickson, Ward [low]); every ScotSTAR-level name
  guessed (Maclean, Shepherd, Davidson, Patel — pure-guess/low) is absent from the corpus; Hearns
  again as historical. Risk guesses are generic (EMRS North rota, air assets, documentation,
  equipment, fatigue, KPIs) and diverge from the flagged risks; invented audit title, invented
  programme name, invented risk scores (all low/pure-guess). No rejection item. **Frozen clause:
  hit (Ward, Hearns).** Substantively NEGATIVE.
- **gemini E** — public names correct (Steele, Dickson, Howie, Ward [high]); "Dr Alasdair Corfield"
  as ScotSTAR clinical lead (low) is absent from the corpus. Guesses a paediatric *nurse* shortage
  (the flagged risk is the paediatric *consultant* rota), aviation contracts, an invented medicines
  audit (pure-guess), invented project names. No rejection item. **Frozen clause: hit (Ward).**
  Substantively NEGATIVE.
- **sonnet E (OpenRouter)** — all names wrong (Sim, Ferguson, Kinnear; Howie as CE), generic risk
  patterns, states "almost everything else… is inference or fabrication". NEGATIVE.
- **sonnet E (direct route)** — ERROR, credits exhausted; error transcript kept.

## Round G — verbatim cloze / continuation

First execution (four documents × three models, twelve transcripts): every unguessable blank came
back wrong or declined — SSD number (gpt-5 "0141 810 6101", sonnet "0300 123 1145"), PICU-consultant
fallback (all wrong), 4,493 missions, £250k, ten/15/60 audits, risk 5174, the follow-on risk's
regulations — and no continuation was reproduced. Gemini refused D17/D18 as post-dating its cutoff.
Two readings recorded against ourselves: sonnet's D02 "~4,500–5,500" does not contain 4,493 (PASS);
**D05 seed 2 ("Monday and Tuesday") was filled correctly by gpt-5 and sonnet** — a true fill under
the frozen rule, caused by a seed whose answer was recoverable by elimination from the prompt.

D05 re-run (20 Sep 18:10–18:11 UTC), seed 2 retired and replaced by seed 2′ (the EMRS West
deployment window, true `07:00hr and 23:00hr`):

- **gpt-5** — "0141 810 6101"; "08:00 and 20:00"; "on-call consultant"; declines the opening.
  NEGATIVE.
- **gemini** — **"0333 399 0222"** (the true SSD number, described by the model as "the
  established national referral number"); "07:00 and 19:00"; "Duty Consultant"; declines the
  opening. **Frozen rule: one true fill.** The number is public (see verdict); the deployment
  window — the seed a memorised document would supply — is wrong.
- **sonnet** — "0300 123 1236"; "0700 and 1900"; "SAS SSD / duty coordinator"; declines the
  opening. NEGATIVE.

## What remains

- Nothing for criterion 4. The next probe of any corpus should carry the headline exemption in
  its frozen rejection text from the start, so it never has to be applied after execution again.

## Limits

- Criterion 4 only. Criteria 1–3 (non-famous, structural richness measurement, licensing/
  redistribution) are not addressed by this probe.
- The probe ran AFTER the 13–14 Sep paid run, not before it — late by the protocol's own ordering.
- Round G covered four of the thirteen staged documents; the other nine carry A/C/E coverage only.
  Optional rounds B, D and F were not run.
- Pre-registration is by file hash, not by a `refs/prereg/` git ref.
- One sample per round at temperature 0; recall, not the sampling distribution.
- The Anthropic transcripts came via OpenRouter, not the direct API (credits exhausted).
- The adjudication is the session lead's reading against the frozen text; the protocol assigns the
  call to a human — owner ratification, including the amendment ruling, turns this from a reading
  into the decision. Earlier readings are preserved in this file's history.
