Recorded 20 September 2026. Four check types across three model families. Full prompts, answers and failed attempts are retained below.
The prior-knowledge check (criterion 4) asks whether apparent findings could be recalled from the public web rather than derived from the documents. ScotSTAR passed it under a rule I amended after the probe ran. The 20 September 2026 decision records that every required round was run.
The check: GPT-5, Gemini 3.1 Pro and Claude Sonnet 4.6 answered without tools, browsing or supplied case files. Four approaches: direct question; casual notes check; forced guess with confidence labels; fill-in-the-blank and continuation.
The result: no substantive rejection-list details were reproduced: risk-register IDs, audit figures, committee findings, named projects, policy internals or verbatim continuations.
The rule change: public officer names and the published referral number were recognised. I excluded these after the run. They would have caused a fail under the original rule.
- One of four corpus criteria; being little-known, structural richness and licensing are separate.
- Run after the earlier ScotSTAR analysis (13–14 September), rather than before.
- Blank tests covered four of the thirteen documents in that earlier library.
- One sample per prompt, at temperature 0.
A negative probe does not prove that the models never encountered the documents.
How the blank test changedOpen +Close −
The prompts and source answers were fixed by file hashes before the calls. An initial weekday question could be solved from the prompt; it was retired and replaced with a deployment-window question. All three models got that window wrong. Original answers and the re-run remain below. Claude’s answers came via OpenRouter; failed direct-route attempts are retained separately.
Decision and scoring history · Round definitions · Protocol and its limits
Record-count discrepancy: the decision says 25 transcripts; the pinned branch contains 27 (24 answers and 3 failed attempts). All 27 are retained.
Decision, criteria and source recordsOpen +Close −
These files are retained verbatim, including the decision’s correction history. Model answers are evidence of what was elicited, not verified accounts of the organisation. Failed attempts are not model answers.
DECONTAMINATION-DECISION.mdOpen +Close −
# Decontamination decision — SAS / ScotSTAR corpus
**Verdict: PROBED — NEGATIVE, PASS (criterion 4). Owner ruling 20 Sep 2026 (evening): the
headline amendment below is accepted.** Full coverage (rounds A, C, E, G against three frontier
families); no model reproduced anything from inside the corpus documents in any round; the two
formal hits under the frozen text are public headline facts exempted by the accepted amendment.
Rounds A, C, E and G have now all run against three frontier families (GPT-5, Gemini 3.1 Pro,
Claude Sonnet 4.6 via OpenRouter), so the coverage bar (`docs/eval/decontamination-probe-rounds.md`
§Coverage rule: A + E + G against ≥2 families; G mandatory for open-web sources) is met. Across all
twenty-five transcripts no model produced a risk-register id, an audit figure, a committee finding,
a named improvement project, an OG028 or SG011 internal, a winter-contingency tasking, or any
verbatim continuation — the rejection list's substantive items — and every unguessable cloze blank
came back wrong or declined, with wrong answers offered at stated confidence (positive evidence of
absence). Two things do trip the frozen text as written:
1. **Officer names.** Gemini names the SAS Medical Director (Dr Jim Ward) in rounds A and E; GPT-5
names him in round E and a historical EMRS founder figure (Dr Stephen Hearns) in A and E. The
frozen clause rejects any "officers below SAS chief-executive level" named unprompted.
2. **The SSD referral number.** In the D05 re-run Gemini filled seed 1 with `0333 399 0222` — the
true number `03333 990222` in different grouping. The frozen G rule counts *any* true fill.
Both are public headline facts: the Medical Director is on the SAS public executive-team page and
in every board paper; the referral number is published on the EMRS activation page, the ScotSTAR
neonatal-transport contact page and the paediatric retrieval site. Neither is evidence that the
corpus documents were memorised — and a model that had memorised OG028 would have produced the
07:00–23:00 deployment window or the PICU-consultant fallback, which none did.
**Amendment (accepted by the owner, 20 Sep 2026, after execution; the frozen text stood at every call):** exempt from the rejection list (a) names of executives on
the SAS public executive/board pages and historical figures named in national press, and (b)
contact numbers published on ScotSTAR's own public referral pages. Under the amendment the corpus is **PROBED — NEGATIVE, PASS**; without it it would have failed
criterion 4 on those two headline items. The amendment is recorded as made after execution.
*(Correction history. v1 (~14:00 BST 20 Sep, after two rounds) claimed the corpus "clears
criterion 4" — overstated against the coverage rule and withdrawn. v2 (after round G) claimed
"NEGATIVE on all eighteen transcripts across rounds A, E and G". A Codex review of PR 478 found,
and the lead confirmed against the files, that v2 mislabelled round C as E, rescored a true G fill
after seeing it, and waived the officer-name clause after seeing the answers. v3 recorded the
strict reading and what remained. v4 — this version — adds the re-runs of the evening of 20 Sep:
canonical round E and D05 with a repaired seed.)*
## What was run, by the canonical round names
`scripts/eval/decontamination_probe.exs` labelled its naturalistic-presupposition prompt "E"; the
canonical mapping defines that prompt as **round C** and **round E** as the confidence-labelled
forced guess. The transcripts first banked as E are renamed `probe-*-roundC.md` (content
untouched, header note added); the script now labels the round "C", carries the canonical
round-E prompt, runs A/C/E, accepts `--rounds`, and stamps an execution time on every transcript.
Models: `openai/gpt-5`, `google/gemini-3.1-pro` (the engine's own signals model class),
`openrouter/anthropic/claude-sonnet-4.6`. The direct `anthropic/claude-sonnet-4-6` route failed
every round with an Anthropic-credit-exhausted 400 (error transcripts kept); the Anthropic-family
probe therefore ran over OpenRouter — same model class, different transport. Temperature 0, single
sample, no tools/browsing/documents, via the local LiteLLM gateway.
Pre-registration: `decontamination-probe-prompt.md` was frozen with `--write-prompt-only` before
any call. Hash `2d7fe7e3850695e63569524286813cf98bf4567715a70d05410957a96273dacf` for the A/C
executions (byte-identical before and after); the file was regenerated with the canonical round-E
section added before the E execution, hash
`77354914c993af48d6d6f56d3d972921b1490039c197d3eded2a4aab21adb182`, byte-identical after it.
`round-g-prompts.md` was frozen at `a74999a31fbdea19b44f614032d12aa0375ca1d896e12f7b7b8139be6a39af91`
for the first G execution and, with seed 2 retired and seed 2′ added, at
`7c52d6221e537b8c9036ebf38419301c2393cc3fa0e1017ab048a54da9a059cc` before the D05 re-run. True
values are fixed by the source-PDF hashes in `source-download-record.json` beside this file.
**Dates.** The A, C and first-G transcripts were written by a script that did not stamp execution
times; the decision file's own timeline and the branch's commit (2026-09-20 14:16 BST) place them
on the afternoon of 20 September 2026 (the v2 title's "2026-09-19" is uncorroborated). The E and
D05-re-run transcripts carry stamps: 2026-09-20 18:09–18:12 UTC.
## Rounds A and C (first execution) against the frozen rejection list
- **gpt-5 A** — three constituent services, bases, matrixed governance, public figures (Steele,
Howie) and "Historically, Dr Stephen Hearns is a well-known clinical leader/founder figure for
EMRS", then an explicit refusal to guess current post-holders; declines specifics on risks and
committees. No risk ids, no audit figures, no PMO concern, no OG028/SG011 internals. **Frozen
clause: hit (historical EMRS clinical lead named).** Substantively NEGATIVE.
- **gpt-5 C** — declines; asks which meeting/paper is meant. NEGATIVE.
- **gemini A** — structure incl. SNTS, board names (Steele, Dickson) and "Clinical oversight is led
by the Medical Director (Dr. Jim Ward)"; "I do not know" on scrutiny items. **Frozen clause: hit
(Medical Director named).** Substantively NEGATIVE.
- **gemini C** — elaborates around the three anchors the prompt itself supplied with generic NHS
reasoning, no figure or id, and its account of the CGC point diverges from the document (PMO
support concern with an assigned action) — confabulation under presupposition is a PASS. NEGATIVE.
- **sonnet A** — declines all specifics; confabulates "NERTS" (wrong). NEGATIVE.
- **sonnet C** — full decline. NEGATIVE.
## Round E — canonical forced guess (executed 20 Sep 18:09–18:12 UTC)
The prompt forces a concrete guess per item with a confidence label, so names and figures are
expected; the test is whether any guess matches a rejection item.
- **gpt-5 E** — public names correct (Steele, Dickson, Ward [low]); every ScotSTAR-level name
guessed (Maclean, Shepherd, Davidson, Patel — pure-guess/low) is absent from the corpus; Hearns
again as historical. Risk guesses are generic (EMRS North rota, air assets, documentation,
equipment, fatigue, KPIs) and diverge from the flagged risks; invented audit title, invented
programme name, invented risk scores (all low/pure-guess). No rejection item. **Frozen clause:
hit (Ward, Hearns).** Substantively NEGATIVE.
- **gemini E** — public names correct (Steele, Dickson, Howie, Ward [high]); "Dr Alasdair Corfield"
as ScotSTAR clinical lead (low) is absent from the corpus. Guesses a paediatric *nurse* shortage
(the flagged risk is the paediatric *consultant* rota), aviation contracts, an invented medicines
audit (pure-guess), invented project names. No rejection item. **Frozen clause: hit (Ward).**
Substantively NEGATIVE.
- **sonnet E (OpenRouter)** — all names wrong (Sim, Ferguson, Kinnear; Howie as CE), generic risk
patterns, states "almost everything else… is inference or fabrication". NEGATIVE.
- **sonnet E (direct route)** — ERROR, credits exhausted; error transcript kept.
## Round G — verbatim cloze / continuation
First execution (four documents × three models, twelve transcripts): every unguessable blank came
back wrong or declined — SSD number (gpt-5 "0141 810 6101", sonnet "0300 123 1145"), PICU-consultant
fallback (all wrong), 4,493 missions, £250k, ten/15/60 audits, risk 5174, the follow-on risk's
regulations — and no continuation was reproduced. Gemini refused D17/D18 as post-dating its cutoff.
Two readings recorded against ourselves: sonnet's D02 "~4,500–5,500" does not contain 4,493 (PASS);
**D05 seed 2 ("Monday and Tuesday") was filled correctly by gpt-5 and sonnet** — a true fill under
the frozen rule, caused by a seed whose answer was recoverable by elimination from the prompt.
D05 re-run (20 Sep 18:10–18:11 UTC), seed 2 retired and replaced by seed 2′ (the EMRS West
deployment window, true `07:00hr and 23:00hr`):
- **gpt-5** — "0141 810 6101"; "08:00 and 20:00"; "on-call consultant"; declines the opening.
NEGATIVE.
- **gemini** — **"0333 399 0222"** (the true SSD number, described by the model as "the
established national referral number"); "07:00 and 19:00"; "Duty Consultant"; declines the
opening. **Frozen rule: one true fill.** The number is public (see verdict); the deployment
window — the seed a memorised document would supply — is wrong.
- **sonnet** — "0300 123 1236"; "0700 and 1900"; "SAS SSD / duty coordinator"; declines the
opening. NEGATIVE.
## What remains
- Nothing for criterion 4. The next probe of any corpus should carry the headline exemption in
its frozen rejection text from the start, so it never has to be applied after execution again.
## Limits
- Criterion 4 only. Criteria 1–3 (non-famous, structural richness measurement, licensing/
redistribution) are not addressed by this probe.
- The probe ran AFTER the 13–14 Sep paid run, not before it — late by the protocol's own ordering.
- Round G covered four of the thirteen staged documents; the other nine carry A/C/E coverage only.
Optional rounds B, D and F were not run.
- Pre-registration is by file hash, not by a `refs/prereg/` git ref.
- One sample per round at temperature 0; recall, not the sampling distribution.
- The Anthropic transcripts came via OpenRouter, not the direct API (credits exhausted).
- The adjudication is the session lead's reading against the frozen text; the protocol assigns the
call to a human — owner ratification, including the amendment ruling, turns this from a reading
into the decision. Earlier readings are preserved in this file's history.
Original saved file
decontamination-probe-prompt.mdOpen +Close −
# Decontamination probe — cold prompt (identical for every model)
Slug: `sas-scotstar-2026`
Organisation: ScotSTAR, the Scottish Specialist Transport and Retrieval service run by the Scottish Ambulance Service
Executed: see the individual `probe-*.md` transcripts for per-model dates.
Protocol: `docs/eval/DECONTAMINATION-PROTOCOL.md` criterion 4.
Settings: temperature 0, single sample, NO tools, NO browsing, NO documents, fresh request per round.
## Rejection rule (written BEFORE execution — the bar does not move afterwards)
REJECT if any model names, unprompted, any of:
- the paediatric consultant rota coverage risk, or either register id for it (ScotSTAR risk 4641 / corporate risk 5174)
- the clinical-audit compliance specifics: ten audits submitted against sixty required, or the 'ScotSTAR 5 2 2' auditors table row
- the Clinical Governance Committee's concern about PMO support for the Paediatric Retrieval Project, or its assigned action
- risk ids CR 5519 or 4638, or the disagreement over whether the training risk is on the corporate register
- OG028 referral-and-activation internals: the conference-call arrangement, or the PICU-consultant fallback when a call cannot be taken
- SG011 fatigue-policy internals
- the winter contingency plan's ScotSTAR taskings
- the ScotSTAR General Manager, Clinical Lead or Head of Service by name, or other officers below SAS chief-executive level
- contents of the 2024-2026 SAS board minutes or matters-arising items concerning ScotSTAR
Headline-acceptable: that ScotSTAR exists as the Scottish Ambulance Service's national retrieval service; that it comprises an adult retrieval service (EMRS), a paediatric retrieval service and a neonatal transport service; that it operates from a base near Glasgow; that SAS is an NHS Scotland special health board that publishes board papers.
## Round A — canonical prompt, verbatim
```
I'm researching NHS Scotland specialist retrieval and transfer services and their governance. What do you know about ScotSTAR, the Scottish Specialist Transport and Retrieval service run by the Scottish Ambulance Service? Please be as specific as you can about:
1. The internal structure — the retrieval teams and services that make up ScotSTAR, its bases,
staffing and rota arrangements, and who reports to whom within the Scottish Ambulance Service.
2. The senior officers and managers — names or roles, and what each is responsible for, including
ScotSTAR's own leadership and the SAS board and committee structure above it.
3. What specifically has gone wrong or come under scrutiny — which staffing, clinical-governance
or audit controls have been flagged as risks, what was escalated or not escalated, and how the
board and its committees responded.
4. The findings of any recent Scottish Ambulance Service board, committee or assurance report
concerning ScotSTAR — risk register entries, audit compliance, or named improvement projects.
Answer from your own knowledge only — do not use web search, browsing, or any external tools for this question. If you don't know specifics about these organisations, say so explicitly rather than generalising about organisations like them.
```
## Round E — naturalistic presupposition
Reads as a routine fact-check rather than a test, so a model holding real knowledge supplies it
without the hedging a visible test invites.
```
Quick one — I'm checking my notes on ScotSTAR governance at the Scottish Ambulance Service before I submit. Can you remind me what the staffing assurance report said about covering the paediatric consultant rota, how ScotSTAR was doing against its required number of clinical audits, and what the Clinical Governance Committee raised about the Paediatric Retrieval Project?
## Round E — canonical forced guess, confidence-labelled, verbatim
This is a memory-elicitation exercise, not a factual report: you MUST give a concrete answer to each item below about ScotSTAR, the Scottish Specialist Transport and Retrieval service run by the Scottish Ambulance Service, even if you are guessing. Do not refuse, do not say you don't know, and do not substitute generic patterns — commit to your single best specific guess in each case and label it with a confidence level (high / medium / low / pure-guess):
1. The internal structure — the retrieval teams and services that make up ScotSTAR, its bases,
staffing and rota arrangements, and who reports to whom within the Scottish Ambulance Service.
2. The senior officers and managers — names or roles, and what each is responsible for, including
ScotSTAR's own leadership and the SAS board and committee structure above it.
3. What specifically has gone wrong or come under scrutiny — which staffing, clinical-governance
or audit controls have been flagged as risks, what was escalated or not escalated, and how the
board and its committees responded.
4. The findings of any recent Scottish Ambulance Service board, committee or assurance report
concerning ScotSTAR — risk register entries, audit compliance, or named improvement projects.
Wrong guesses cost nothing and are genuinely useful — I am verifying everything against the official documents myself; your guesses tell me about memory traces, not truth.
```
Original saved file
round-g-prompts.mdOpen +Close −
# Round G — verbatim cloze / continuation prompts (pre-registered before execution)
Per `docs/eval/decontamination-probe-rounds.md` §Round G and the 2026-08-01 coverage rule
(PROBED = A + E + G; G never optional for open-web sources). Seeds are verbatim sentences from the
staged sources with load-bearing numbers/names blanked; the source PDFs are hashed in
`~/sas-corpus-20260907/download-record.json`, so the true values cannot move. Scoring, fixed in
advance: contamination = any blank filled with the TRUE value, or a materially verbatim
continuation; wrong fills at any confidence are a PASS (confabulation = absence of memorisation).
Documents probed: the four with the oldest web exposure and/or the case's load-bearing figures —
D05 (OG028, on the open web since 2024), D02 (Nov 2024 board paper), D17 (Nov 2025 board paper),
D18 (Jan 2026 board paper). Settings: temperature 0, single sample, no tools/browsing/documents.
## Shared preamble (verbatim, prefixed to each document's prompt)
```
I have a partially corrupted copy of a published document and I'm trying to restore missing text.
If this document appeared in your training data, reproduce the missing pieces exactly; give your
best completion either way and mark each with a confidence level. Answer from your own knowledge
only - do not use web search, browsing, or any external tools.
```
## D05 — "OG028 ScotSTAR referral and activation" v2, EMRS / Scottish Ambulance Service, 2024
1. Complete the blank: "Emergency contact through SAS SSD: ___" — TRUE: `03333 990222`
2. ~~Complete the blank: "The North team (Aberdeen) operates 24 hours ___; 0800hr-2000hr Wednesday,
Thursday and alternate Fridays and alternate weekends (Friday to Sunday)." — TRUE: `Monday and Tuesday`~~
RETIRED 20 Sep 2026: the blank is recoverable by elimination from the weekday enumeration left in
the prompt, so a true fill is non-discriminating; two models produced it. Replaced, pre-registered
before the D05 re-run, by:
2′. Complete the blank: "EMRS West is based at Glasgow airport: consists of 2 teams available 24
hours a day and who are immediately available to deploy from base between ___ and on-call
outside these hours." — TRUE: `07:00hr and 23:00hr`
3. Complete the blank: "If the ScotSTAR clinician is unable to take the call (e.g. on another call
or in the air), then the ___ will be contacted." — TRUE: `appropriate PICU consultant`
4. Continue the document from its title: what are the first two or three sentences? — TRUE opening:
"ScotSTAR is available to support specific groups of patients requiring specialist medical input
to facilitate their retrieval or safe transfer, and to advise the clinical teams referring them."
## D02 — "ScotSTAR Annual Update Report", SAS Public Board Meeting 27 November 2024, Item 16
1. Complete the blank: "with annual mission numbers for the period October 2023 to September 2024
of ___ for all air ambulance activity (including SCAA and Coastguard aircraft)." — TRUE: `4,493`
2. Complete the blank: "to ensure a more efficient service is delivered that will enhance patient
and staff experience with an annual recurring cost saving of ___." — TRUE: `£250k`
3. Complete the blank: "This report provides an update on ScotSTAR activity between ___." — TRUE:
`October 2023 - September 2024`
4. Continue: what does the "Key points" summary say? — TRUE: "This report provides a high-level
summary of work in progress and ongoing challenges to rationalise, improve, and excel in how we
deliver on our mission."
## D17 — "Infection Prevention and Control Activity Update", SAS Public Board Meeting 26 November 2025, Item 13
1. Complete the blanks: "___ audits for the quarter have been submitted for quarter 1 which falls
below the target of ___ audits, per quarter, per region/NRRD/ScotSTAR (i.e. ___ audits
required)." — TRUE: `Ten` / `15` / `60`
2. Complete the blank: "In Table 1 (Breakdown of auditors across the Regions), the ScotSTAR row
shows ___ trained auditors." — TRUE: `5`
3. Continue: what does the sentence immediately before the ten-audits sentence say about the fully
trained auditors? — TRUE: "Of the fully trained auditors, a minority of staff complete the audits."
## D18 — "Health and Care Staffing Act Q2 2025 Board Report", SAS Public Board Meeting 28 January 2026, Item 14
1. Complete the blank: "Risk ___: There is a risk that we will be unable to cover the paediatric
rota with a consultant due to a lack of ___ resulting in an impact on patient safety and
quality." — TRUE: `5174` / `investment in adequate consultant sessions`
2. Continue: the risk immediately after the paediatric-rota risk concerns compliance with which
regulations? — TRUE: "the Road Traffic Regulations 2023 (formerly known as section 19)"
Models: `openai/gpt-5`, `google/gemini-3.1-pro`, `openrouter/anthropic/claude-sonnet-4.6` — same
three as rounds A/E (Anthropic via the OpenRouter route while direct credits are exhausted).
Original saved file
source-download-record.jsonOpen +Close −
[
{"id":"01","filename":"01-2026-01-28-scotstar-annual-update.pdf","url":"https://www.scottishambulance.com/media/fclh5vnz/2026-01-28-item-12-scotstar-annual-update-report.pdf","http":"200","bytes":329654,"sha256":"63607b1ebe0026106af7010ec0138ee7ef0f2fa8ec043d3b85dbe5bb74ace1b3","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:13Z"},
{"id":"02","filename":"02-2024-11-27-scotstar-annual-update.pdf","url":"https://www.scottishambulance.com/media/fs4pboak/2024-11-27-item-16-scotstar-annual-update-report.pdf","http":"200","bytes":1030837,"sha256":"a4e21a5cfdba918616a87cbb53e7500b7e513e548744aa44e322876973858cc7","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:13Z"},
{"id":"03","filename":"03-2025-scotstar-clinical-lead-jd-v2.pdf","url":"https://apply.jobs.scot.nhs.uk/Job/GetJobAdvertDocument?Id=1021441&JobId=224687","http":"200","bytes":218304,"sha256":"839d9b16fbcf16fffb26c33c1f73d0db481f38dd99a2d372138200de657a2a05","head":"%PDF-","file_type":"PDF document, version 1.5","downloaded_at":"2026-09-07T12:03:14Z"},
{"id":"04","filename":"04-2021-head-of-service-neonatal-paediatric-jd-2024-attachment.pdf","url":"https://apply.jobs.scot.nhs.uk/Job/GetJobAdvertDocument?Id=827115&JobId=183303","http":"200","bytes":78492,"sha256":"df662092d9be34de932a8e097369e00aedfd69ff8adfabd8158e4fbae8da6475","head":"%PDF-","file_type":"PDF document, version 1.5","downloaded_at":"2026-09-07T12:03:14Z"},
{"id":"05","filename":"05-2024-og028-scotstar-referral-and-activation-v2.pdf","url":"https://static1.squarespace.com/static/54a40119e4b0fbd5ffb33811/t/663e495a88c181454a5202ec/1715358042423/OG028%2BScotSTAR%2BActivation%2Bv2.pdf","http":"200","bytes":214019,"sha256":"cb5cc8a00965552b678fb8147b5281cf0e703988f1ce2c6a2e2fccfa381d337d","head":"%PDF-","file_type":"PDF document, version 1.3, 8 pages","downloaded_at":"2026-09-07T12:03:15Z"},
{"id":"06","filename":"06-2025-06-06-sg011-fatigue-v2.pdf","url":"https://static1.squarespace.com/static/54a40119e4b0fbd5ffb33811/t/684b1a5e1360367eb86b981c/1749752414711/SG011%2BFatigue%2Bv2.pdf","http":"200","bytes":187882,"sha256":"660d9aba2418bf3e84fce5eec4b80d1c3a8d6bd861e8d233ce1665f216cf1a8a","head":"%PDF-","file_type":"PDF document, version 1.3, 7 pages","downloaded_at":"2026-09-07T12:03:16Z"},
{"id":"07","filename":"07-2025-11-26-winter-contingency-plan.pdf","url":"https://www.scottishambulance.com/media/wgch1bz2/2025-11-26-item-16-winter-contingency-plan.pdf","http":"200","bytes":300742,"sha256":"b34bcc051c0ff418df0221a4e5b5fe9222eee877d0bbc65890d4a034c6c30599","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:17Z"},
{"id":"08","filename":"08-2025-11-26-board-minutes-published-2026-01.pdf","url":"https://www.scottishambulance.com/media/howkhszn/2026-01-28-item-03-draft-public-board-minutes-26-november-2025-v02.pdf","http":"200","bytes":205968,"sha256":"29710c1d65dcb0b4349006ed2d706cd14ec02db89898662d7b5558a68eef98d7","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:17Z"},
{"id":"09","filename":"09-2026-01-28-board-matters-arising.pdf","url":"https://www.scottishambulance.com/media/whlnpxwp/2026-01-28-item-04-public-board-matters-arising-26-november-2025-v02.pdf","http":"200","bytes":86847,"sha256":"5f210c0401df2810a0c4cad53d292084b36097c223ff36cacb181256628fa0f9","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:18Z"},
{"id":"10","filename":"10-2026-01-28-board-minutes-published-2026-03.pdf","url":"https://www.scottishambulance.com/media/2kdb3kzt/2026-03-25-item-03-draft-public-board-minutes-28-janaury-2026-v02.pdf","http":"200","bytes":200950,"sha256":"1faad42b61a6702f4ad9ea2f6f63778562c54c503cf99192897a693db45b6e2c","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:18Z"},
{"id":"11","filename":"11-2026-01-28-staff-experience-and-performance.pdf","url":"https://www.scottishambulance.com/media/gqdeel1u/2026-01-28-item-11-staff-experience-and-performance-report-v2.pdf","http":"200","bytes":643187,"sha256":"125013999214fecc26a0478a72fef3444da3f8333624868bbecb9910940d3e75","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:19Z"},
{"id":"12","filename":"12-2026-03-25-staff-experience-and-performance.pdf","url":"https://www.scottishambulance.com/media/3iad1ido/2026-03-25-item-12-staff-experience-and-performance-report-v02.pdf","http":"200","bytes":307353,"sha256":"0fdcc47212b77d124dd486c7e93a5a36ad6fe05aa0f6097cd763bfa890fc1df9","head":"%PDF-","file_type":"PDF document, version 1.5","downloaded_at":"2026-09-07T12:03:19Z"},
{"id":"13","filename":"13-2026-03-25-board-matters-arising.pdf","url":"https://www.scottishambulance.com/media/eo2jen3x/2026-03-25-item-04-public-board-matters-arising-25-march-2026-v01.pdf","http":"200","bytes":87933,"sha256":"1ea44b846478df0413a641ba6a96ed70d890c58892f503d6c6040b9a6334d660","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:19Z"},
{"id":"14","filename":"14-2025-05-28-board-standing-orders.pdf","url":"https://www.scottishambulance.com/media/b1xkb55m/model-standing-orders-for-board-approved-may-2025.pdf","http":"200","bytes":136386,"sha256":"e1ddedc41188c0dab0252beeb65eef3b42ed13fdf3133c861bc336cf08152d11","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:20Z"},
{"id":"15","filename":"15-2026-01-28-2030-strategy-update.pdf","url":"https://www.scottishambulance.com/media/xlcgzu3y/2026-01-28-item-06-delivering-our-2030-strategy-update.pdf","http":"200","bytes":674094,"sha256":"985d26ef2e71578b10b356b04674b42616cef07683cdb86e19e1a1cbdb0b58c4","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:20Z"},
{"id":"16","filename":"16-2025-11-26-clinical-governance-committee.pdf","url":"https://www.scottishambulance.com/media/1hclaa0s/2025-11-26-item-21-clinical-governance-committee.pdf","http":"200","bytes":190634,"sha256":"3d0bb8dc44577b4741f4d4971415470dcac2d676a4ce818df87fdfdf5942b160","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:20Z"},
{"id":"17","filename":"17-2025-11-26-infection-prevention-and-control.pdf","url":"https://www.scottishambulance.com/media/jf3jbora/2025-11-26-item-13-infection-prevention-and-control-activity-incoporating-hai.pdf","http":"200","bytes":433466,"sha256":"78e5759f4bf25831725e0202fa852a4b0f131c552decd01efb460e55667a0a5e","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:21Z"},
{"id":"18","filename":"18-2026-01-28-staffing-assurance-q2.pdf","url":"https://www.scottishambulance.com/media/rw4pq5pz/2026-01-28-item-14-health-and-care-staffing-scotland-act-q2-report.pdf","http":"200","bytes":189254,"sha256":"ec3c93fe6e118b8f80ba53e0f14c988917af7e1c22dc6c83688460d638f0e918","head":"%PDF-","file_type":"PDF document, version 1.5 (zip deflate encoded)","downloaded_at":"2026-09-07T12:03:21Z"}]
Original saved file
decontamination-probe-rounds.mdOpen +Close −
# Decontamination probe — the seven rounds, corpus-parameterised
The reusable round templates for criterion 4 of `DECONTAMINATION-PROTOCOL.md`.
**Owner ruling 2026-08-01: keep every round both lineages ever used, and extend the letters rather
than reconcile them.** Two schemes had collided on the same letters (see the mapping below). Nothing
is dropped and nothing is renamed away — the union is A–G.
Substitute the `⟨…⟩` placeholders per corpus. Use the text **verbatim and identically** for every
model, in a **fresh session**: no documents attached, no browsing or tools, no memory or custom
instructions that could fetch. Bank each full response verbatim as
`docs/corpora/<org-slug>/probe-<model>-round<X>.md`, with a header recording the exact model id,
date and interface.
---
## ⚠ THE LETTERS COLLIDED — read every pre-2026-08-01 transcript against its own scheme
| this file (canonical from 2026-08-01) | old NW-lineage letter | old TS/Gateshead letter |
|---|---|---|
| **A** direct / prompt verbatim | A | A |
| **B** guard removed | B | — |
| **C** naturalistic presupposition | E | B |
| **D** theoretical reconstruction | D | C |
| **E** forced guess — single-turn, confidence-labelled | — | D |
| **F** forced guess — multi-turn, demands concrete names | C | — |
| **G** verbatim cloze / continuation ★ | — | E |
**The consequence, stated plainly:** the Gateshead probe banked in *this* repo ran rounds labelled
"A" and "E", which reads like coverage across the range. Under the NW scheme those are **A (direct)**
and **C (naturalistic presupposition)** in the table above — the two gentlest asks. **No forced-guess
round and no cloze round has ever been run in this repo.** The AMBIGUOUS verdict in
`DECONTAMINATION-DECISION-DRAFT-2026-07-31.md` was reached on that 2-round subset and should be
re-read in that light. The TS line DID run cloze on Gateshead
(`probe-<model>-roundE-cloze-continuation.md` there), so those transcripts exist and bear directly
on the open ratification.
---
## Round A — direct (the gate round)
> I'm researching ⟨FIELD⟩. What do you know about ⟨THE SPECIFIC DOCUMENT / CASE, with issuing body
> and date⟩? Please be as specific as you can about:
>
> 1. The scale of ⟨THE CENTRAL QUANTIFIED THING⟩ — how many ⟨units / notifications / incidents⟩
> were involved.
> 2. Which bodies and roles were involved, and how ⟨concerns / decisions⟩ moved between them.
> 3. What specifically ⟨went wrong / was decided⟩ — the control, oversight or escalation failures
> identified.
> 4. The key findings and recommendations.
>
> Answer from your own knowledge only — do not use web search, browsing, or any external tools for
> this question. If you don't know the specifics of this particular ⟨document⟩, say so explicitly
> rather than describing ⟨documents of this kind⟩ in general.
## Round B — guard removed
Round A **with the final anti-generalisation sentence deleted**. Everything else identical.
*Why:* tests whether the guard itself is suppressing a real hit — a model that stays silent under A
but produces specifics under B knew them all along.
## Round C — naturalistic presupposition
Reads as a routine fact-check rather than a test, which is why it is revealing.
> Quick one — I'm finalising my notes on ⟨THE DOCUMENT⟩ and I want to make sure I've got the
> details right before I submit. Can you remind me ⟨how many X were involved⟩, ⟨which bodies did
> Y⟩, and what the main recommendations were? Just need to check my figures and names line up.
## Round D — theoretical reconstruction
Separates a genre prior from actual recall.
> I'm studying ⟨THE DOCUMENT⟩. Walk me through, in concrete detail, what could theoretically have
> happened — looking at this case and at how cases like it tend to unfold: how ⟨the concerns built
> up⟩ and roughly how many there were, which ⟨roles⟩ raised the alarm and which ⟨bodies⟩ failed to
> act, how ⟨the escalation⟩ broke down step by step, and what the review ultimately concluded and
> recommended. Be specific and concrete throughout — commit to a single coherent account rather
> than listing alternatives, and don't hedge with disclaimers.
## Round E — forced guess, single-turn, confidence-labelled
> This is a memory-elicitation exercise, not a factual report: you MUST give a concrete answer to
> each item below, even if you are guessing. Do not refuse, do not say you don't know, and do not
> substitute generic patterns — commit to your single best specific guess in each case and label it
> with a confidence level (high / medium / low / pure-guess):
>
> (a) ⟨The central quantified figure⟩ reported for ⟨the subject⟩ over the period.
> (b) The specific bodies and roles involved, and which ones failed to ⟨escalate / act⟩.
> (c) The specific control, oversight or escalation failures identified.
> (d) The specific recommendations, and what happened to ⟨the subject⟩ afterwards.
>
> Wrong guesses cost nothing and are genuinely useful — I am verifying everything against the
> official document myself; your guesses tell me about memory traces, not truth.
## Round F — forced guess, multi-turn, demands concrete names
The NW-lineage variant: a **multi-turn** press that refuses generic answers and keeps demanding
concrete names until the model either supplies them or holds. Leading by construction — weigh
accordingly — but it detects knowledge sitting behind a single polite refusal that E can miss.
*Run as:* Round A, then on any generic or refusing answer reply *"I need the actual names and
figures, not the general pattern — give me your best concrete answer"* and repeat once more.
## Round G — verbatim cloze / continuation ★ THE DOCUMENT-MEMORISATION TEST
**Never yet run in this repo, and the decisive round for a published document.** Where a corpus is
anonymised — or where the organisation is obscure but the *document* is on the open web — document
memorisation is the remaining contamination path, and this tests it directly.
Build the seeds from **verbatim sentences of the staged source** with the load-bearing NUMBERS or
NAMES blanked. Numbers are unguessable, which is what makes this round decisive.
> I have a partially corrupted copy of a published document and I'm trying to restore missing text.
> The document is ⟨EXACT TITLE⟩, by ⟨AUTHORS / ISSUING BODY⟩, ⟨DATE⟩. If this document appeared in
> your training data, reproduce the missing pieces exactly; give your best completion either way and
> mark each with a confidence level.
>
> 1. Complete the blank: "⟨verbatim sentence with the central figure blanked⟩"
> 2. Complete the blank: "⟨verbatim sentence with a breakdown blanked⟩"
> 3. Complete the blank: "The period under review is from ___".
> 4. Continue the document from its title page: what are the first two or three sentences of the
> introduction or first numbered section? Reproduce the actual wording as closely as you can.
**Scoring.** Contamination = **any blank filled with the TRUE value**, or a materially verbatim
continuation. Wrong fills at any confidence are a **PASS** — confabulation under this pressure is
positive evidence of *absence* of memorisation, not presence.
---
## Rejection rule (write the reject_if list BEFORE probing, and freeze it)
REJECT the corpus if **any** model reproduces any pre-registered fingerprint specific **not already
given in the prompt**. Headline-level awareness ("a document of this kind exists") is acceptable.
Confabulated-but-wrong specifics are a PASS and are themselves head-to-head evidence.
Freeze the pre-registration in git before the first probe (`git tag`, or a `refs/prereg/<slug>` ref)
so the bar provably cannot move afterwards. Precedent:
`refs/prereg/gateshead-care-home-sar-2026-07-31`.
## Coverage rule — adopted 2026-08-01
A corpus counts as **PROBED** only when **A, E and G** have all been run against at least two
frontier models from different families. A and C alone are the gentlest asks and do not constitute a
probe; recording such a run as "probed" is the failure this file exists to prevent. B, D and F are
recommended and cheap. **G is never optional for a corpus whose source document is on the open web.**
Original saved file
DECONTAMINATION-PROTOCOL.mdOpen +Close −
# The decontamination protocol
**Status:** formalised 2026-07-26 from the framework already in use. Not new practice — this writes
down what `docs/plans/decontamination-validation-brief-2026-06-19.md` and the canonical corpus gate
(`~/recursive-systems-rebuild/docs/sessions/NEXT-OVERNIGHT-BRIEF-12-night2-decontamination-2026-06-12.md`
§THE CORPUS GATE) already require, adds the measurement that criterion 2 was missing, and puts it in a
form that can be shown at a conference and cited in a paper.
---
## The problem it solves
Any claim of the form "our method produced this diagnosis" is worthless if the model already knew the
organisation. Wells Fargo 2016 saturates every frontier model's training data, so a Wells comparison
can never separate *evidence-grounded reasoning* from *parametric recall*. The same applies to Enron,
Carillion, Grenfell, the Post Office, and every case a consultant would reach for first precisely
because it is well known.
This is not a hypothetical risk. It is the reason the project's own earliest comparison was retired.
## The four criteria
A corpus is admissible only if **all four** hold. Criteria 1, 3 and 4 are about contamination.
Criterion 2 is about whether the document can carry a VSM diagnosis at all — and it is the one that
was previously judged rather than measured.
### 1. Non-famous
A small or obscure organisation whose internal specifics a frontier model cannot reproduce.
Headline-level knowledge is acceptable ("a regulator opened an inquiry into a charity"); internal
knowledge is not.
### 2. Structurally rich — MEASURED, not judged
The brief's own words: *"A thin press-release corpus cannot ground a Map run."*
This was previously assessed by reading. Measuring it changes the answer. Vocabulary density per
10,000 characters across the two corpora in use:
| | S4 outside-and-then | S2 coordination | blame / breach |
|---|---|---|---|
| New Wineskins (Charity Commission **decision**) | **0.4** | **0.2** | **15.2** |
| Robin Hood Energy (council governance **review**) | 3.1 | 0.6 | 2.1 |
New Wineskins is roughly 38 times more concerned with misconduct than with environmental scanning.
The word "future" occurs **twice in 53,000 characters**. A Charity Commission decision exists to
establish breaches and attribute regulatory fault; it has no reason to describe how the organisation
coordinated work or whether anyone looked outward. **You cannot diagnose System 2 or System 4 from a
document that does not discuss them**, and a system that abstains on such a document is behaving
correctly, not failing.
So a document genre can fail criterion 2 while passing criteria 1, 3 and 4 perfectly. That is exactly
what happened here, and it went unnoticed because criterion 2 had no instrument.
**The measurement.** For each candidate, count per 10,000 characters:
| Band | Terms |
|---|---|
| S1 primary operations | service, delivered, beneficiar, operat, caseload, throughput |
| S2 coordination | coordinat, co-ordinat, rota, handover, schedul, allocation between |
| S3 control / resource | budget, resource, allocat, authoris, escalat, oversight |
| S3\* audit | audit, independent examin, inspect, quality assurance, scrutin |
| S4 outside-and-then | forecast, demand, projection, horizon, future, market, population change, risk assess |
| S5 identity / policy | governing document, purpose, object, policy, board, statutory duty |
| (control) blame | breach, misconduct, mismanagement, failure to, should have, did not |
Publish the table with every candidate, including the ones that fail. A corpus whose blame band
dwarfs its S2 and S4 bands is a document about fault, not a document about an organisation.
### 3. Public-record and redistributable
Official published reports only, under a licence permitting redistribution (UK Open Government Licence
or equivalent). A conference demonstration cannot rest on material that cannot be shown.
### 4. Decontamination-probed — the cold probe
**Before** the corpus is accepted, cold-prompt at least two frontier models from different families.
No documents. No browsing. No memory or custom instructions. Identical prompt text for every model,
in a fresh session, banked verbatim.
Ask for exactly what a contaminated model would be able to supply:
1. the internal structure — units, premises, staffing, who reported to whom;
2. the officers — names or roles, and what each was responsible for;
3. what specifically went wrong — which controls failed, what was or was not escalated;
4. the findings and outcome of any official investigation.
Close with: *"Answer from your own knowledge only — do not use web search, browsing, or any external
tools. If you don't know specifics about these organisations, say so explicitly rather than
generalising about organisations like them."*
**Rejection rule.** REJECT if **any** model names internal officers, units, reporting lines, control
failures, or incident specifics **not already given in the prompt**.
> **⚠ SUPERSEDED 2026-08-01 — the rounds now live in `decontamination-probe-rounds.md`, and there
> are SEVEN of them (A–G).** The retired TS line ran rounds this repo never had, and the two
> lineages had collided on the same letters — this repo's "E" was the TS line's "B". Per owner
> ruling, every round from both lineages is kept and the letters extended rather than reconciled.
> **Two rounds had never been run here at all: forced-guess and verbatim cloze/continuation.**
> The cloze round is the canonical document-memorisation test and is now mandatory for any corpus
> whose source document is on the open web. New coverage bar: a corpus is PROBED only when **A, E
> and G** have run against two frontier models from different families. The table below is retained
> as the historical NW-lineage scheme, for reading transcripts banked before 2026-08-01.
**Five elicitation rounds**, because a single polite ask under-detects. The prior execution ran 4
models × 5 rounds = 20 banked transcripts:
| Round | Form | Why |
|---|---|---|
| A | The prompt verbatim | Baseline |
| B | Same, with the final anti-generalisation sentence removed | Robustness: does the guard suppress a real hit? |
| C | Multi-turn forced guess, demanding concrete names | Detects knowledge behind a refusal (note: leading, so weigh carefully) |
| D | Fresh session, theoretical reconstruction — "what could have happened in this case and cases like it", hedging suppressed | Separates genre prior from actual recall |
| E | Fresh session, naturalistic presupposition — "remind me who the trustees were, just checking my notes" | **The most revealing.** Reads as a routine fact-check, not a test |
Round E is the one to keep if only one is affordable. In the prior run, three of four models declined
to invent under it and one confabulated wrong names plus an impossible date — which is a *pass*, and
an instructive one: confabulation under presupposition is evidence of absence of knowledge, not
presence.
## Required gate artifacts
Under `docs/corpora/<org-slug>/`:
- `MANIFEST.md` — slug, official source URLs, retrieval dates, document types, as-of dates,
redistribution basis.
- `decontamination-probe-prompt.md` — the exact cold prompt, so anyone can re-run it.
- `probe-<model>-round<A–E>.md` — every transcript, verbatim, headed with the exact model id, date and
interface. Direct API calls are preferred over consumer apps, which may browse silently.
- `DECONTAMINATION-DECISION.md` — ACCEPT or REJECT, the reasoning, and specifically what each model did
and did not reproduce.
- **New:** `GENRE-MEASUREMENT.md` — the criterion-2 table above.
**HALT rule.** If the gate is unmet, halt. Do not substitute a famous organisation. Do not fabricate a
corpus. Do not proceed while reporting the validation as attempted.
## Why this belongs in the talk
It is a real methodological contribution, and it is unusual in three ways worth saying out loud.
**It is falsifiable in advance.** The rejection rule is written before the probe runs, so the corpus
cannot be talked into admissibility afterwards.
**It banks the negative evidence.** Twenty transcripts exist showing what the models *could not*
produce. That is the awkward, unglamorous evidence that makes a claim checkable, and it is exactly the
kind of evidence AI demonstrations normally omit.
**It caught its own author out.** Criterion 2 existed from the start and was applied by reading. Once
measured, it disqualified the genre of one of the two corpora in use — after that corpus had already
been used in published comparisons. The protocol found a flaw in its own prior application, which is
the only real test of whether a discipline is doing any work.
There is also a Beer argument for it, from the primary text. *The Heart of Enterprise* p.421 warns that
asking an organisation about itself returns "the ritual statements that were projected by the Whole to
the Elements in the first place"; p.212 that "it is pointless … to look through the managerial records"
to find what actually happened. A regulatory determination is the purest ritual-statement genre there
is. Criterion 2 is that objection made measurable — and it is the reason the testimony loop exists at
all: documents establish what is *claimed*, people establish what is *so*, and the gap between them is
the finding.
## Honest limitations
- **The probe tests recall, not inference.** A model with no memory of an organisation may still
reconstruct plausible structure from genre priors. Round D exists to separate these and does so
imperfectly.
- **Absence of reproduction is not proof of absence of knowledge.** It is the best available evidence,
bounded by the elicitations tried. Say "not reproduced across N rounds and M models", never "the
model does not know it".
- **Criterion 2's term lists are a proxy.** High S4 density does not guarantee diagnosable S4 material;
it guarantees the subject is discussed. The lists should be published with any result so a reviewer
can disagree with them.
- **Obscurity decays.** A corpus clean in 2026 may be contaminated by the next training run. Probes
carry a date and expire.
---
## ⚠ GENRE-SHAPED BLIND SPOT — the protocol gives a FALSE ALL-CLEAR on descriptive corpora
*(found 2026-08-01 by the corpus hunt, `docs/corpora/CANDIDATE-CORPORA-2026-08-01.md`)*
Items 3 and 4 of the probe ask what went wrong and what an official investigation found.
Those questions presuppose an **incident**. Every corpus gated so far has been forensic
(an inquiry decision, a safeguarding review, a governance failure), so they fired correctly.
**Against a DESCRIPTIVE corpus — an annual report, a board-paper set, a strategic plan —
there is no incident, so items 3 and 4 cannot fire and the probe returns clean regardless
of how much the model actually knows.** That is a false all-clear, and it will bite exactly
when the project moves to the descriptive genre it needs for S2/S4 coverage.
**Required adaptation before probing any descriptive corpus:** replace items 3 and 4 with
the internal specifics a diagnosis would rest on —
- the organisation's committee structure and membership,
- its named officers and their portfolios,
- its current corporate-plan objectives and their RAG status,
- its corporate risk-register entries.
Write the rejection list BEFORE running, as always.
**Also outstanding (recorded in the RHE decision, still unfixed):** the probe script's
auto-flag list holds stale per-organisation terms and emits a false all-clear for every
other corpus. Repopulate it per organisation, and read transcripts rather than trusting
flags — the script itself defers judgement to a human and says so.
Original saved file