Which annotations deserve a second look?

Average precision describes the complete ordering of the review queue. Recall at the primary 5% budget describes the operational slice an expert would actually inspect. The random-review baseline tells us whether that prioritisation is better than spending the same review effort without a model.

Restoration utility is deliberately separate. It asks what happens after simulated review when found injected changes are restored and the downstream classifier is retrained. A ranking can retrieve controlled inconsistencies efficiently while still failing to improve the later classifier.

Intervals and adjusted tests also answer different questions. The displayed 95% intervals come from paired whole-group bootstrap resampling; registered Holm-adjusted values are one-sided. At 0% corruption, average precision is not applicable because the controlled positive event does not exist.

The H4 experiment restored exactly the same 5% review budget under audit-guided and random selection, then trained the same downstream model and evaluated it on the untouched final reference fold. Audit-guided restoration was not favoured.

Subgroup average precision is reported only when the preregistered support rule is met: at least 100 samples and at least 10 injected corruptions. Otherwise the study retains counts without inventing an unstable estimate.

The ranges expose heterogeneity that would disappear inside one aggregate score. They are descriptive, not a biological ranking: the preregistration did not define an omnibus test that could support a causal explanation for tissue, class, mechanism or corruption-rate differences.

Every value in the article is read from the accepted sealed run and carried into evidence.json. The table below traces each statement to its hypothesis, raw identifier, seed, point difference, confidence interval, adjusted p-value and bootstrap count.

The timestamped July freeze records a null commit, a dirty tree and untracked files; the first public Git commit followed on 19 August 2026. Internal hashes preserve file identity, but the later public history is not independent evidence that outcomes were unseen.

The public primary evidence release contains all completed-cell OOF predictions and rankings, the full group bootstrap, subgroup table and H4 restoration arrays. A standalone evidence recalculator that does not import the primary analysis package recomputes the saved H1-H7 comparison statistics; this is not third-party validation. Fold checkpoints were not retained, and a second image-to-result execution still requires a lawful PanNuke copy.

NuCLS is the closest completed test of genuine multi-rater disagreement. Its five-patient result did not satisfy the frozen ranking gate and the guided intervention made downstream macro-F1 worse. MoNuSAC then showed strong controlled retrieval but no statistically supported downstream gain or complete class safety.

The selected AANCA candidate was next recorded internally as frozen before PUMA outcomes. On a previously unused histopathology source it passed all seven internally pre-specified retrieval, downstream, direction, convergence and class-safety gates under controlled corruption. The 144/62 development/final partition was created by AANCA from the 206 public PUMA ROIs; it was not the official hidden PUMA challenge test. The downstream flag_exclude arm omitted the highest-ranked 5% of training rows without expert review or relabelling. This is meaningful controlled-noise transfer evidence, but PUMA does not provide paired natural pre/post expert decisions and therefore cannot prove pathologist-error detection.

Public-history limit. The PUMA protocol, configuration and result first appeared together in public commit c5bd44193b2abd67bc7e7f1bd9384aa87435d500, so public Git history does not independently verify the intended pre-outcome timing. The PUMA verifier is a project-coupled evidence-readback script that recomputes metrics from saved predictions but does not retrain all 44 models. It is not third-party validation or a second image-to-result replication.

Before any primary split is frozen, the inspector verifies folds, shapes, channels, class order, instance identifiers and representative overlays. Cross-class overlaps, void pixels and affected instances are recorded rather than silently corrected.

The overlay is evidence about ingestion and representation, not improved segmentation. Eligibility flags may follow the frozen policy, but the raw masks and source annotations remain unchanged.

Source groups stay together, each audit score is out of fold, the final reference fold is withheld from selection, and every label state remains separate. The matrix retained all 36 comparison entries; unavailable optional pathology cells were not replaced with estimates.

Outcome exposure during technical recovery does not change the stored measurements, but it narrows what can responsibly be claimed. The accepted primary analysis is therefore permanently described as amended and exploratory.

The controlled positive event is an injected label change. Performance against it measures retrieval of that injected process, not naturally occurring annotation inconsistency, expert disagreement or biological truth. The positive PUMA result is a new-source controlled transfer result, while the later stress analysis and observed-label sensitivity are exploratory because PUMA outcomes were already open.

Grouping strength differs by source: the primary PanNuke study guarantees source-patch separation, NuCLS uses patient groups, MoNuSAC uses patient groups and PUMA uses one ROI per case. These safeguards reduce leakage but do not transform a final expert reference label into guaranteed biological truth.

Natural and operational validity still require newly recruited blinded reviewers, ambiguity and abstention labels, a frozen policy evaluated on untouched patients or whole slides, and a prospective multi-site comparison of work with and without AANCA.

The current project has reached PRIMARY_STUDY_COMPLETE, EXTERNAL_VALIDATION_COMPLETE and DEMO_COMPLETE. It has not reached CONFIRMATORY_COMPLETE. The binding natural-data action is retain_uncorrected: AANCA may prioritise cases for qualified review, but it may not silently exclude, relabel or overwrite them.

The next research phase is provisionally named AANCA v2. This is a prospective evidence programme for the same core auditing system, not a retroactive rename of the existing results. Opened PanNuke, NuCLS, MoNuSAC and PUMA outcomes may inform diagnosis of weaknesses, but they cannot serve as the untouched final confirmation for the next claim.