External CRC benchmark, round 3: lab book

Parallax Pathology · Russell Parrish, analysis carried out with Claude (Anthropic) · 5 November 2026 · lab book for round 3 of the external CRC benchmark · published November 2026

Published as written at the time, apart from this header. File and folder names refer to the project record, available on request. Benchmark note · Claim ledger
Erratum (6 November 2026): Balanced accuracy at look 1 is 82.0% (82.05%); it appears here as 82.1%. Rounding only.

Summary. Round 3 kept round 2's measurements and worked on the readout and on orientation. On the 1,800 development tiles the best version (direction-free fields, gradient boosting averaged with logistic regression) reached 91.0%, against 89.1% for round 2. Frozen and run once on the 5,380 sealed tiles (look 3), it reached 91.6% (balanced 89.9%), against 91.5% (89.4%) at look 2: the same within noise. The development gain did not transfer. The errors moved instead: normal called tumour fell from 61 to 48, while stroma and muscle confusions rose from 94 to 118. On these 148 measurements the readout looks to be at its operating limit; further gains need new measurements, chiefly for the stroma and muscle boundary.

Entry 1 · Bookkeeping

Before round 3 the sealed tiles had three logged looks (0 to 2). Same rule as before: develop on NCT (all) and the 1,800 development tiles only; one logged look at a frozen version (../EXTERNAL_CRC_ROUND2_20261105/LOOK_LOG.md).

Entry 2 · Readout tuning

Same 154 fields as round 2; development tiles (outputs/01_readout_tuning.log).

Readout	Development accuracy	Stroma	Muscle	Normal
Gradient boosting, 300 iterations (round-2 primary)	89.1%	0.71	0.81	0.81
Gradient boosting, 800 iterations, learning rate 0.05	89.2%	0.72	0.79	0.81
Gradient boosting, 63 leaves	88.6%	0.69	0.78	0.82
Gradient boosting, L2 1.0	88.7%	0.69	0.79	0.83
Gradient boosting, balanced class weights	89.3%	0.71	0.79	0.84
Gradient boosting, 500 iterations, no early stopping	89.6%	0.71	0.81	0.82
Logistic regression	86.4%	0.66	0.73	0.91
Average of gradient boosting (300) and logistic	90.1%	0.72	0.79	0.91

Tree settings move accuracy by about a point (88.6% to 89.6%). Logistic regression is weaker overall but much better on normal tissue (0.91), so averaging the two readouts' class probabilities gains about a point.

Entry 3 · Looking at normal called tumour

Normal and tumour tiles — Rows: development normal called tumour; development normal called correctly; development tumour called correctly; NCT normal; NCT tumour.

The missed normal tiles still show crypts with goblet cells, often cut obliquely or crowded. A goblet-vacuole detector (bright round vacuoles, filtered by saturation and clustering; scripts/goblet.py) was tried as a new measure. It does not separate the classes in the training collection (NCT normal against tumour, density AUC 0.47, 150 tiles each), On development tiles it does separate them (AUC 0.70), but missed and correct development normal tiles have the same median density (21.0 and 21.7), so it does not reach the errors. Recorded as a negative and not used.

Entry 4 · Orientation

Development tiles were measured in their other rotations and flips (scripts/04_tta_measure.py, a 64-measurement test: 9 tiles in all 7 other orientations, 1 tile in one, plus the originals). 146 of 154 fields move by less than 5% of the between-tile spread. Eight move more: b_dx, b_dy, k_dx, k_dy (direction offsets; within-tile spread 0.7 to 1.1 times the between-tile spread) and sci, g3_kvar, g3_infl, vcli_g. Tissue tiles have no preferred direction, so these fields carry orientation, not tissue.

Development tiles	Gradient boosting (500)	Logistic	Average of both
All 154 fields	89.6%	86.4%	90.8%
Without the 8 (146)	89.4%	86.6%	90.6%
Direction-free: offset pairs as lengths, other 4 dropped (148)	89.1%	86.4%	91.0%

The three are within half a point, which is noise on 1,800 tiles. The direction-free set was chosen on principle (no field should change when a tile is turned). Averaging predictions over 8 orientations was planned, then dropped: with the direction fields removed, nothing that orientation changes is left to average.

Entry 5 · Freeze (21:43 UTC)

FROZEN_ROUND3.md and hashes. Measurements: round 2, direction-free (148 fields). Primary readout: the average of the class probabilities of gradient boosting (500 iterations, no early stopping, fixed seed) and logistic regression, fitted once on all 100,000 NCT tiles. Secondary, same look: each alone. Chosen from about 18 combinations on the same development tiles, so the 91.0% development estimate is optimistic. Erratum: FROZEN_ROUND3.md describes the orientation test as 8 tiles × 7 orientations; it was 64 measurements over 10 tiles (Entry 4). The field list and the version are unaffected.

Entry 6 · Look 3 (sealed, run once)

5,380 sealed tiles	Accuracy (95% tile-level)	Balanced	Stroma	Muscle	Normal
Look 1: violet protection, 81 fields, linear discriminant	87.0% (86.0–87.8)	82.1%	0.353	0.717	0.773
Look 2: + extensions, gradient boosting	91.5% (90.7–92.2)	89.4%	0.706	0.852	0.819
Look 3: direction-free, average of boosting and logistic	91.6% (90.9–92.3)	89.9%	0.719	0.814	0.860
Look 3: gradient boosting (500) alone	91.5% (90.8–92.2)	89.6%	0.733	0.821	0.802
Look 3: logistic alone	86.0% (85.1–86.9)	84.7%	0.611	0.707	0.900

Primary recall by class: adipose 0.921, background 0.997, debris 0.942, lymphocytes 0.972, mucus 0.927, muscle 0.814, normal 0.860, stroma 0.719, tumour 0.936. Tile by tile against look 2: 113 tiles newly right, 105 newly wrong (paired sign test p = 0.64). Confusions, look 2 to look 3: normal to tumour 61 to 48 of 541; stroma to muscle 40 to 48 of 221; muscle to stroma 54 to 70 of 392.

Entry 7 · Reading

Look 3 equals look 2. The 1.9-point development gain came from choosing among many readouts on the same 1,800 tiles. Round 2's gain came from new measurements and held on sealed tiles; round 3 changed only the readout, and its gain did not hold.
The averaged readout trades classes: logistic regression helps normal tissue and costs stroma and muscle. The total stays at about 91.5%.
This maps the readout's operating limit on these 148 measurements. What remains is the stroma and muscle boundary (118 of the 450 sealed errors) and normal against tumour (48), which need new measurements, not another readout.
The sealed tiles have now had four logged looks (0 to 3). Further rounds on this test set are development, not confirmation; independent confirmation needs a new collection.