CP-HRIC · Model · cp-hric-s1 family

CP-HRIC-S1 · multi-task model

A System One model that scores each cell and is asked two typed questions with fixed answer options: the five-way genetic subtype and APL or another AML. One model, both answers. The genetic subtype is scored on the Munich folds (every patient tested once); APL on DeepAPL's 24 held-out patients, never used for training or tuning. For MILLIE's test, the same recipe is re-run on MILLIE's own 3-fold split of all 106 Johns Hopkins patients.

Cell encoder
our own encoder (v3e), frozen
Trained on
the Munich patients plus DeepAPL's 82 Johns Hopkins training patients
The orange System One robot at a workstation reading several results at once

Against the three papers

Each paper’s own test, measured its way.

  1. SCEMILAmacro-F1 over the five subtypes, the paper’s five patient-level folds · paper 0.810.809± 0.016 · 10 seeds−0.001 vs 0.81Matched
  2. MILLIEAUROC 3-fold cross-validation · paper 0.96 ± 0.020.885± 0.025 · 10 seeds−0.075 vs 0.96In progress
  3. DeepAPLAUROC on 24 held-out patients · paper 0.86 (published score; model not re-run)0.864± 0.038 · 10 seeds+0.004 vs 0.86Matched within margin of error

How it got here

Key changes.

One version so far: the results above are it.

Show all steps

Each step is a change to the model or its data, in order. Bar = mean across seeds, whisker = ± the spread across seeds, diamond = the seeds’ models combined, dashed line = the paper. The axis starts at 0.5.

SCEMILA · Munich · genetic subtype · macro-F1 SCEMILA (0.81)
  1. Step 9 Munich + Johns Hopkins, one model for both questions10 seeds · ± 0.016Matched0.809−0.001
MILLIE · Johns Hopkins · APL · 3-fold CV · AUROC MILLIE (0.96)
  1. Step 9 Munich + Johns Hopkins, one model for both questions10 seeds · ± 0.025 · 10 models combined: 0.944In progress0.885−0.075
DeepAPL · Johns Hopkins · APL · 24 held-out · AUROC DeepAPL (0.86)
  1. Step 9 Munich + Johns Hopkins, one model for both questions10 seeds · ± 0.038 · 10 models combined: 0.917Matched within margin of error0.864+0.004

How much it learned from: the training data, image counts and sizes →

Beats = above the paper by more than run-to-run noise; Matched within margin of error = above it by less than that; Matched = our mean ± spread overlaps the paper’s range; Approaching = within 0.05; In progress = further away. The chip takes the model’s family colour. Every number is the declared configuration's mean across seeds ± the spread across seeds; a combined model (several seeds' models combined into one) is shown after it, never instead of it. Johns Hopkins scores use the cleaned data (prep v2). "Not run · queued" cells are scheduled to be filled.