Demo: zero-shot cell typing from H&E images¶
ALHENA performs zero-shot cell-type annotation of nuclei in H&E images using similarity-weighted k-nearest-neighbor (kNN) classification against a labeled single-cell RNA-seq reference.
In this demo, ALHENA uses the Breast reference to annotate nuclei from the
PanNuke dataset. The predictions are compared with PanNuke labels mapped to
the three broad cell classes of compartment_v1:
Epithelial_Melanocyte, Immune and Stromal_Vascular.
No model training or fine-tuning is performed during this demo.
IMPORTANT: This is not a performance benchmark.
Only the first 8 Breast image fields are used, in file order. They are not randomly selected or intended to represent the diversity of breast tissue.
The metrics describe this small sample only and must not be interpreted as ALHENA's general performance on Breast tissue.
What data does ALHENA use?¶
Three elements have different roles:
- PanNuke H&E images: the images and nucleus annotations that ALHENA processes.
- Breast single-cell reference: the default DISCO-derived RNA-seq reference used to assign cell classes through weighted kNN. This reference is not the ground truth used to evaluate the predictions.
- PanNuke labels: the labels used to evaluate agreement after the
predictions have been generated. Nuclei labeled
Deadare excluded before annotation.
The PanNuke labels do not determine which cell class ALHENA predicts.
Before running the notebook, make sure the UNI2-h model weights are available locally. See Set up the UNI2-h model weights.
import warnings
import alhena
from alhena import demo
# Optional path to the UNI2-h weights.
# Leave as None to use ALHENA's standard model lookup.
BACKBONE = None
# Optional local resources.
PARQUET = None # PanNuke fold file
BUNDLE = None # ALHENA encoder bundle directory
REFERENCE = None # prebuilt reference file for TISSUE
# Tissue and number of PanNuke image fields to analyze.
# Fields are selected from the beginning of the tissue's rows in file order.
TISSUE = "Breast"
N_TILES = 8
Run the demo¶
The following cell runs ALHENA on the selected PanNuke images, assigns cell classes using the single-cell reference, and compares the predictions with PanNuke labels.
The command-line equivalent is:
alhena demo Breast --n-tiles 8
The optional paths in the configuration above allow you to use local copies of the PanNuke file, ALHENA bundle, or reference.
Without a local PanNuke file, ALHENA reads the selected fields remotely. The bundle and reference are downloaded if needed. UNI2-h weights must already be available locally.
The default axis is compartment_v1, which allows comparison with PanNuke.
The alternative axis lineage_v1 contains five cell classes, separating
Immune into T_NK, B_Plasma and Myeloid. Predictions on that axis are
not evaluated against PanNuke labels.
See the demo command options.
demo.run(
tissue=TISSUE,
n_tiles=N_TILES,
parquet=PARQUET,
backbone=BACKBONE,
bundle=BUNDLE,
reference=REFERENCE,
)
Understanding the results¶
Which nuclei receive predictions?¶
ALHENA first identifies which annotated nuclei can be processed.
Some nuclei are excluded before prediction:
- Dead: nuclei labeled
Deadin PanNuke. - No pixels after overlap: nuclei with no pixels remaining after overlapping annotations are resolved.
- Not captured after resizing: nuclei that are not represented by any model image token after resizing.
ALHENA attempts to annotate the remaining nuclei.
If a nucleus has an unusable image embedding, ALHENA cannot assign it a cell class. This is a failed prediction, not an excluded nucleus. It remains in the evaluation and counts as an error.
Therefore, the number of evaluated nuclei includes both successful and failed prediction attempts. Nuclei excluded before prediction are not included.
How are the predictions evaluated?¶
PanNuke and ALHENA do not use exactly the same class definitions. The demo maps the PanNuke labels to ALHENA's three broad cell classes:
| PanNuke label | ALHENA class used for comparison |
|---|---|
| Neoplastic | Epithelial_Melanocyte |
| Epithelial | Epithelial_Melanocyte |
| Inflammatory | Immune |
| Connective | Stromal_Vascular |
The results show how many nuclei belong to each class according to PanNuke, how many ALHENA predicts in that class, and how many predictions agree with the PanNuke labels.
Accuracy measures the fraction of evaluated nuclei classified correctly.
Macro F1 summarizes classification performance across the classes while giving each class equal weight.
What are the baselines?¶
ALHENA's results are compared with two simple controls:
- Commonest-class baseline: predicts the most frequent PanNuke class for every evaluated nucleus.
- Frequency-matched baseline: assigns classes according to their frequencies in the evaluated population. Its macro F1 is calculated analytically from expected counts, without sampling random predictions.
These are control scores, not predictions produced by ALHENA.
They are calculated on the same evaluated nuclei as ALHENA but ignore the image, so they show the scores that class frequencies alone produce.
What does confidence mean?¶
ALHENA uses a similarity-weighted vote from its nearest reference cells.
The confidence score is the fraction of that vote assigned to the predicted class.
A high confidence score indicates strong support from the reference neighbors. It is not a calibrated probability that the prediction is correct.
The confidence distribution includes only nuclei with valid predictions. Its percentages are calculated over that population.
What does confidence filtering show?¶
The confidence-filtering analysis evaluates ALHENA after removing predictions below a chosen confidence threshold.
At each threshold, the results show:
- How many nuclei retain a valid prediction.
- The fraction of evaluated nuclei retained.
- Accuracy and macro F1 on those retained nuclei.
- The two baselines recalculated on the same retained nuclei.
Increasing the threshold reduces or preserves the number of predictions. Performance may improve or worsen as the evaluated population changes.
Nuclei without valid predictions are never retained, including at threshold 0.00.
Per-class recall in this analysis is calculated among the retained nuclei of that class. If none remain, no recall is reported for that class.
See Evaluating confidence-based filtering.
Important limitation: cell class is not malignancy¶
Although PanNuke distinguishes Neoplastic from Epithelial, both are
mapped to Epithelial_Melanocyte in this comparison.
The mapping assumes Neoplastic nuclei are epithelial or melanocytic tumor cells, such as carcinoma or melanoma cells.
ALHENA does not predict whether a nucleus is malignant. Therefore, agreement on the mapped class does not demonstrate tumor recognition.
The PanNuke labels also do not distinguish the finer immune classes needed
to evaluate lineage_v1.
Image scale¶
PanNuke fields are 256 by 256 pixels at a declared, nominal resolution of
0.25 micron per pixel (mpp=0.25). This value is not guaranteed to reflect
the native optical resolution.
They cover a smaller area than the model's input field of view. ALHENA therefore places the image within a larger input and pads the remaining area with white pixels.
This may affect predictions because the model sees a partially blank image context.
The calculation below shows how much of the model input is occupied by the original image.
See Geometry for further details.
side, covered = demo.canvas_geometry(demo.PANNUKE_TILE_PX)
print("field", demo.PANNUKE_TILE_PX, "px at", demo.PANNUKE_MPP, "micron per pixel")
print("covering", demo.PANNUKE_TILE_PX * demo.PANNUKE_MPP, "micron of tissue")
print("on the", alhena.INPUT_SIZE, "px canvas:", side, "px,", round(100 * covered, 1), "percent")
field 256 px at 0.25 micron per pixel covering 64.0 micron of tissue on the 224 px canvas: 151 px, 45.4 percent
Optional: reproduce the workflow using the Python API¶
The demo above already runs the complete workflow.
The following cells show how to perform the same steps directly using ALHENA's public Python API. They are optional and repeat the computation, including loading the model again.
demo.fetch_pannuke() retrieves the selected images, nucleus annotations,
and PanNuke labels.
The first N_TILES fields of the selected tissue are used in file order.
See Label maps for details about how nucleus annotations are represented.
with warnings.catch_warnings(record=True) as raised:
warnings.simplefilter("always")
cohort = demo.fetch_pannuke(tissue=TISSUE, n_tiles=N_TILES, parquet=PARQUET)
for entry in raised:
print(type(entry.message).__name__, entry.message)
print(cohort.tissue, "->", cohort.reference_tissue)
print(cohort.n_nuclei, "non-Dead nuclei before processing exclusions")
# Load the tissue reference, or use the local reference file when REFERENCE is set.
typer = alhena.load_celltyper(
tissue=cohort.reference_tissue if REFERENCE is None else None,
reference=REFERENCE,
axis=demo.DEMO_AXIS,
backbone=BACKBONE,
bundle=BUNDLE,
)
result = typer.annotate(cohort.images, cohort.label_maps, mpp=demo.PANNUKE_MPP)
print("k", result.meta["k"], "tau", result.meta["tau"])
print("classes absent from the reference:", result.meta["unexpressed_classes"])
result.to_frame()[["cell_id", "label", "confidence", "valid"]].head()
GeometryWarning PanNuke declares 0.25 microns per pixel, which is nominal and resampled rather than native optical: fields scanned at 20x were enlarged to 40x by the authors, and the release marks none of them, so they cannot be filtered out. The declared resolution is passed on as it stands. Breast -> Breast 72 non-Dead nuclei before processing exclusions
k 30 tau 0.03 classes absent from the reference: ()
| cell_id | label | confidence | valid | |
|---|---|---|---|---|
| 0 | 1 | Immune | 0.541217 | True |
| 1 | 2 | Epithelial_Melanocyte | 1.000000 | True |
| 2 | 3 | Epithelial_Melanocyte | 1.000000 | True |
| 3 | 4 | Epithelial_Melanocyte | 1.000000 | True |
| 4 | 5 | Immune | 1.000000 | True |
Optional: evaluate the predictions with the Python API¶
The next cell compares ALHENA's predictions with the corresponding PanNuke labels.
align_truth() aligns the PanNuke labels with the rows ALHENA returns,
using result.cell_id, rather than assuming they appear in the same order.
Failed predictions remain in the evaluated population and count as errors. Nuclei excluded before prediction are not evaluated.
With macro_min_support=0, every class present among evaluated nuclei
contributes to macro F1.
score() calculates ALHENA's metrics and the two baselines on the same
evaluated population.
confidence_sweep() recalculates the metrics after retaining only valid
predictions meeting each confidence threshold. Its retained_fraction
uses the original evaluated population as the denominator.
See Evaluating confidence-based filtering.
For a detailed evaluation with figures, see Tutorial 6.
truth = alhena.align_truth(result, cohort.truth_by_cell())
# The scores and the two baselines are fields of the report, such as report.macro_f1.
report = alhena.score(result, truth, macro_min_support=0)
print(report.n_cells, "evaluated nuclei on", result.axis)
# The sweep holds the same scores at each threshold, beside the predictions it keeps.
sweep = alhena.confidence_sweep(result, truth, thresholds=demo.DEMO_THRESHOLDS)
sweep[["threshold", "n_retained", "retained_fraction"]]
72 evaluated nuclei on compartment_v1
| threshold | n_retained | retained_fraction | |
|---|---|---|---|
| 0 | 0.0 | 72 | 1.000000 |
| 1 | 0.4 | 72 | 1.000000 |
| 2 | 0.5 | 70 | 0.972222 |
| 3 | 0.6 | 68 | 0.944444 |
| 4 | 0.7 | 65 | 0.902778 |
| 5 | 0.8 | 64 | 0.888889 |
| 6 | 0.9 | 63 | 0.875000 |
What this demonstration establishes¶
This notebook demonstrates the complete ALHENA workflow, from H&E image processing and nucleus annotation to zero-shot cell typing using a single-cell RNA-seq reference and comparison with external labels.
The reported accuracy and F1 values describe agreement with PanNuke labels on the selected images. They are not estimates of ALHENA's general performance on Breast tissue.
Additional limitations of this particular dataset include:
- PanNuke provides no slide identifiers for determining how many independent slides are represented.
- Its declared image resolution is nominal.
- The evaluation uses broad cell classes and does not measure recognition of malignant cells.
PanNuke is distributed under CC BY-NC-SA 4.0: non-commercial use only, with attribution and share-alike. ALHENA reads it from the dataset repository and does not redistribute it.
Next: Choose your task