Reproducibility of Arterial Spin Labeling Cerebral Blood Flow image processing: A Report of The ISMRM Open Science Initiative for Perfusion Imaging and the ASL MRI Challenge

Purpose: Arterial Spin Labeling (ASL) is widely used in clinical research as a contrast-free MRI method for
assessment of cerebral blood flow (CBF). While the recommended guideline for ASL acquisition is
generally adopted to standardize quantification of CBF, ASL analysis still produces wide variability in CBF
estimates, limiting research and clinical interpretation of ASL results. This study explored the extent of
variability in ASL CBF quantification through the ISMRM OSIPI ASL MRI Challenge. The goal of the challenge
was to minimize sources of variability in ASL analysis by establishing best practice in ASL data processing
to make ASL analysis more reproducible and clinically meaningful.
Methods: Eight international teams analyzed the challenge data consisting of a high-resolution T1-
weighted anatomical image and ten pseudo-continuous ASL (PCASL) datasets. The datasets were
simulated using an ASL digital reference object to produce ground-truth CBF values in normal and
pathological states. The accuracy of CBF quantification from each team’s analysis was compared to
ground-truth values across all voxels and within pre-defined brain regions. Reproducibility of CBF
estimates across analysis pipelines was assessed using intra-class correlation coefficient (ICC), the limits
of agreement (LOA) and the replicability of generating similar CBF estimates from the image processing
approaches as documented.
Results: The absolute errors in CBF estimates compared to the ground-truth synthetic data ranged from
18.36 to 48.12 ml/100g/min. Realistic motion incorporated in three of the ten synthetic data produced
the largest absolute CBF error, largest variability between teams, and the least agreement (ICC and LOA)
with ground truth results. Fifty percent (4/8) of the teams’ methods were replicated, and one method
produced three times larger CBF errors (46.59 ml/100g/min) compared to submitted results.
Conclusions: The apparent variability in CBF measurements, influenced by differences in image processing
strategies, particularly in compensating for motion, demonstrates the significance for standardization of
ASL image analysis workflow. Therefore, we provide a recommendation for ASL image processing based
on top performing approaches as a step towards standardization of ASL imaging for clinical use.