Attention based deep 3d multiple instance survival models for oropharyngeal carcinoma patients

Introduction
Attention-based convolutional neural networks (CNNs) have the capability to use multiple parts of the same image to predict outcomes of interest. Especially in the domain of medical image analysis, where whole images are typically described by a single label but the identification of important image regions is unclear, this approach allows to combine competitively performing CNNs with enhanced interpretability of the decision-making process.

Materials & Methods
We developed risk models for the prediction of overall survival (OS) for 518 patients of a publicly available oropharyngeal carcinoma (OPC) cohort. Patients were randomly split into training, validation, and test cohorts (388/30/100 patients). A baseline Cox model using clinical information only and three attention-based CNNs using different likelihood functions were trained on multiple 3D instances of the pre-treatment computed tomography (CT) images. Subsequently, patients were stratified into groups at low and high risk of death using median cutoff values based on predictions determined on the training cohort. Model performance was measured using the concordance index (C-index) and differences between Kaplan-Meier curves were assessed by the log-rank test.

Results
The baseline Cox model achieved a C-Index of 0.22 and the CNN models based on the Cox, Weibull and Lognormal likelihood functions achieved C-indices of 0.34, 0.35 and 0.35, respectively, on the test cohort. All models stratified the patients into two risk groups with a statistically significant difference in OS. Attention scores between the multiple instances of a patient were similar, suggesting that all CT instances were equally important for the network decision.

Summary
We investigated the potential of attention-based multiple-instance learning for prediction of OS on an OPC cohort. Since all attention-based CNNs generated risk groups with significantly different OS based on imaging data alone, we consider this approach promising for future validation studies.