AI Model Differentiates Prefibrotic Primary Myelofibrosis From Essential Thrombocythemia With High Accuracy

There are overlapping clinical, molecular, and histopathological characteristics for prefibrotic primary myelofibrosis (prePMF) and essential thrombocythemia (ET) that lead to challenges in differentiating each disease and the correct therapeutic approach for treatment, Andrew Srisuwananukorn, MD, explained during a session at the 65th American Society of Hematology Annual Meeting & Exposition in San Diego, California. Srisuwananukorn, a physician with a specialty in internal medicine and hematology at The Ohio State University Comprehensive Cancer Center – Arthur G. James Cancer Hospital and Richard J. Solove Research Institute in Columbus, explained further that with a novel artificial intelligence (AI) model, his team was able to distinguish between prePMF and ET in distinct clinical cohorts with high accuracy.^1,2

Image credit: kras99 | stock.adobe.com

prePMF and ET are both myeloproliferative neoplasms (MPNs); however, there is a significant difference in median overall survival, with prePMF at 11.9 years and ET at 22.2 years, according to Srisuwananukorn. This difference highlights the need to distinguish between the 2 diseases to select disease-specific therapeutic options.^1,2

“[MPN] diseases are related in that they share a common biological underpinning with constitutive [Janus kinase 2] activation leading to downstream phenotypes, including elevated cell counts of hemoglobin platelets as well as constitutional symptoms, including splenomegaly,” Srisuwananukorn said. “But outside common forms of MPNs, it’s been increasingly recognized that in a subset of [patients with ET, the disease] can have slightly more aggressive behavior.”¹

In 2016, the World Health Organization differentiated this entity as a new disease termed prePMF, according to Srisuwananukorn. Compared with patients with ET, patients with prePMF are at higher risk for constitutional symptoms, major bleeding, and progression to overt forms of myelofibrosis and acute myeloid leukemia.¹

“As you can imagine, differentiating between these 2 diseases can be quite challenging [because] the diagnostic criteria for ET and prePMF both rely on similar characteristics, including clinical and laboratory abnormalities, mutational profiling, and assessment of the bone marrow [BM] biopsy [results], which can be subjective for analysis of the megakaryocyte morphologies as well as fibrosis grading,” Srisuwananukorn said.¹

Because of these similar characteristics, there is a high interobserver variability among pathologists when making these diagnoses, Srisuwananukorn explained. Consensus varies widely between 50% and 100%.^1,2

“There’s a pressing need for improved diagnostics to differentiate between these 2 diseases,” Srisuwananukorn said. “We posit that a potential solution to this diagnostic dilemma is with the use of the novel computational tool sets within the realm of AI. We’ve developed an AI tool that’s able to visualize only the bone marrow biopsy [results] of the patients for its predictions of disease.”¹

To accomplish this, Srisuwananukorn and his colleagues developed a biologically motivated AI algorithm that can diagnose prePMF and ET from BM biopsy digital whole-slide images (WSIs). To train and validate the AI model, the investigators identified patients with a clinical or histopathological diagnosis of prePMF or ET as determined by the International Consensus Classification of myeloid neoplasms at the University of Florence in Italy and Moffitt Cancer Center in Tampa, Florida. The period at the University of Florence assessed in the trial was June 2007 to May 2023, and the period assessed at Moffitt Cancer Center was January 2013 to January 2022.^1,2

At both trial sites, the investigators digitized diagnostic hematoxylin- and eosin-stained BM biopsy slides using slide scanners (Aperio AT2 DX System; Leica Biosystems). The training cohort comprised 200 patients from the University of Florence (100 with prePMF and 100 with ET), and the external test cohort entailed 26 patients from Moffitt Cancer Center (6 with prePMF and 20 with ET).^1,2

Investigators then trained the model using 32,226 WSIs. The selected pretrained neural network (retrieval with clustering-guided contrastive learning) had also been previously trained on 32,000 diagnostic WSIs to represent a histologically informed model, according to Srisuwananukorn.^1,2

For model training, BM WSIs were tessellated into representative image tiles extracted at x10 magnification or 302 μm per image dimensions. By using attention-based multiple instance learning, a prediction for each patient’s WSI was calculated. This method then automatically assigned a numeric weight to an image portion that represented its importance relative to the classification task at hand.^1,2

The performance of the model was assessed using the area under the receiver operator curve (AUC), with a cutoff threshold for diagnosis classification determined by maximizing the Youden index. Furthermore, attention scores were plotted as a heat map across the BM WSI. These were then reviewed for morphological features by a hematopathologist for a qualitative assessment.^1,2

The investigators also had custom scripts written using an open-source AI framework (Slideflow), with model development performed on the Minerva Supercomputer at The Mount Sinai Hospital in New York, New York. Additionally, the investigators estimated evaluation time for a single WSI using a consumer-grade computer.^1,2

“Once we were finished with model optimization and we were happy with the final parameters, we validated our model on 26 patients treated at Moffitt [Cancer Center],” Srisuwananukorn said. “It is true that training an algorithm such as this does require expensive and complex computational servers, but the deployment or the use of an algorithm only entails using a consumer-grade computer or laptop or most mobile devices. Our workflows are highly optimized, and we are able to produce a prediction on a whole slide in roughly 6 seconds.”¹

In the training cohort, investigators conducted 5-fold cross-validation, which resulted in a mean AUC of 0.9 and SD of 0.04. A final locked model retrained on the entire training cohort resulted in an AUC of 0.9 upon evaluation of the test cohort. The classification threshold was balanced for sensitivity and specificity, with the final diagnostic classification accuracy in the test cohort at 92.3%, with a sensitivity for prePMF diagnosis of 66.6% and specificity of 100%.^1,2

“But it’s very important for us as clinicians and researchers to critique our AI models,” Srisuwananukorn said. “We wanted to visualize what the AI algorithm is seeing and using for itself, or in other words, [we were] opening the black box, which is a really important step.”¹

Upon review of the slides with highest prediction value per class, Srisuwananukorn noted that attention heat maps were able to highlight the model’s reliance on areas of cellular marrow without reliance on image artifacts or background. Using consumer-grade hardware, investigators evaluated previously unseen WSIs in approximately 6.1 seconds (4.9 for preprocessing and 1.2 for evaluation).1,2 Additionally, the AI was able to learn to highlight areas of importance for its predictions of prePMF or ET, according to Srisuwananukorn.¹

“These areas were automatically learned from the AI algorithm. These were not prespecified by myself,” Srisuwananukorn said. “Reassuringly, it appeared that the AI algorithms are using areas of bone marrow cellularity as opposed to areas of fat or cortical bone or background tissue. This algorithm seems to be biologically reasonable. This is an important step as we use these algorithms in clinical practice.”¹

Additionally, Srisuwananukorn noted that there are ongoing analyses being conducted by his team to understand higher resolution at higher magnification morphologies, particularly at the megakaryocytes and other cells, which can differentiate between these 2 diseases. Furthermore, Srisuwananukorn explained that this study is the largest image-based AI study investigating MPNs with external validation.

According to Srisuwananukorn, the AI model may help clinicians identify patients who would benefit from disease-specific therapies or enrollment in clinical trials. Furthermore, the use of a highspeed, low-cost algorithm that is capable of reliably distinguishing prePMF from ET with high specificity may support the democratization of MPN diagnosis and management in routine practice as well as drive clinical trial accrual for biologically rational novel therapeutics.^1,2

Additionally, Srisuwananukorn noted that the model workflow is open source and currently available for use.1 “Our open-source packages are available and developed by our lead software developer, James M. Dolezal, MD, at the University of Chicago [in Illinois], who himself is a treating physician oncologist,” Srisuwananukorn said.¹

References

1. Srisuwananukorn A. New approaches to old diseases. Presented at: 65th American Society of Hematology Annual Meeting & Exposition; December 9-12, 2023; San Diego, CA.

2. Srisuwananukorn A, Loscocco GG, Kuykendall AA, et al. Interpretable artificial intelligence (AI) differentiates prefibrotic primary myelofibrosis (prePMF) from essential thrombocythemia (ET): a multi-center study of a new clinical decision support tool. Abstract presented at: 65th American Society of Hematology Annual Meeting & Exposition; December 9-12, 2023; San Diego, CA. Abstract 901. Accessed December 12, 2023. https://ash.confex.com/ash/2023/webprogram/Paper173877.html