AI Model Can Differentiate Prefibrotic Primary Myelofibrosis From Essential Thrombocythemia With High Accuracy

There are overlapping clinical, molecular, and histopathological characteristics for prefibrotic primary myelofibrosis (prePMF) and essential thrombocythemia (ET) which lead to challenges in differentiating each disease and the correct therapeutic approach for treatment, explained Andrew Srisuwananukorn, MD, during a session at the 65th American Society of Hematology (ASH) Annual Meeting and Exposition. Srisuwananukorn, an internal medicine specialis in the Division of Hematology, Department of Internal Medicine at The Ohio State University Comprehensive Cancer Center in Columbus, OH, explained further that through the use of a novel artificial intelligence (AI) model, his team was able to distinguish between prePMF and ET in distinct clinical cohorts with high accuracy.^1,2

prePMF and ET are both myeloproliferative neoplasms (MPNs), although there is a significant difference in median overall survival, with prePMF at 11.9 years and ET at 22.2 years, according to Srisuwananukorn. This difference highlights the need to distinguish between the 2 diseases to select disease-specific therapeutic options.^1,2

prePMF and ET are both myeloproliferative neoplasms (MPNs), however there is a significant difference in median overall survival, with prePMF at 11.9 years and ET at 22.2 years, according to Srisuwananukorn. Image Credit: © MdBabul - stock.adobe.com

“[MPN] diseases are related in that they share a common biological underpinning with constitutive [janus kinase 2 (JAK2)] activation leading to downstream phenotypes, including elevated cell counts of hemoglobin platelets, as well as constitutional symptoms, including splenomegaly,” Srisuwananukorn said. “But outside of common forms of MPNs, it's been increasingly recognized that in a subset of [patients with ET, the disease] can have a slightly more aggressive behavior.”¹

In 2016, the World Health Organization differentiated this entity as a new disease termed prePMF, according to Srisuwananukorn. Compared to patients with ET, patients with prePMF are at higher risk for constitutional symptoms, major bleeding, and progression to overt forms of myelofibrosis and acute myeloid leukemia.¹

“As you can imagine, differentiating between these 2 diseases can be quite challenging as the diagnostic criteria for ET and prePMF both rely on similar characteristics, including clinical and laboratory abnormalities, mutational profiling, and assessment of the bone marrow biopsies, which can be subjective for analysis of the megakaryocyte morphologies, as well as fibrosis grading,” Srisuwananukorn said.¹

Because of these similar characteristics, there is high interobserver variability among pathologists when making these diagnoses, Srisuwananukorn explained. Consensus varies widely between 50% and 100%.^1,2

“There's a pressing need for improved diagnostics to differentiate between these 2 diseases,” Srisuwananukorn said. “We posit that a potential solution to this diagnostic dilemma is with the use of the novel computational toolsets within the realm of AI. We've developed an AI tool that's able to visualize only the bone marrow [BM] biopsies of the patients for its predictions of disease.”¹

To accomplish this, Srisuwananukorn and his colleagues developed a biologically-motivated AI algorithm that can diagnose prePMF and ET from BM biopsy digital whole-slide images (WSI). To train and validate the AI model, the investigators identified patients with a clinical or histopathological diagnosis of prePMF or ET as determined by the International Consensus Classification of Myeloid Neoplasms at the University of Florence in Italy (Florence) and Moffitt Cancer Center in Tampa, Florida (Moffitt). The period in Florence assessed in the trial was June 2007 to May 2023 and in Florida was between January 2013 and January 2022.^1,2

At both trial sites, the investigators digitized diagnostic haematoxylin and eosin–stained BM biopsy slides using slide scanners (Aperio AT2; Leica Biosystems). The training cohort was comprised of 200 patients from Florence (100 prePMF, 100 ET), and the external test cohort entailed 26 patients from Moffitt (6 prePMF, 20 ET).^1,2

Investigators then trained the model using 32,226 WSI. The selected pretrained neural network (RetCCL) had also been previously trained on 32,000 diagnostic WSIs to represent a histologically-informed model, according to Srisuwananukorn.^1,2

For model training, BM WSI were tessellated into representative image tiles extracted at 10-times magnification, or 302 microns per image dimensions. By using attention-based multiple instance learning, a prediction for each patient’s WSI was calculated. This method then automatically assigned a numeric weight to an image portion that represented its importance relative to the classification task at hand.^1,2

The performance of the model was assessed utilizing the area under the receiver operator curve (AUC), with a cutoff threshold for diagnosis classification determined by maximizing Youden’s Index. Further, attention scores were plotted as a heatmap across the BM WSI. These were then reviewed for morphological features by a hematopathologist for a qualitative assessment.^1,2

The investigators also had custom scripts written using an open-source AI framework (Slideflow), with model development performed on the Minerva High Performance Computer at Mount Sinai Hospital. Additionally, the investigators estimated evaluation time for a single WSI using a consumer-grade computer.^1,2

“Once we were finished with model optimization, and we were happy with the final parameters, we validated our model on 26 patients treated at Moffitt,” Srisuwananukorn said. “Now it is true that training an algorithm such as this does require pretty expensive and complex computational servers, but the deployment or the use of an algorithm really only entails using a consumer grade computer or laptop, or most mobile devices even. Our workflows are highly optimized, and we are able to produce a prediction on a whole slide in roughly 6 seconds or so.”¹

In the training cohort, investigators conducted 5-fold cross validation, which resulted in a mean AUC of 0.90 and standard deviation of 0.04. A final locked model re-trained on the entire training cohort resulted in an AUC of 0.90 upon evaluation of the test cohort. The classification threshold was balanced for sensitivity and specificity, with the final diagnostic classification accuracy in the test cohort at 92.3% with a sensitivity for prePMF diagnosis of 66.6% and specificity of 100%.^1,2

“But it's very important for us as clinicians and researchers to critique our AI models,” Srisuwananukorn said. “We wanted to visualize what the AI algorithm is seeing and using for itself, or in other words, opening the ‘black box,’ which is a really important step.”^1,2

Upon review of the slides with highest prediction value per class, Srisuwananukorn noted that attention heatmaps were able to highlight the model’s reliance on areas of cellular marrow without reliance on image artifacts or background. Using consumer-grade hardware, investigators evaluated previously unseen WSI in approximately 6.1 seconds (4.9 for preprocessing and 1.2 for evaluation).^1,2

Additionally, the AI was able to learn to highlight areas of importance for its predictions for prePMF for ET, according to Srisuwananukorn.¹

“I want to mention these areas were automatically learned from the AI algorithm. These were not prespecified by myself,” Srisuwananukorn said. “Reassuringly, it appeared that the AI algorithms are using areas of bone marrow cellularity as opposed to areas of fat or cortical bone or background tissue. To us, this algorithm seems to be biologically reasonable. And again, this is an important step as we use these algorithms in clinical practice.”¹

Additionally, Srisuwananukorn noted that there are ongoing analysis being conducted by his team to understand higher resolution at higher magnification morphologies, particularly at the mega carriers and other cells, which are able to differentiate between these 2 diseases. Further, Srisuwananukorn explained that this study is the largest image-based AI study investigating MPNs with external validation.^1,2

According to Srisuwananukorn, the AI model may help clinicians identify patients who would benefit from disease-specific therapies or enrollment onto clinical trials. Further, the use of a high-speed, low-cost algorithm that is capable of reliably distinguishing prePMF from ET with high specificity may support the democratization of MPN diagnosis and treatment in routine practice, as well as drive clinical trial accrual for biologically rational novel therapeutics.^1,2

Additionally, Srisuwananukorn noted that the model workflow is open source and currently available for use.¹

“Our open source packages are available and developed by our lead software developer James M Dolezal, MD, at the University of Chicago, who himself is a treating physician oncologist,” Srisuwananukorn said.¹

REFERENCES

1. Srisuwananukorn A. New Approaches to Old Diseases. Presented at: 65th American Society of Hematology Annual Meeting and Exposition; December 9-12; San Diego, CA.
2. Srisuwananukorn A, Loscocco GG, Kuykendall AA, et al. 901 Interpretable Artificial Intelligence (AI) Differentiates Prefibrotic Primary Myelofibrosis (prePMF) from Essential Thrombocythemia (ET): A Multi-Center Study of a New Clinical Decision Support Tool. American Society of Hematology. 2023. Accessed December 11, 2023. https://ash.confex.com/ash/2023/webprogram/Paper173877.html