A machine learning model demonstrated the potential to improve early detection and management of Alzheimer disease (AD), but did not satisfy fairness metrics across race and ethnicity, according to results of a study published in JAMA Network Open.
Investigators of the study aimed to determine the potential of predictive models in the early detection and management of the disease as well as the tools' fairness among racial and ethnic groups for AD progression. Investigators included data available from the Alzheimer Disease Neuroimaging Initiative, which included patients aged 54 to 91 years across 57 sites in the United States and Canada. All individuals had at least 2 visits between September 2005 and May 2017, where clinical status was classified as cognitively normal, mild cognitive impairment (MCI), or AD, according to the study authors.
A total of 1730 individuals were included, with 44.9% female, 4% Hispanic, 1.7% Asian, 4.5% Black, 92.4% White, and 1.4% identified as another race. The mean age was 73.81 years old. Approximately 86.2% were cognitively normal, 13.8% were cognitively normal-MCI trajectory, 62.4% were MCI-stable trajectory, and 38,6% were MCI-AD trajectory, according to the study.
3 Key Takeaways
- A study explored the potential of machine learning models, including logistic regression, support vector machine, and recurrent neural network, in predicting AD progression.
- The study assessed the fairness of machine learning models to evaluate the models' performance across sex, ethnicity, and race, revealing insights into potential disparities in AD prediction.
- Findings highlighted disparities in true positive rates across different trajectories of AD progression, emphasizing how underrepresentation, particularly in smaller sample sizes for certain ethnic groups, could contribute to unfairness in predicting AD progression.
For the evaluation of fairness, investigators used data on sex, ethnicity, and race, which were all patient reported. To measure the fairness, the study authors said the metrics of equalized odds, equal opportunity, and demographic parity were used. Three machine learning models were used, including logistic regression (LR), support vector machine (SVM), and recurrent neural network (RNN), according to the study authors. LR and SVM are well-established models used in prediction, which makes them important to use as comparators for new models of machine learning. The RNN has shown potential in identifying AD progression, according to the study authors.
The results showed that the true positive rates between cognitively normal-stable and MCI-stable was close to 1, with no major differences between males and females. Further, in the models, the cognitively normal-stable trajectory difference between males and females was 0.5% in the LR, 0.5% for SVM, and 0.6% for RNN. For clinically normal to MCI, the study authors reported notable differences between males and females, with all models tending to perform better for females. For MCI to AD, there were small differences observed.
Additionally, investigators found that across all models and trajectories, the true positive rates were higher for non-Hispanic individuals and Hispanic individuals, with 1.4% for LR, 2.2% for SVM, and 2.1% for RNN. The differences in true positive rates were larger for the cognitively normal to MCI trajectory and the MCI to AD trajectory. Additionally, the cognitively normal-stable trajectory for true positive rates was higher for White individuals and lower for all other groups across the 3 models. MCI-stable trajectory was similar to the previous results.
Furthermore, for cognitively normal to MCI trajectory, those who were Asian had higher true positive rates than other groups for the SVM model. The study authors also said that Black individuals "in the MCI group at baseline tended to be incorrectly predicted to transition to AD."
The results highlight how underrepresentation could affect unfairness. For Hispanic patients, the true positive rates were high due to the small sample size, according to the study authors. Further, they said this was true for Asian, Black, and other racial groups, especially for the cognitively normal to MCI and MCI to AD trajectories.
Reference
Yuan C, Linn KA, Hubbard RA. Algorithmic Fairness of Machine Learning Models for Alzheimer Disease Progression. JAMA Netw Open. 2023;6(11):e2342203. 2023. doi:10.1001/jamanetworkopen.2023.42203
