Current Dementia Risk Assessment Models Limited in Ability to Predict Disease

Assessments of dementia risk using current multifactorial prediction models had high error rates, which suggests that these methods are limited in their ability to assess and prevent dementia, a new study published in the Journal of the American Medical Association shows.

Credit: Feng Yu - stock.adobe.com

“In these cohort studies, individualized assessments of dementia risk using existing risk prediction scores had high error rates. These findings suggest that the scores were of limited value in targeting people for dementia prevention,” the researchers determined.

The investigators used 4 risk prediction models, which are used to distinguish those who may be at high risk of dementia from low risk, to conduct their study. The models were the Cardiovascular Risk Factors, Aging, and Dementia (CAIDE) clinical score, CAIDE-APOE (apolipoprotein E)-supplemented, the Brief Dementia Screening Indicator (BDSI), and the Australian National University Alzheimer Disease Risk Index (ANU-ADRI).

Main outcomes the researchers sought in predicting the 10-year risk for dementia included concordance (C) statistics, detection rate, false-positive rate, and the ratio of true to false positives.

The trials were conducted using 2 large cohort studies: the UK Biobank study, which was used to predict 10-year dementia risk, and the British Whitehall II study, which was analyzed to replicate the findings with a 20-year follow-up among a younger cohort of participants. There were 465,929 participants in the trials who came from the UK Biobank study, whereas 4865 came from Whitehall II.

Throughout 10 years of follow-up, 3421 UK Biobank participants were diagnosed with dementia (7.5 per 10,000 person-years), the researchers found. The C statistics for all-cause dementia were 0.59 (95% CI, 0.58-0.60) for ANU-ADRI, 0.66 (95% CI, 0.65-0.67) for CAIDE clinical, 0.68 (95% CI, 0.67-0.69) for BDSI, and 0.73 (95% CI, 0.72-0.73) for CAIDE-APOE-supplemented.

C statistics were highest among all dementia subtypes when only modeling based on a patient’s age. For each dementia risk score and age alone, the highest C statistics were seen in predicting vascular dementia, and the lowest were observed in frontotemporal dementia, the results of the study show.

Calibrating the test-positive threshold to detect more than 50% of incident dementia led to a true to false positives ratio of 1 to 88 and a 32% false-positive rate for CAIDE clinical and a false positives ratio of 1 to 66 and a false-positive rate of 24% for CAIDE-APOE-supplemented, the investigators wrote.

A true to false positives ratio of 1 to 87 and a false-positive rate of 34% was observed in BDSI, and a true to false positives ratio of 1 to 116 and a false-positive rate of 43% were seen for ANU-ANDRI.

In turn, depending on the risk score, a test-positive threshold that kept the false-positive rate at or lower than 5% missed 8.4 to 9.1 per 10 participants with incident dementia, with a lower error rate when analyzing risk assessment based on age, the investigators observed.

In determining a 20-year dementia prediction using the Whitehall II study to replicate their findings, researchers found that estimates were not materially different than those found in the main analysis. An exception, though, was that the true to false positives ratio improved in the Whitehall II participants because a larger proportion of participants were diagnosed with incident dementia.

In predicting half of future dementia cases using the prediction scores analyzed—CAIDE, BDSI, and ANU-ADRI—the study authors found that each correct prediction corresponded to 66 to 116 false-positive predictions. Further, if the test-positive threshold was defined to provide a false-positive rate at or below 5%, then these predictive scores missed 84% to 91% of incident dementia, according to the investigators.

“These data suggest that population stratification and individualized assessment of dementia risk using existing prediction algorithms have high error rates,” the researchers wrote in their discussion.

The investigators determined that false-positive results in assessing dementia risk could come with undue psychological harm, as it shoulders the burden of a potentially developing an incurable disease—and the myriad lifestyle and health changes that accompany it—onto an individual who will not actually do so.

Further, remedying the issue of false positives by raising the test-positive threshold still missed 8 to 9 per 10 participants who developed incident dementia; therefore, many people who will develop dementia would not be informed of their risk, investigators wrote.

The researchers agreed on the need for further study in developing more accurate risk assessment models for dementia.

“Ideally, risk markers used in algorithms would be surrogate markers responsive to change in risk, as such markers could inform clinical decisions on individualized preventive strategies, a goal increasingly adopted in modern biomarker-based risk prediction tools for chronic conditions,” the investigators concluded.

Reference

Kivimäki, M., et. al. (2023) Estimating dementia risk using multifactorial prediction models. Journal of the American Medical Association. Available at https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2805965. Accessed June 19, 2023.