Machine Learning Model Accurately Predicts Intravenous Immunoglobulin Non-Responders For Kawasaki Disease

Non-responders to intravenous immunoglobulin (IVIG) for treating Kawasaki disease (KD) were accurately predicted using machine learning models according to the results of a study published in Clinical Drug Investigation.¹

Image credit: Soni's | stock.adobe.com

Untreated KD can lead to coronary artery abnormalities (CAA) in some patients. IVIG significantly reduced CAA; however, some patients with KD do not respond to treatment with IVIG and instead exhibit persistent fever or recurrent symptoms. About 15% to 25% of patients with KD do not respond to initial IVIG treatment.¹

Due to the difficulty of predicting IVIG resistance in patients with KD, the investigator’s primary outcome was to use machine learning techniques to develop a predictive scoring system to identify non-responders to IVIG that could be thoroughly tested for reliability and accuracy.¹

In total, 1441 patients were included in the study; 216 patients were excluded from the data set due to a lack of significant laboratory results, and 10 were excluded because they did not receive IVIG treatment during their hospitalization.¹

The exposure group comprised of 228 patients (15.8%) that were in the IVIG non-responder group, and the control group had 997 patients (69.2%) that were in the IVIG responder group. Between these groups, 99 patients were identified as being at high risk of IVIG non-responsiveness and received a combination of IVIG and prednisolone during their initial treatment.¹

Four factors were selected to create a scoring system for the prognosis of IVIG resistance. Each cutoff level for the variables were determined based on Shapley additive explanations (SHAP) values: “Day of the first treatment” (cutoff value: < 4 days), Na (cutoff value: < 133 mEq/L), total bilirubin level (cutoff value: > 0.5 mg/dL), and neutrophil-to-lymphocyte ratio (NLR) (cutoff value: > 2.6).¹

These 4 factors were combined to develop the Shizuoka score, which served as the investigator’s newly developed predictive scoring system for IVIG resistance in KD. They noted that a Shizuoka score exceeding 2 showed an area under the curve of 0.716 (95% CI, 0.669-0.762), with a sensitivity of 0.691 and a specificity of 0.629.¹

The secondary outcome for the investigators was to utilize the Shizuoka score to stratify IVIG responders and compare the outcomes of IVIG treatment to those treated with IVIG-added prednisolone treatment in the high-risk groups.¹

In a separate investigation, IVIG non-responders were evaluated using the new scoring system. Among 842 patients, there was a sensitivity of 0.843 (95% CI, 0.781-0.893), a specificity of 0.505 (95% CI, 0.466-0.543), a positive predictive value of 0.313 (95% CI, 0.272-0.357), and a negative predictive value of 0.923 (95% CI, 0.890-0.948), according to the study authors.¹

A comparative analysis within the Shizuoka high-scoring patients found that the incidence of IVIG non-responders was significantly elevated in the initial IVIG group, which comprised 152 patients (32.1%). This is opposed to the IVIG-added prednisolone group, which accounted for 8 patients (9.4%).¹

Several other models have been proposed in previous literature to predict IVIG non-response in patients with KD. A comprehensive meta-analysis by Kuniyoshi et al. aimed to evaluate the most accurate developed prediction models for IVIG resistance.²

According to the results, all 5 models had low positive predictive values (0.14-0.39) and high negative predictive values (0.85-0.92).² The Shizuoka score aligned with these findings, yielding a positive predictive value of 0.272 to 0.357 and a negative predictive value of 0.890-0.948, showing that “although simplified, the Shizuoka score is comparable in performance to prior models,” according to the investigators of the current study.¹

It is crucial that external validation using independent datasets is completed for assessing the generalizability of machine learning models such as this one. Many prediction models using machine learning are hampered by a high risk of bias and poor methodological quality.

Given these considerations, the establishment of an IVIG non-responder predictive model “warrants rigorous endeavors aimed at enhancing the design, execution, reporting, and validation,” of such a model, which are “indispensable for facilitating seamless integration into clinical practice and alleviating research inefficiency,” according to the study authors.¹

References

1. Miyagi Y, Iwashima S. Prediction models for intravenous immunoglobulin non-responders of Kawasaki disease using machine learning. Clin Drug Investig. 2024;44:425-437. doi:10.10007/s40261-024-01373-z

2. Kuniyoshi Y, Tsujimoto Y, Banno M, et al. Prediction models for intravenous immunoglobulin resistance in Kawasaki disease: a meta-analysis. Pediatrics. 2023;151(5):e2022059175. doi:10.1542/peds.2022-059175