Machine Learning Model Accurately Predicts Intravenous Immunoglobulin Non-Responders For Kawasaki Disease
By developing the Shizuoka score, which features a new predictive model using machine learning, patients with Kawasaki disease who are non-responders to intravenous immunoglobulin were able to be accurately identified.
Non-responders to intravenous immunoglobulin (IVIG) for treating Kawasaki disease (KD) were accurately predicted using machine learning models according to the results of a study published in Clinical Drug Investigation.1
Image credit: Soni's | stock.adobe.com
Untreated KD can lead to coronary artery abnormalities (CAA) in some patients. IVIG significantly reduced CAA; however, some patients with KD do not respond to treatment with IVIG and instead exhibit persistent fever or recurrent symptoms. About 15% to 25% of patients with KD do not respond to initial IVIG treatment.1
Due to the difficulty of predicting IVIG resistance in patients with KD, the investigator’s primary outcome was to use machine learning techniques to develop a predictive scoring system to identify non-responders to IVIG that could be thoroughly tested for reliability and accuracy.1
In total, 1441 patients were included in the study; 216 patients were excluded from the data set due to a lack of significant laboratory results, and 10 were excluded because they did not receive IVIG treatment during their hospitalization.1
The exposure group comprised of 228 patients (15.8%) that were in the IVIG non-responder group, and the control group had 997 patients (69.2%) that were in the IVIG responder group. Between these groups, 99 patients were identified as being at high risk of IVIG non-responsiveness and received a combination of IVIG and prednisolone during their initial treatment.1
Four factors were selected to create a scoring system for the prognosis of IVIG resistance. Each cutoff level for the variables were determined based on Shapley additive explanations (SHAP) values: “Day of the first treatment” (cutoff value: < 4 days), Na (cutoff value: < 133 mEq/L), total bilirubin level (cutoff value: > 0.5 mg/dL), and neutrophil-to-lymphocyte ratio (NLR) (cutoff value: > 2.6).1
These 4 factors were combined to develop the Shizuoka score, which served as the investigator’s newly developed predictive scoring system for IVIG resistance in KD. They noted that a Shizuoka score exceeding 2 showed an area under the curve of 0.716 (95% CI, 0.669-0.762), with a sensitivity of 0.691 and a specificity of 0.629.1
The secondary outcome for the investigators was to utilize the Shizuoka score to stratify IVIG responders and compare the outcomes of IVIG treatment to those treated with IVIG-added prednisolone treatment in the high-risk groups.1
In a separate investigation, IVIG non-responders were evaluated using the new scoring system. Among 842 patients, there was a sensitivity of 0.843 (95% CI, 0.781-0.893), a specificity of 0.505 (95% CI, 0.466-0.543), a positive predictive value of 0.313 (95% CI, 0.272-0.357), and a negative predictive value of 0.923 (95% CI, 0.890-0.948), according to the study authors.1
A comparative analysis within the Shizuoka high-scoring patients found that the incidence of IVIG non-responders was significantly elevated in the initial IVIG group, which comprised 152 patients (32.1%). This is opposed to the IVIG-added prednisolone group, which accounted for 8 patients (9.4%).1
Several other models have been proposed in previous literature to predict IVIG non-response in patients with KD. A comprehensive meta-analysis by Kuniyoshi et al. aimed to evaluate the most accurate developed prediction models for IVIG resistance.2
According to the results, all 5 models had low positive predictive values (0.14-0.39) and high negative predictive values (0.85-0.92).2 The Shizuoka score aligned with these findings, yielding a positive predictive value of 0.272 to 0.357 and a negative predictive value of 0.890-0.948, showing that “although simplified, the Shizuoka score is comparable in performance to prior models,” according to the investigators of the current study.1
It is crucial that external validation using independent datasets is completed for assessing the generalizability of machine learning models such as this one. Many prediction models using machine learning are hampered by a high risk of bias and poor methodological quality.
Given these considerations, the establishment of an IVIG non-responder predictive model “warrants rigorous endeavors aimed at enhancing the design, execution, reporting, and validation,” of such a model, which are “indispensable for facilitating seamless integration into clinical practice and alleviating research inefficiency,” according to the study authors.1
