Observational Study Designs of Metformin in Type 2 Diabetes Insufficient to Residual Confounding

Metformin, a generic, glucose-lowering medication, has been used to successfully treat type 2 diabetes (T2D) since 1995 in the United States.¹

Since then, various observational studies have attributed benefits of metformin to several end points and diseases in off-label usage, such as asthma, age-related macular degeneration, heart failure, and more. Certain observational studies have even suggested benefits of metformin usage to the likes of cancer treatment and prevention, antiaging, and neurodegenerative disease prevention among others. But do these study designs sufficiently address concerns about residual confounding? A recent cohort study found that, in fact, they do not.

The authors constructed an observational study inclusive of aspects frequently seen in metformin literature. Overall, study design in this space varies considerably (eg, incident use and prevalent use), comparison group selection (eg, metformin users vs nonusers and metformin users vs insulin users), (confounder selection (eg, general health indicators, diabetes severity indicators, comorbidity indices, laboratory values, and demographic characteristics), and statistical methods (eg, univariate or multivariate regression, matching, and various propensity score models) to name a few differences.

Although the authors explored the role residual confounding plays in these designs, they did not attempt to fix the limitations of the studies, such as by adjusting for diabetes duration, glycemic control, number and type of background diabetes treatments, or social determinants of health. However, they did expand the study design to include negative control outcomes, defined as outcomes with no direct, mechanistic connection to metformin, and included a complementary cohort (ie, patients with prediabetes), for which the authors expected any bias observed in the T2D population to be reversed.

Here, the study authors sought to determine whether common designs for observational studies of metformin are subject to substantial residual confounding and hypothesized that the inclusion of negative control outcomes and a complementary cohort could reveal residual confounding. This cohort included administrative claims data for individuals with Medicare Advantage or commercial insurance during the baseline year of 2018. The observational year was 2019 for the selected outcomes.

The study population included individuals aged 18 to 89 years who had both medical and pharmacy coverage with 24 months of continuous enrollment spanning across 2018-2019, and had at least 1 medical claim with a primary, secondary, or tertiary diagnosis of prediabetes or T2D in 2018. Importantly, the T2D cohort served as the primary analysis cohort because it represented the most metformin users and conforms to standard practice in the literature.

The T2D cohort included 404,458 participants (mean age 74.5, 52.7% female), and the prediabetes cohort was comprised of 81,791 participants. As evidenced in prior trials, the study found that there was a strong metformin impact estimate associated with reduced inpatient admissions (OR, 0.60; 95% CI, 0.58-0.62) and lower medical expenditures (OR, 0.57; 95% CI, 0.55-0.60) in the Medicare Advantage population.

Although metformin users did appear healthier across several health indicators, the inverse propensity weighting demonstrates acceptable covariate balance in both the T2D and prediabetes cohorts, as seen by the standardized mean differences for all covariates with absolute values less than 0.1. These effect sizes, small P values, large E-values, and covariate balance plots could suggest that metformin is associated with these outcomes.

The authors then considered these results as part of the more robust study design that involved negative control outcomes and a complementary prediabetes cohort. They found that the negative control outcomes exposed the biases in the primary and complementary cohorts.

“Although the observed covariates are well balanced after adjustment, the substantial residual confounding associated with overall health may influence our primary result,” the study authors wrote. “In the prediabetes cohort, the metformin users and nonusers were similar before balancing, making the groups appear comparable. Nevertheless, this same collection of negative control outcomes exhibited the opposite bias for metformin users (ie, a shift towards higher event rates.”

This finding underscores the failings of the study design to sufficiently address residual confounding associated with overall health and disease severity, according to the authors.

Overall, the observational study found that this more comprehensive study design strongly discredits the primary results and exemplifies how commonly published metformin study designs may not be sufficiently addressing residual confounding.

References

1. Bailey C. Metformin: historical overview. Diabetologia. 2017; 09;60(9):1566-1576. doi: 10.1007/s00125-017-4318-z. Epub 2017 Aug 3.
2. Powell M, Clark C, Alyakin A, et al. Exploration of residual confounding in analysis of associations of metformin use and outcomes in adults with type 2 diabetes. JAMA Netw Open. 2022;5(11):e2241505.doi:10.1001/jamanetworkopen.2022.41505