Assessing Clinical Trial Efficacy vs Real-World Effectiveness in the Treatment of Multiple Myeloma

Although phase 3 clinical trials are the gold standard for regulatory approval of treatments and the basis by which evidence-based treatment guidelines are made, the efficacy of treatments (ie, the outcomes in the ideal clinical trial setting) are known to often be better than the effectiveness of those treatments (ie, the outcomes of patients in the real-world setting), explained Alissa Visram, MD, MPH, during a presentation at the 65th American Society of Hematology (ASH) Annual Meeting and Exposition. According to Visram, a hematologist at the Ottawa Hospital and an assistant professor in the Department of Medicine at the University of Ottawa Heart Institute, the difference between efficacy and effectiveness for treatments of certain specific disease states, such as multiple myeloma (MM), has remained unclear.¹

To investigate this further, Visram and her research team looked to compare clinical trial efficacy vs real-world effectiveness with regards to progression free survival (PFS) and overall survival (OS) in patients with newly diagnosed and relapsed/refractory (R/R) MM treated with standard of care regimens. The investigators started by making a list of all standard of care regimens reimbursed under Ontario's public health care plan as of December 2020, with 2020 set as a cutoff to ensure there was enough follow up time to assess outcomes.¹

“We then identified the key registrational trials that led to the provincial reimbursement of these regimens,” Visram said during the session. “Our real-world cohort was then identified using the [Institute for Clinical Evaluative Sciences] administrative database, which has health data for over 13 million Ontarians with public health care coverage.”¹

The investigators identified patients treated with 7 standard of care regimens for newly diagnosed and R/R MM regimens, including lenalidomide (Revlimid; Bristol Myers Squibb) plus dexamethasone (Rd) and bortezomib (Velcade; Takeda Pharmaceuticals) plus Rd for newly diagnosed transplant ineligible patients; the R/R MM regimens included carfilzomib (Kyprolis; Amgen) plus Rd, carfilzomib plus dexamethasone, daratumumab (Darzalex; Janssen Biotech) plus Rd, daratumumab plus bortezomib plus dexamethasone, and pomalidomide (Pomalyst; Bristol Myers Squibb) plus dexamethasone.^1,2

A total of 3951 patients were included in the real-world cohort. For the clinical trial cohort, the investigators generated individual patient level estimates of PFS and OS by digitizing the published Kaplan-Meier curves from pivotal phase 3 randomized clinical trials (RCTs), so that they could perform a meta-analysis.^1,2

Visram noted that the results showed that for R/R regimens, there was a longer time between MM diagnosis and start of the regimen in the real world vs RCT. Overall, patients included in the real-world cohort were also older than those in the RCTs.¹

Visram noted that the results showed that for R/R regimens, there was a longer time between MM diagnosis and start of the regimen in the real world vs RCT. Image Credit: © David A Litman - stock.adobe.com

With regards to the efficacy-effectiveness gap, Visram noted that patients with MM treated in routine practice in the real-world setting had a worse PFS, and this occurred despite the overestimation of real-world PFS compared to highly selected patients in the RCT setting for 6 of the 7 MM regimens evaluated, with a pooled HR of 1.44 (95% CI 1.34-1.54) in the meta-analysis. Similarly, real-world patients had a worse OS compared to patients in RCTs who were treated with 6 of the 7 regimens, with a pooled HR of 1.75 (95% ci 1.63-1.88) in the meta-analysis. Finally, patients with R/R MM in the real-world setting had higher rates of prior lenalidomide exposure compared to patients in RCTs.^1,2

The only regimen which showed a trend towards performing better in the real world as compared to the RCT setting was pomalidomide plus dexamethasone. Visram explained that the reason for this is likely multifactorial, but perhaps patients included in the RCTs may have had higher rates of R/R MM compared to real-world patients.^1,2

“What we see is that for 6 of the 7 standard of care regimens, the median PFS was at least 3 to 18 months longer in the clinical trial cohort, and the median OS was at least 19 months longer in the clinical trial compared to real-world patients. The only regimen where the median PFS and OS was comparable was in pomalidomide plus dexamethasone,” Visram said during the session.¹

In the meta-analysis, the investigators observed that the risk of progression or death was 44% higher in real world compared to the RCT setting. Additionally, the analysis showed that the risk of death is 75% higher in the real world compared to RCT patients treated with the same standard of care regimens.^1,2

“The differences were most apparent in patients treated with [R/R MM] regimens. Interestingly, patients treated with pomalidomide plus dexamethasone fared slightly better in the real world,” Visram said. “We think that this is because this is the only regimen where the prior treatment exposures were comparable between the real world and trial patients.”¹

Visram noted that this is one of the largest population level studies highlighting the significant efficacy-effectiveness gap across multiple standard of care regimens for MM. Further, the study highlights why it's important to use real-world data to see whether the reported study outcomes are generalizable to the patient population.¹

“Understanding the effectiveness of treatment is important for policymakers who regulate the use of drugs, but also for patients and clinicians to make more informed treatment decisions. These real-world studies on effectiveness are going to become increasingly relevant, especially as we evaluate complex treatments, such as immunotherapies, that are associated with significant toxicities,” Visram said. “Our next steps will be to identify the key contributors to the efficacy-effectiveness gap within our Ontario population, so that we can work towards overcoming them. But what we really need is for future trials to be more pragmatic in their trial design and more inclusive in their trial eligibility to see if this really reduces the efficacy-effectiveness gap.”¹

REFERENCES

1. Visram A. Focusing on Health Equity in Hematology. Comparison of the Efficacy in Clinical Trials Versus Effectiveness in the Real-World of Treatments for Multiple Myeloma: A Population-Based Cohort Study. Presented at: 65th ASH Annual Meeting and Exposition; Virtual on December 6, 2023.
2. Visram A, Chan KKW, Seow H, et al. Comparison of the Efficacy in Clinical Trials Versus Effectiveness in the Real-World of Treatments for Multiple Myeloma: A Population-Based Cohort Study. American Society of Hematology. 2023. Accessed December 9, 2023. https://ash.confex.com/ash/2023/webprogram/Paper189506.html