Assessing Clinical Trial Efficacy vs Real-World Effectiveness in the Management of Multiple Myeloma

Although phase 3 clinical trials are the gold standard for regulatory approval of treatments and the basis on which evidence-based treatment guidelines are made, the efficacy of treatments (ie, the outcomes in the ideal clinical trial setting) are known to often be better than the effectiveness of those treatments (ie, the outcomes of patients in the real-world setting), Alissa Visram, MD, MPH, explained during a presentation at the 65th American Society of Hematology Annual Meeting & Exposition in San Diego, California. According to Visram, a hematologist at the Ottawa Hospital in Ontario, Canada, and an assistant professor in the Department of Medicine at the University of Ottawa Heart Institute, the difference between efficacy and effectiveness for management of certain specific disease states, such as multiple myeloma (MM), has remained unclear.¹

Image credit: David A Litman | stock.adobe.com

To investigate this further, Visram and her research team looked to compare clinical trial efficacy vs real-world effectiveness with regard to progression-free survival (PFS) and overall survival (OS) in patients with newly diagnosed and relapsed/refractory (R/R) MM treated with standard-of-care regimens. The investigators started by making a list of all standard-of-care regimens reimbursed under Ontario’s public health care plan as of December 2020, with 2020 set as a cutoff to ensure there was enough follow-up time to assess outcomes.¹

“We then identified the key registrational trials that led to the provincial reimbursement of these regimens,” Visram said during the session. “Our real-world cohort was then identified using the [Institute for Clinical Evaluative Sciences] administrative database, which has health data for over 13 million Ontarians with public health care coverage.”¹

The investigators identified patients treated with 7 standard-of-care regimens for newly diagnosed and R/R MM. These regimens included Rd (lenalidomide [Revlimid; Bristol Myers Squibb], dexamethasone) and bortezomib (Velcade; Takeda Pharmaceuticals) plus Rd for newly diagnosed, transplant-ineligible patients. The R/R MM regimens included carfilzomib (Kyprolis; Amgen) plus Rd, carfilzomib plus dexamethasone, daratumumab (Darzalex; Janssen Biotech) plus Rd, daratumumab plus bortezomib plus dexamethasone, and pomalidomide (Pomalyst; Bristol-Myers Squibb) plus dexamethasone.^1,2

A total of 3951 patients were included in the realworld cohort. For the clinical trial cohort, the investigators generated individual patient-level estimates of PFS and OS by digitizing the published Kaplan-Meier curves from pivotal phase 3 randomized clinical trials (RCTs) so that they could perform a meta-analysis.^1,2

Visram noted that the results showed that for R/R regimens, there was a longer time between MM diagnosis and start of the regimen in the real-world setting vs RCT setting. Overall, patients included in the realworld cohort were also older than those in the RCTs.¹

With regard to the efficacy-effectiveness gap, Visram noted that patients with MM treated in routine practice in the real-world setting had a lower PFS and this occurred despite the overestimation of real-world PFS compared with highly selected patients in the RCT setting for 6 of the 7 MM regimens evaluated, with a pooled HR of 1.44 (95% CI, 1.34-1.54) in the meta-analysis. Similarly, patients in the real world had a lower OS compared with patients in RCTs who were treated with 6 of the 7 regimens, with a pooled HR of 1.75 (95% CI, 1.63-1.88) in the meta-analysis. Finally, patients with R/R MM in the real-world setting had higher rates of prior lenalidomide exposure compared with patients in RCTs.^1,2

The only regimen that showed a trend toward performing better in the real world compared with the RCT setting was pomalidomide plus dexamethasone. Visram explained that the reason for this is likely multifactorial but that patients included in the RCTs may have had higher rates of R/R MM compared with patients in the real world.^1,2

“What we see is that for 6 of the 7 standard-of-care regimens, the median PFS was at least 3 to 18 months longer in the clinical trial cohort and the median OS was at least 19 months longer in the clinical trial [cohort] compared with [that of] real-world patients. The only regimen where the median PFS and OS were comparable was in pomalidomide plus dexamethasone,” Visram said during the session.¹

In the meta-analysis, the investigators observed that the risk of progression or death was 44% higher in the real-world setting compared with the RCT setting. Additionally, the analysis showed that the risk of death is 75% higher for patients in the real world compared with patients in RCTs treated with the same standard-of-care regimens.^1,2

“The differences were most apparent in patients treated with [R/R MM] regimens. Interestingly, patients treated with pomalidomide plus dexamethasone fared slightly better in the real world,” Visram said. “We think this is because this is the only regimen where the prior treatment exposures were comparable between the real-world and trial patients.”¹

Visram noted that this is one of the largest population-level studies highlighting the significant efficacy-effectiveness gap across multiple standard-of-care regimens for MM. Furthermore, findings from the study highlight why it’s important to use real-world data to see whether the reported study outcomes are generalizable to the patient population.¹

“Understanding the effectiveness of treatment is important for policy makers who regulate the use of drugs but also for patients and clinicians to make more informed treatment decisions. These real-world studies on effectiveness are going to become increasingly relevant, especially as we evaluate complex treatments, such as immunotherapies, that are associated with significant toxicities,” Visram said. “Our next steps will be to identify the key contributors to the efficacy-effectiveness gap within our Ontario population so that we can work toward overcoming them. But what we need is for future trials to be more pragmatic in their trial design and more inclusive in their trial eligibility to see whether this reduces the efficacy-effectiveness gap.”¹

References

1. Visram A. Comparison of the efficacy in clinical trials versus ef fectiveness in the real-world of treatments for multiple myeloma: a population-based cohort study. Presented at: 65th American Society of Hematology Annual Meeting & Exposition; December 9-12, 2023; San Diego, CA.

2. Visram A, Chan KKW, Seow H, et al. Comparison of the efficacy in clinical trials v ersus effectiveness in the real-world of treatments for multiple myeloma: a population-based cohort study. Abstract presented at: 65th American Society of Hematology Annual Meeting & Exposition; December 9-12, 2023; San Diego, CA. Abstract 541. Accessed December 12, 2023. https://ash.confex.com/ash/2023/webprogram/Paper189506.html