Biases in study design, implementation, and data analysis that distort the appraisal of clinical benefit and ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) scoring.

Background: The European Society for Medical Oncology-Magnitude of Clinical Benefit Scale (ESMO-MCBS) is a validated, widely used tool developed to score the clinical benefit from cancer medicines reported in clinical trials. ESMO-MCBS scores assume valid research methodologies and quality trial implementation. Studies incorporating flawed design, implementation, or data analysis may generate outcomes that exaggerate true benefit and are not generalisable. Failure to either indicate or penalise studies with bias undermines the intention and diminishes the integrity of ESMO-MCBS scores. This review aimed to evaluate the adequacy of the ESMO-MCBS to address bias generated by flawed design, implementation, or data analysis and identify shortcomings in need of amendment.

Methods: As part of a refinement of the ESMO-MCBS, we reviewed trial design, implementation, and data analysis issues that could bias the results. For each issue of concern, we reviewed the ESMO-MCBS v1.1 approach against standards derived from Helsinki guidelines for ethical human research and guidelines from the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, the Food and Drugs Administration, the European Medicines Agency, and European Network for Health Technology Assessment.

Results: Six design, two implementation, and two data analysis and interpretation issues were evaluated and in three, the ESMO-MCBS provided adequate protections. Seven shortcomings in the ability of the ESMO-MCBS to identify and address bias were identified. These related to (i) evaluation of the control arm, (ii) crossover issues, (iii) criteria for non-inferiority, (iv) substandard post-progression treatment, (v) post hoc subgroup findings based on biomarkers, (vi) informative censoring, and (vii) publication bias against quality-of-life data.

Conclusion: Interpretation of the ESMO-MCBS scores requires critical appraisal of trials to understand caveats in trial design, implementation, and data analysis that may have biased results and conclusions. These will be addressed in future iterations of the ESMO-MCBS.

Competing Interests: Disclosure TA personal fees and travel grants from Bristol-Myers Squibb (BMS), personal fees, grants and travel grants from Novartis, personal fees from Pierre Fabre, grants from Neracare, Sanofi, and SkylineDx, personal fees from CeCaVa outside the submitted work; JBa reports grants, personal fees, and nonfinancial support from Ipsen; personal fees and nonfinancial support from Pfizer, Novartis; nonfinancial support from AAA, Nanostring, Roche; grants and personal fees from Servier; and personal fees from Nutricia outside the submitted work; JBo is a statistician on the SAG-O (scientific advice committee oncology) for EMA, and scientific director of EORTC Headquarters. EORTC carries out clinical trials with many (most) pharma and some biotechs either with financial or material support, or with the company as the sponsor (intent to register new indication); he is co-responsible for the management of EORTC. AC has received honoraria/consulting fees from AstraZeneca, Boehringer-Ingelheim, Pfizer, Roche/Genentech, Eli Lilly and Company, Takeda, Novartis, Merck Sharp & Dohme (MSD), and BMS; GC scientific advisory board for BMS, Roche, Novartis, Lilly, Pfizer, Seagen, AstraZeneca, Daichii Sankyo, and Veracyte; UD Tumour Agnostic Evidence Generation Working Group Member, Roche; BG reports receiving consulting fees from Vivio Health. CG-R BMS (institutional research and travel grants, speaker’s honoraria), Roche/Genentech (institutional research and travel grants, speaker’s honoraria), Pierre Fabre (travel and educational grants, speaker’s honoraria); MSD (travel grants); Eisai (speaker’s honoraria); Foundation Medicine (research grant, speaker’s honoraria); BK Honoraria for lectures from Ipsen, Novartis and MSD; SO research grants from Celldex and Novartis to the institution; AP received honoraria for consulting, advisory role or lectures from AstraZeneca, Agilent/Dako, Boehringer Ingelheim, BMS, Eli Lilly, Janssen, MSD, Pfizer and Roche Genentech; GP received institutional financial support for advisory board/consultancy from Roche, Amgen, Merck, MSD, BMS, and institutional support for clinical trials or contracted research from Amgen, Roche, AstraZeneca, Pfizer, Merck, BMS, MSD, Novartis, Lilly; MP consulting or advisory role: AstraZeneca, Lilly, MSD, Novartis, Pfizer, Roche, Genentech, Crescendo Biologics, Periphagen, Huya Bioscience, Debiopharm Group, Odante Therapeutics; Consulting or Advisory Role: G1 Therapeutics, Menarini, Seattle Genetics, Camel-IDS, Immunomedics, Oncolytics, Radius Health; Research Funding: AstraZeneca (Inst), Lilly (Inst), MSD (Inst), Novartis (Inst), Pfizer (Inst), Roche (Inst), Genentech (Inst), Radius Health (Inst), Synthon (Inst), Servier (Inst); Other Relationship: Radius Health; JT personal financial interest in the form of scientific consultancy role for Array Biopharma, AstraZeneca, Bayer, Boehringer Ingelheim, Chugai, Daiichi Sankyo, F. Hoffmann-La Roche Ltd, Genentech, Inc., HalioDX SAS, Ikena Oncology, IQVIA, Imedex, Lilly, MSD, Menarini, Merck Serono, Mirati, Novartis, Peptomyc, Pfizer, Pierre Fabre, Samsung Bioepis, Sanofi, Seattle Genetics, Servier, Taiho, Tessa Therapeutics and TheraMyc. Institutional financial interest in the form of financial support for clinical trials or contracted research for Amgen Inc., Array Biopharma Inc., AstraZeneca Pharmaceuticals LP, Debiopharm International SA, F. Hoffmann-La Roche Ltd, Genentech Inc., Janssen-Cilag SA, MSD, Novartis Farmacéutica SA, Taiho Pharma USA Inc., Pharma Mar, Spanish Association Against Cancer Scientific Foundation and Cancer Research UK; EGEdV declares: institutional financial support for advisory board/consultancy from Sanofi, Daiichi, Sankyo, NSABP, Pfizer and Merck, and institutional support for clinical trials or contracted research from Amgen, Genentech, Roche, AstraZeneca, Synthon, Nordic Nanovector, G1 Therapeutics, Bayer, Chugai Pharma, CytomX Therapeutics, Servier and Radius Health; CZ consultancies and speaker’s honoraria: Roche, Novartis, BMS, MSD, Imugene, Ariad, Pfizer, Merrimack, Merck KGaA, Fibrogen, AstraZeneca, Tesaro, Gilead, Servier, Shire, Eli Lilly, Athenex. Institution (Central European Cooperative Oncology Group): BMS, MSD, Pfizer, AstraZeneca, Servier, Eli Lilly; GZ received speaker’s honoraria from Amgen and Ipsen. All other authors have declared no conflicts of interest.

(Copyright © 2021 The Author(s). Published by Elsevier Ltd.. All rights reserved.)