General Session: Value and Outcomes in Spine Surgery - Hall F

Presented by: A. Araghi

Author(s):

J. Batts(1), A. Araghi(2)

(1) Telos Partners, Broomfield, CO, United States
(2) The CORE Institute, Phoenix, AZ, United States

Abstract

Purpose: Most investigational device exemption (IDE) trials of spinal devices for the FDA are designed as non-inferiority. Many of these non-inferiority trials proceed to test for statistical superiority of the investigational device post-hoc once non-inferiority has been substantiated. There are many nuances associated with this strategy that make interpretation of the superiority claim difficult. Superiority trial designs define all of the superiority criteria a priori and the effect size can be readily interpreted, especially when valid minimum clinically important differences (MCID) are employed. This study reviewed peer-reviewed literature of IDE trials for spinal devices that were published since 2000 to identify the frequency of non-inferiority trials that make superiority claims as well as how often those trials account for an MCID or discuss the superiority effect size in terms of clinical relevance.

Methods: A PubMed search was conducted using the search terms 'spine AND (investigational device exemption OR IDE)' between 2000 and 2017. The search was narrowed to only include articles in English and those providing details on the study design with follow-up of at least 2 years. Single arm studies were not included. Resulting articles were reviewed for study design, whether superiority was concluded and use of MCID.

Results: The initial search returned 192 publications related to IDE studies. There were 22 unique studies (40 publications) that met the inclusion/exclusion criteria. Of those 22 IDE studies, 21 (95%) were non-inferiority designs and 1 (5%) was a superiority design. Fourteen (64%) of the non-inferiority studies made superiority conclusions. In all 14 of those studies, the superiority claim was solely based on statistical difference between the experimental and control groups, without any criteria for a MCID or discussion of the clinical significance of the effect size.

Conclusion: The relevance of statistical superiority to treatment efficacy is not clinically meaningful without consideration of the effect size and the MCID, which brings into question the impact of such conclusions derived from a non-inferiority trial in which no clinical superiority criteria are defined a priori. We urge researchers to implement superiority trial designs using a priori definitions of superiority that are clinically relevant, especially when evaluating novel technologies against standard of care. In this way, health care providers and payers will make evidence-based decisions with greater confidence in the reported clinical outcomes.