A sensitivity analysis framework for generalizing randomized clinical trial results in the presence of unmeasured treatment effect modifiers

ABSTRACT Randomized controlled trials (RCTs) are the gold standard for assessing interventions for preventing and treating cancer, but their external validity is only guaranteed if the trial participants are a random sample from the target population. Unfortunately, most cancer-related RCTs use convenience samples, not probability samples, and differences between the trial sample and the target population are likely to exist. If these differences are related to the effectiveness of the treatment being studied (“effect modifiers”), trial results will fail to generalize. While observable differences may be assessed and potentially adjusted for (e.g., underrepresentation of certain demographic groups), these differences have been shown to not completely explain the so-called efficacy-effectiveness gap. We posit that unmeasured differences between who chooses to participate in an RCT and who does not may be an important contributor to the failure of some trial results to generalize. In this project, we propose to develop a statistical framework for quantifying the potential impact of unmeasured differences between the trial sample and the target population on trial results. The resulting sensitivity analysis will bound the potential bias in the treatment effect estimate when generalizing from the trial sample to a target population. The methodology will be based on our prior work developing sensitivity analyses in the areas of survey nonresponse and selection bias which similarly consider the issue of differences between who is in a study sample and who is not. This work will have broad applicability beyond cancer trials, as generalizability is a universal concern of randomized trials across application areas.