Clinical prediction rules: towards a framework for meta-analysis, validation, and updating

Karel Moons, Ewout Steyerberg

The past decades there is a clear shift from implicit to explicit diagnosis and prognosis. This includes appreciation of clinical -diagnostic and prognostic- prediction models or rules, which is likely to increase with the introduction of fully computerized patient records. Prediction models aim to provide a probability of disease/outcome presence (diagnosis) or occurrence (prognosis) in an individual. Unfortunately, there are many examples of prediction models that show optimistic accuracy in the data from which they were developed. They show substantially lower accuracy when validated in new populations, compromising patient management and outcome. It is widely agreed that each prediction model is validated before application in practice. In case of poor accuracy in the validation data, investigators often proceed to develop their "own" prediction model. This urge to develop a new model from each data set at hand is an unfortunate habit; it makes prediction research particularistic and prior knowledge is not optimally used. Moreover, validation studies are often smaller than development studies, such that the accuracy of the new model (from the validation set) in future patients can actually be worse than applying the original model. Instead, meta-analysis-like methods may be considered to synthesize available evidence from previously published models and derive a more up-to-date and better generalisable model. We propose that one should start from a (well developed) multivariable model - obtained from either a single study or from a meta-analytical approach of several studies - and update this model with the validation data, to enhance its accuracy in future patients.



Vergouwe Y, Nieboer D, Oostenbrink R, Debray TPA, Murray G, Kattan M, Koffijberg H, Moons KGM, Steyerberg EW. A closed testing procedure to select an appropriate method for updating prediction models. Stat Med 2016:36;4529-39.

Riley RD, Ensor J, Snell KI, Debray TP, Altman DG, Moons KG, Collins GS. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ 2016:353;i3140.

Damen JA, Hooft L, Schuit E, Debray TP, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 2016:353;i2416.

Debray TP, Riley RD, Rovers MM, Reitsma JB, Moons KG, on behalf of the Cochrane IPD Meta-analysis Methods group. Individual participant data (IPD) meta-analyses of diagnostic and prognostic modeling studies: guidance on their use. PLoS Med 2015:12;e1001886.

Snell KI, Hua H, Debray TP, Ensor J, Look MP, Moons KG, Riley RD. Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model. J Clin Epidemiol 2016:69;40-50.

Debray TP, Jolani S, Koffijberg H, van Buuren S, Moons KG. Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE. Stat Med 2015:34;1841-63.

Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KG. A new framework to enhance the interpretation of external validation studies of clinical prediction models. J Clin Epidemiol 2015:68;279-89.

Debray TP, Koffijberg H, Nieboer D, Vergouwe Y, Steyerberg EW, Moons KG. Meta-analysis and aggregation of multiple published prediction models. Stat Med 2014:33;2341-62.

Ahmed I, Debray TP, Moons KG, Riley RD. Developing and validating risk prediction models in an individual participant data meta-analysis. BMC Med Res Methodol 2014:14;3.

Debray TP, Moons KG, Abo-Zaid GM, Koffijberg H, Riley RD. Individual participant data meta-analysis for a binary outcome: one-stage or two-stage?. PLoS One 2013:8;e60650.

Debray TP, Moons KG, Ahmed I, Koffijberg H, Riley RD. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat Med 2013:32;3158-80.

Debray TP, Koffijberg H, Lu D, Vergouwe Y, Steyerberg EW, Moons KG. Incorporating published univariable associations in diagnostic and prognostic modeling. BMC Med Res Methodol 2012:12;121.

Debray TPA, Koffijberg H, Vergouwe Y, Moons KGM, Steyerberg EW. Aggregating published prediction models with individual patient data: a comparison of different approaches. Stat Med 2012:31;2697-712.

The Netherlands Organisation for Health Research and Development

Source of Funding

The Netherlands Organisation for Scientific Research supports a strong system of sciences in the Netherlands by encouraging quality and innovation in science. Our conviction is that scientific research contributes to our prosperity and well-being and that it provides for our growing need for knowledge: for facing societal challenges, for economic development and to better understand ourselves and the world.

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley, Lorem Ipsum is simply dummy text of the printing and typesetting industry.