The myth of generalisability in clinical research and machine learning in health care

Abstract

Dr Lee, an esteemed intensivist from the USA, is rounding in an intensive care unit (ICU). He is asked by a team member who is taking care of patients with COVID-19 if they can triage their patients to optimise use of scarce resources, such as ventilators, with their hospital’s new machine learning model to predict mortality.1 He is about to say yes, but stops himself. Do the findings of the preprints and fast-tracked published articles that this model is based on apply to his patient population?2 Problems with the increase in hastily written articles notwithstanding, are the conclusions of research based on patients with COVID-19 in China and Italy from several months ago still valid in his ICU today, given the differences in practice patterns and rapidly changing guidelines and protocols?