TRD&1: Model Management and Credibility Infrastructure

TECHNOLOGY RESEARCH & DEVELOPMENT 1: Project Summary Computational modeling and simulation continue to be popular in biomedical research. However, doubts remain as to the reliability or trustworthiness of such efforts. We say that a model is credible for an application if its predictions are trusted and useful. Although what constitutes a credible model depends on its intended usage, many current models are not credible. For example, it has been shown that a vast number of published models cannot even be easily reproduced. That is, recreating the results of a published study is either impossible or very difficult to achieve. If a model cannot be reproduced, then its credibility is immediately suspect. Moreover, during the COVID pandemic, there has been much discussion of the credibility and utility of COVID models, with a great deal focused on the way COVID models were built and tested. With more and more models finding their way into the clinic, being used by policymakers, and in pharmaceutical companies, the credibility of biomedical models has become a pressing problem. We therefore feel it is very important that we begin to consider a systematic approach to assessing the credibility of biomedical models that allows non- experts (or even researchers outside their particular domain) a means to gauge the credibility of a given biomedical model. Work over the last decade to improve model reproducibility is a natural first step towards improving model credibility. The Center for Reproducible Biomedical Modeling has advanced the reproducibility of biomedical models by developing the biosimulation portal, a service that provides access to reproducible models across a wide range of biological domains, not just systems biology. However, even though a model may be reproducible, the question still arises as to how credible is the model. The goals of this project include: (1) Develop a model management system (MMS) which will be a repository or simulatable and reusable model parts. Add sophisticated query systems and model decomposition capabilities; (2) Develop a flexible import/export layer that can accommodate many of the ways that modelers build models; (3) Create a series of credibility tools that can be used to assess various credibility metrics of models either held in the MMS or provided by users directly. Technology from TR&D 1 will be used by TR&D 3 as well as our CPs and SPs.