## OptHoldoutSize: an R package for estimating the optimal holdout set size for a predictive risk score to be deployed in a population.

This R package implements procedures for estimating an ‘optimal holdout size’ for a predictive score in order for it to be safely updated. Procedures are detailed in the manuscript ‘Optimal sizing of a holdout set for safe predictive model updating’ by Sami Haidar-Wehbe, Samuel R. Emerson, Louis J. M. Aslett, and James Liley.

When a predictive risk score for binary outcome $$Y$$ given covariates $$X$$ is deployed in a population, it may be used to guide interventions so as to avoid $$Y$$. This makes it difficult to update the predictive score safely, since $$X$$ can influence incidence of $$Y$$ in two ways: through the system being modelled, or through the predictive score itself.

A simple way to safely update a predictive is to with-hold calculation of the risk score for a proportion of the population maintained as a ‘holdout’ set. The predictive score can then be updated using data $$X$$, $$Y$$ from this holdout set. A question naturally arises over how large this hold-out set should be: too small, and a new predictive score cannot be trained sufficiently accurately; too large, and too many members of the population miss out on potential benefits of the risk score.

install.packages("OptHoldoutSize")
library(OptHoldoutSize)
For examples demonstrating use of this package, see vignettes simulated_example and ASPRE_example. For a comparison of the two major algorithms implemented in this package, see vignette comparison_of_algorithms.