Precision Implementation: Developing and Validating Predictive Models of Information Technology Tool Adoption

NATHALIE HUGUET
Oregon Health & Science University

There is strong evidence that different implementation support strategies (e.g., facilitation, audit and feedback, performance benchmarking) can help clinical practices with adoption and maintenance of evidence-based guidelines. There are, however, costs to both providing and receiving implementation support. While evidence demonstrates the effectiveness of various implementation strategies, relatively little is known about which practices will benefit most from a particular implementation strategy, how much assistance a practice might need, or if practices could improve on their own. New methods are needed to predict which practices may implement targeted changes with less support and which will need more. The objective of this study is to develop and validate predictive models that estimate the likelihood of adoption of an electronic health record (EHR)-related tool.

EHR data from 351 community health centers (CHCs) from the OCHIN Network in which an insurance support information technology (IT) tool was implemented 05/01/2018. The insurance support tool was designed for clinic eligibility specialists to document health insurance assistance provided and assist with HRSA required reporting.

We used LASSO penalized logistic regression to develop and validate models predicting adoption and sustainability of the tool. Predictive performance was assessed using ROC curve (AUC). Adoption is defined as any instance of tool use within the 12-month follow-up period (up to 6/30/2019). Sustainability is defined as at least one tool use in the last four months of the follow-up period. Variables included in the models were geographic variables, number and type of departments/clinics, patient panel, patient panel demographic characteristics, type and number of encounters, payer distribution, provider type, number of encounters with eligibility specialists.

About 42% of CHCs adopted the tool and 25% demonstrated sustained use. Models for adoption (AUC= 0.784; 95%CI: 0.710 – 0.858) and sustainability (AUC=0.829; 95%CI: 0.746 – 0.912) show high classification accuracy. Out of the 25 variables entered in the model, three predicted adoption (years in EHR, total number of visits, and percent of visits that were ambulatory) and one (total number of visits) predicted sustainability.

EHR data can be used to predict EHR tool use. This work is the next step toward advancing the science of ‘precision implementation’ and how to efficiently tailor and deploy implementation support strategies for IT innovations.