JAMES PITTMAN
University of California San Diego, VA San Diego Center of Excellence for Stress and Mental Health

The Veterans Health Administration (VHA) developed a comprehensive mobile screening technology (eScreening) that provides customized and automated self-report health screening via mobile tablet for veterans seen in VHA settings. There is agreement about the value of health technology, but limited knowledge of how best to broadly implement and scale up health technologies. Quality improvement (QI) methods may offer solutions to overcome barriers related to broad scale implementation of technology in health systems. We aimed to develop a process guide for eScreening implementation in VHA clinics to automate self-report screening of mental health symptoms and psychosocial challenges.

Stakeholders within the VHA.

This was a two-phase, mixed methods implementation project building on an adapted quality improvement method. In phase one, we adapted and conducted a Rapid Process Improvement Workshop (RPIW) to develop a generalizable process guide for eScreening implementation (eScreening Playbook). In phase two, we integrated the eScreening Playbook and RPIW with additional strategies of training and facilitation to create a multicomponent implementation strategy (MCIS) for eScreening.
 
We then piloted the MCIS in two VHA sites. Quantitative eScreening pre-implementation survey data and qualitative implementation process “mini interviews” were collected from individuals at each of the two sites who participated in the implementation process. Survey data were characterized using descriptive statistics, and interview data were independently coded using a rapid qualitative analytic approach.

Pilot data showed overall satisfaction and usefulness of our MCIS approach and identified some challenges, solutions, and potential adaptations across sites. Both sites used the components of the MCIS, but site 2 elected not to include the RPIW. Survey data revealed positive responses related to eScreening from staff at both sites. Interview data exposed implementation challenges related to the technology, support, and education at both sites. Workflow and staffing resource challenges were only reported by site 2.

A RPIW can be an important factor in the adoption of health technology, but organizational factors also need to be addressed. Through our experience implementing eScreening, we have found that successful adoption of health technology needs to be flexible and contain multiple components. Overall, our use of RPIW and other QI methods to both develop a playbook and an implementation strategy for eScreening has created a testable implementation process to employ automated, patient-facing assessment. The efficient collection and communication of patient information has the potential to greatly improve access to and quality of healthcare.

BRYAN R. GARNER, PhD
RTI International

To help “speed the translation of research findings into routine practice” Curran and colleagues (2012) codified three types of hybrid trial designs (i.e., Type 1, Type 2, and Type 3). As part of type 2 hybrid trials they proposed “dual testing of clinical and implementation interventions/strategies.” Despite their note about using the term test in a “liberal manner” (i.e., the clinical and implementation interventions/strategies need not all be tested with randomized, strongly powered designs), in 2014 the National Institute on Drug Abuse funded a dual-randomized type 2 implementation-effectiveness hybrid trial called the Substance Abuse Treatment to HIV care (SAT2HIV) Project. Consistent with theme area 1 (Pragmatic Trial Examples), this presentation/poster will provide a concrete example of what Landes, McBain, and Curran (2019) highlighted as a “rarer” type 2 hybrid trial example.

Thirty-nine HIV service organizations, 78 HIV service organization staff, and 824 people with HIV and a comorbid substance use disorder.

A dual-randomized type 2 implementation-effectiveness hybrid trial, which simultaneously included: 1) a 39-site cluster-randomized implementation trial focused on testing the effectiveness of the team-focused Implementation & Sustainment Facilitation (ISF) Strategy as an adjunct to the staff-focused Addiction Technology Transfer Center.
 
(ATTC) Strategy, and 2) a multisite randomized controlled trial testing the effectiveness of a motivational interviewing-based brief intervention for substance use as an adjunct to HIV service organization’s usual care for substance use disorders. Both staff-level outcomes and client-level outcomes were examined.

The ISF Strategy had a significant impact on implementation effectiveness (i.e., the consistency and the quality of implementation; ? = .65, p = .01), but not on time-to-proficiency (? = ?.02), or level-of-sustainment (? = .09). Additionally, the ISF Strategy had a significant impact on intervention effectiveness (i.e., the effectiveness of the MIBI), at least in terms of significantly decreasing the odds (odds ratio = 0.11, p = .02) of clients using their primary substance daily during follow-up.

Although not for the faint of heart, dual-randomized type 2 hybrid trials can be successfully completed with the right infrastructure and team. Building upon the SAT2HIV Project, the SAT2HIV-II Project is a type 3 hybrid trial that was recently funded by the National Institute on Drug Abuse that is focused on testing a pay-for-performance (P4P) strategy as an adjunct to the ATTC+ISF Strategy found to be most effective as part of the original SAT2HIV Project.

ERIN KENZIE, PhD
Oregon Health & Science University

A complex array of factors affect the ability of primary care clinics to successfully integrate evidence-based practices into routine care. Models like i-PARIHS (integrated Promoting Action on Research Implementation in Health Services) identify factors related to the intervention, recipients (motivation, skill), and multiple levels of context including local (workflows, past experience), organization (culture, structure), and external (policy drivers). To effectively support clinics, practice facilitators-individuals trained to build the capacity of primary care practices-must accurately assess clinics’ needs and identify corresponding means of implementation support. Examining how this tailoring happens is key to evaluating program outcomes and maximizing program success.

This research is being conducted as part of the ANTECEDENT study, an AHRQ-funded EvidenceNOW unhealthy alcohol use project led by the Oregon Rural Practice-based Research Network (ORPRN). In ANTECEDENT, ORPRN practice facilitators provide technical assistance and supportive services to primary care clinics to adopt or improve evidence-based methods of addressing unhealthy alcohol use through screening, brief intervention, and medication assisted treatment (MAT). Efforts are aligned with the state’s Medicaid quality incentive metric for SBIRT (screening, brief intervention, and referral to treatment) and in partnership with SBIRT Oregon (www.sbirtoregon.org).

In this mixed methods evaluation, we combine qualitative interviews with causal-loop diagramming, a systems science method for describing complex interrelationships. This poster will outline how we are using causal-loop diagramming to enhance our qualitative analysis and structure our understanding of how practice facilitators respond to clinic needs. We will describe our approach for generating causal-loop diagrams illustrating practice facilitators’ mental models of practice change from qualitative interviews.

Preliminary results from baseline analyses will be presented by illustrating causal-loop diagrams of practice facilitators’ mental models of practice change and tailoring implementation support to context. By analyzing the structure and content of the diagrams, insight can be gained about the range of perspectives held by practice facilitators. Strengths and limitations of this approach to modeling from qualitative data will be identified.

System dynamics, and causal-loop diagramming in particular, is well suited for enhancing qualitative analysis. Our novel approach provides a framework to specify documented or assumed cause-and-effect relationships. This approach can illustrate the mental models of practice facilitators or researchers and help improve evaluation as well as implementation outcomes.