Implementation Science

Although an evidence-based approach is the ideal model for planning and delivering healthcare, barriers exist to using research evidence to implement and evaluate service change. This paper aims to inform policy implementation and evaluation by understanding the role of research evidence at the local level through implementation of a national chronic conditions management policy. We conducted a national email survey of health service commissioners at the most devolved level of decision-making in Wales (Local Health Boards – LHBs) followed by in-depth interviews with representatives of LHBs, purposively selecting five to reflect geographic and economic characteristics. Survey data were analysed descriptively; we used thematic analysis for interview data. All LHBs (n = 22) completed questionnaires. All reported they routinely assessed the research literature before implementing interventions, but free-text answers revealed wide variation in approach. Most commonly reported information sources included personal contacts, needs assessments, information or research databases. No consistent approach to evaluation was reported. Frequently reported challenges were: insufficient staff capacity (17/22); limited skills, cost, limited time, competing priorities (16/22); availability and quality of routine data (15/22). Respondents reported they would value central guidance on evaluation. Five interviews were held with managers from the five LHBs contacted. Service delivery decisions were informed by Welsh Government initiatives and priorities, budgets, perceived good practice, personal knowledge, and local needs, but did not include formal research evidence, they reported. Decision making was a collaborative process including clinical staff, patient representatives, and partner organization managers with varying levels of research experience. Robust evaluation data were required, but they were constrained by a lack of skills, time, and resources. They viewed evaluation as a means of demonstrating that targets had been met. There is a gap between evidence-based aims of national health policy and how health services are commissioned, implemented, and evaluated at local level. Commissioners and managers are unable to routinely incorporate research evidence. If health services research is to identify most effective ways to implement high quality care, it should be incorporated into commissioning and service delivery. Local commissioners and managers need to build the critical use of research evidence and evaluation into health policy implementation at local level in order to provide consistent and effective healthcare services.

The GRADE-CERQual (Confidence in Evidence from Reviews of Qualitative research) approach has been developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) working group. The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. CERQual includes four components for assessing how much confidence to place in findings from reviews of qualitative research (also referred to as qualitative evidence syntheses): (1) methodological limitations; (2) coherence; (3) adequacy of data; and (4) relevance. This paper is part of a series providing guidance on how to apply CERQual and focuses on CERQual’s adequacy of data component. We developed the adequacy of data component by searching the literature for definitions, gathering feedback from relevant research communities and developing consensus through project group meetings. We tested the CERQual adequacy of data component within several qualitative evidence syntheses before agreeing on the current definition and principles for application. When applying CERQual, we define adequacy of data as an overall determination of the degree of richness and the quantity of data supporting a review finding. In this paper, we describe the adequacy component and its rationale and offer guidance on how to assess data adequacy in the context of a review finding as part of the CERQual approach. This guidance outlines the information required to assess data adequacy, the steps that need to be taken to assess data adequacy, and examples of adequacy assessments. This paper provides guidance for review authors and others on undertaking an assessment of adequacy in the context of the CERQual approach. We approach assessments of data adequacy in terms of the richness and quantity of the data supporting each review finding, but do not offer fixed rules regarding what constitutes sufficiently rich data or an adequate quantity of data. Instead, we recommend that this assessment is made in relation to the nature of the finding. We expect the CERQual approach, and its individual components, to develop further as our experiences with the practical implementation of the approach increase.

While feasibility of new health technologies in well-resourced healthcare settings is extensively documented, it is largely unknown in low-resourced settings. Uganda's decision to deploy and scale up malaria rapid diagnostic tests (mRDTs) in public health facilities and at the community level provides a useful entry point for documenting field experience, acceptance, and predictive variables for technology acceptance and use. These findings are important in informing implementation of new health technologies, plans, and budgets in low-resourced national disease control programmes. A cross-sectional qualitative descriptive study at 21 health centres in Uganda was undertaken in 2007 to elucidate the barriers and facilitators in the introduction of mRDTs as a new diagnostic technology at lower-level health facilities. Pre-tested interview questionnaires were administered through pre-structured patient exit interviews and semi-structured health worker interviews to gain an understanding of the response to this implementation. A conceptual framework on technology acceptance and use was adapted for this study and used to prepare the questionnaires. Thematic analysis was used to generate themes from the data. A total of 52 of 57 health workers (92%) reported a belief that a positive mRDT result was true, although only 41 of 57 (64%) believed that treatment with anti-malarials was justified for every positive mRDT case. Of the same health workers, only 49% believed that a negative mRDT result was truly negative. Factors linked to these findings were related to mRDT acceptance and use, including the design and characteristics of the device, availability and quality of mRDT ancillary supplies, health worker capacity to investigate febrile cases testing negative with the device and provide appropriate treatment, availability of effective malaria treatments, reliability of the health commodity supply chain, existing national policy recommendations, individual health worker dynamism, and vitality of supervision. mRDTs were found to be acceptable to and used by the target users, provided clear policy guidelines exist, ancillary tools are easy to use and health supplies beyond the diagnostic tools are met. Based on our results, health workers' needs for comprehensive case management should be met, and specific guidance for managing febrile patients with negative test outcomes should be provided alongside the new health technology. The extent, to which the implementation process of mRDT-led, parasite-based diagnosis accommodates end user beliefs, attitudes, perceptions, and satisfaction, as well as technology learnability and suitability, influences the level of acceptance and use of mRDTs. The effectiveness of the health system in providing the enabling environment and the integration of the diagnostic tool into routine service delivery is critical.

Task-shifting is promoted widely as a mechanism for expanding antiretroviral treatment (ART) access. However, the evidence for nurse-initiated and managed ART (NIMART) in Africa is limited, and little is known about the key barriers and enablers to implementing NIMART programmes on a large scale. The STRETCH (Streamlining Tasks and Roles to Expand Treatment and Care for HIV) programme was a complex educational and organisational intervention implemented in the Free State Province of South Africa to enable nurses providing primary HIV/AIDS care to expand their roles and include aspects of care and treatment usually provided by physicians. STRETCH used a phased implementation approach and ART treatment guidelines tailored specifically to nurses. The effects of STRETCH on pre-ART mortality, ART provision, and the quality of HIV/ART care were evaluated through a randomised controlled trial. This study was conducted alongside the trial to develop a contextualised understanding of factors affecting the implementation of the programme. This study was a qualitative process evaluation using in-depth interviews and focus group discussions with patients, health workers, health managers, and other key informants as well as observation in clinics. Research questions focused on perceptions of STRETCH, changes in health provider roles, attitudes and patient relationships, and impact of the implementation context on trial outcomes. Data were analysed collaboratively by the research team using thematic analysis. NIMART appears to be highly acceptable among nurses, patients, and physicians. Managers and nurses expressed confidence in their ability to deliver ART successfully. This confidence developed slowly and unevenly, through a phased and well-supported approach that guided nurses through training, re-prescription, and initiation. The research also shows that NIMART changes the working and referral relationships between health staff, demands significant training and support, and faces workload and capacity constraints, and logistical and infrastructural challenges. Large-scale NIMART appears to be feasible and acceptable in the primary level public sector health services in South Africa. Successful implementation requires a comprehensive approach with: an incremental and well supported approach to implementation; clinical guidelines tailored to nurses; and significant health services reorganisation to accommodate the knock-on effects of shifts in practice. ISRCTN46836853

The Portuguese National Health Directorate has issued clinical practice guidelines on prescription of anti-inflammatory drugs, acid suppressive therapy, and antiplatelets. However, their effectiveness in changing actual practice is unknown. The study will compare the effectiveness of educational outreach visits regarding the improvement of compliance with clinical guidelines in primary care against usual dissemination strategies. A cost-benefit analysis will also be conducted. We will carry out a parallel, open, superiority, randomized trial directed to primary care physicians. Physicians will be recruited and allocated at a cluster-level (primary care unit) by minimization. Data will be analyzed at the physician level. Primary care units will be eligible if they use electronic prescribing and have at least four physicians willing to participate. Physicians in intervention units will be offered individual educational outreach visits (one for each guideline) at their workplace during a six-month period. Physicians in the control group will be offered a single unrelated group training session. Primary outcomes will be the proportion of cyclooxygenase-2 inhibitors prescribed in the anti-inflammatory class, and the proportion of omeprazole in the proton pump inhibitors class at 18 months post-intervention. Prescription data will be collected from the regional pharmacy claims database. We estimated a sample size of 110 physicians in each group, corresponding to 19 clusters with a mean size of 6 physicians. Outcome collection and data analysis will be blinded to allocation, but due to the nature of the intervention, physicians and detailers cannot be blinded. This trial will attempt to address unresolved issues in the literature, namely, long term persistence of effect, the importance of sequential visits in an outreach program, and cost issues. If successful, this trial may be the cornerstone for deploying large scale educational outreach programs within the Portuguese National Health Service. ClinicalTrials.gov number NCT01984034 .

Theories occupy different positions in the scientific circle of enquiry as they vary in scope, abstraction, and complexity. Mid-range theories play a crucial bridging role between raw empirical observations and all-encompassing grand-theoretical schemes. A shift of perspective from ‘theories’ as products to ‘theorising’ as a process can enable empirical researchers to capitalise on the two-way relationships between empirical data and different levels of theory and contribute to the advancement of knowledge. This can be facilitated by embracing theoretically informative (in addition to merely theoretically informed) research, developing mechanism-based explanations, and broadening the repertoire of grand-theoretical orientations.

Initiatives to raise the quality of care provided to mothers need to be given priority in Sub Saharan Africa (SSA). The promotion of clinical practice guidelines (CPGs) is a common strategy, but their implementation is often challenging, limiting their potential impact. Through a cross-country perspective, this study explored CPGs for maternal health in Burkina Faso, Ghana, and Tanzania. The objectives were to compare factors related to CPG use including their content compared with World Health Organization (WHO) guidelines, their format, and their development processes. Perceptions of their availability and use in practice were also explored. The overall purpose was to further the understanding of how to increase CPGs' potential to improve quality of care for mothers in SSA. The study was a multiple case study design consisting of cross-country comparisons using document review and key informant interviews. A conceptual framework to aid analysis and discussion of results was developed, including selected domains related to guidelines' implementability and use by health workers in practice in terms of usability, applicability, and adaptability. The study revealed few significant differences in content between the national guidelines for maternal health and WHO recommendations. There were, however, marked variations in the format of CPGs between the three countries. Apart from the Ghanaian and one of the Tanzanian CPGs, the levels of both usability and applicability were assessed as low or medium. In all three countries, the use of CPGs by health workers in practice was perceived to be limited. Our cross-country study suggests that it is not poor quality of content or lack of evidence base that constitute the major barrier for CPGs to positively impact on quality improvement in maternal care in SSA. It rather emphasises the need to prioritise the format of guidelines to increase their usability and applicability and to consider these attributes together with implementation strategies as integral to their development processes.

Operational research (OR) is the discipline of using models, either quantitative or qualitative, to aid decision-making in complex implementation problems. The methods of OR have been used in healthcare since the 1950s in diverse areas such as emergency medicine and the interface between acute and community care; hospital performance; scheduling and management of patient home visits; scheduling of patient appointments; and many other complex implementation problems of an operational or logistical nature. To date, there has been limited debate about the role that operational research should take within implementation science. I detail three such roles for OR all grounded in upfront system thinking: structuring implementation problems, prospective evaluation of improvement interventions, and strategic reconfiguration. Case studies from mental health, emergency medicine, and stroke care are used to illustrate each role. I then describe the challenges for applied OR within implementation science at the organisational, interventional, and disciplinary levels. Two key challenges include the difficulty faced in achieving a position of mutual understanding between implementation scientists and research users and a stark lack of evaluation of OR interventions. To address these challenges, I propose a research agenda to evaluate applied OR through the lens of implementation science, the liberation of OR from the specialist research and consultancy environment, and co-design of models with service users. Operational research is a mature discipline that has developed a significant volume of methodology to improve health services. OR offers implementation scientists the opportunity to do more upfront system thinking before committing resources or taking risks. OR has three roles within implementation science: structuring an implementation problem, prospective evaluation of implementation problems, and a tool for strategic reconfiguration of health services. Challenges facing OR as implementation science include limited evidence and evaluation of impact, limited service user involvement, a lack of managerial awareness, effective communication between research users and OR modellers, and availability of healthcare data. To progress the science, a focus is needed in three key areas: evaluation of OR interventions, embedding the knowledge of OR in health services, and educating OR modellers about the aims and benefits of service user involvement.

How humans think and make decisions is important in understanding behaviour. Hence an understanding of cognitive processes among physicians may inform our understanding of behaviour in relation to evidence implementation strategies. A personality theory, Cognitive-Experiential Self Theory (CEST) proposes a relationship between different ways of thinking and behaviour, and articulates pathways for behaviour change. However prior to the empirical testing of interventions based on CEST, it is first necessary to demonstrate its suitability among a sample of healthcare workers. To investigate the relationship between thinking styles and the knowledge and clinical practices of doctors directly involved in the management of acute coronary syndromes. Self-reported doctors' thinking styles (N = 74) were correlated with results from a survey investigating knowledge, attitudes, and clinical practice, and evaluated against recently published acute coronary syndrome clinical guidelines. Guideline-discordant practice was associated with an experiential style of thinking. Conversely, guideline-concordant practice was associated with a higher preference for a rational style of reasoning. Findings support that while guidelines might be necessary to communicate evidence, other strategies may be necessary to target discordant behaviours. Further research designed to examine the relationships found in the current study is required.

Significant resources and time are invested in the production of research knowledge. The primary objective of this randomized controlled trial was to evaluate the effectiveness of three knowledge translation and exchange strategies in the incorporation of research evidence into public health policies and programs. This trial was conducted with a national sample of public health departments in Canada from 2004 to 2006. The three interventions, implemented over one year in 2005, included access to an online registry of research evidence; tailored messaging; and a knowledge broker. The primary outcome assessed the extent to which research evidence was used in a recent program decision, and the secondary outcome measured the change in the sum of evidence-informed healthy body weight promotion policies or programs being delivered at health departments. Mixed-effects models were used to test the hypotheses. One hundred and eight of 141 (77%) health departments participated in this study. No significant effect of the intervention was observed for primary outcome (p < 0.45). However, for public health policies and programs (HPPs), a significant effect of the intervention was observed only for tailored, targeted messages (p < 0.01). The treatment effect was moderated by organizational research culture (e.g., value placed on research evidence in decision making). The results of this study suggest that under certain conditions tailored, targeted messages are more effective than knowledge brokering and access to an online registry of research evidence. Greater emphasis on the identification of organizational factors is needed in order to implement strategies that best meet the needs of individual organizations. The trial registration number and title are as follows: ISRCTN35240937 -- Is a knowledge broker more effective than other strategies in promoting evidence-based physical activity and healthy body weight programming?