Journal of Evaluation in Clinical Practice

Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta‐analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false‐positive and false‐negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta‐analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.

The objective of this study was to compare the classification of hospitals as outcomes outliers using a commonly implemented frequentist statistical approach vs. an implementation of Bayesian hierarchical statistical models, using 30‐day hospital‐level mortality rates for a cohort of acute myocardial infarction patients as a test case. For the frequentist approach, a logistic regression model was constructed to predict mortality. For each hospital, a risk‐adjusted mortality rate was computed. Those hospitals whose 95% confidence interval, around the risk‐adjusted mortality rate, excludes the mean mortality rate were classified as outliers. With the Bayesian hierarchical models, three factors could vary: the profile of the typical patient (low, medium or high risk), the extent to which the mortality rate for the typical patient departed from average, and the probability that the mortality rate was indeed different by the specified amount. The agreement between the two methods was compared for different patient profiles, threshold differences from the average and probabilities. Only marginal agreement was shown between the Bayesian and frequentist approaches. In only five of the 27 comparisons was the kappa statistic at least 0.40. The remaining 22 comparisons demonstrated only marginal agreement between the two methods. Within the Bayesian framework, hospital classification clearly depended on patient profile, threshold and probability of exceeding the threshold. These inconsistencies raise questions about the validity of current methods for classifying hospital performance, and suggest a need for urgent research into which methods are most meaningful to clinicians, managers and the general public.

Rationale, aims and objectives  Self‐care agency is a fundamental concept in nursing and health care research. Having self‐care agency enhances an individual's health‐promoting behaviours and/or specific capabilities for chronic disease self‐management. The purpose of this study was to continue the development and psychometric testing of the Appraisal of Self‐Care Agency – Revised (ASAS‐R).

Methods  A cross‐sectional methodological design was used to examine the reliability, validity and factor structure of the ASAS‐R among individuals from the general population. The sample consisted of 629 adults who were randomly selected to conduct an exploratory factor analysis (EFA; n = 240) and a confirmatory factor analysis (CFA; n = 389) of the scale. A demographic questionnaire, the ASAS‐R and the Health‐Promoting Lifestyle Profile (HPLP‐II) were the measures used to collect the data.

Results  The final 15‐item three‐factor ASAS‐R had an overall Cronbach's alpha (α) of 0.89. The three factors extracted, rotated and scored in this study were labelled: having power for self‐care, developing power for self‐care and lacking power for self‐care. All inter‐items and item‐to‐total correlations met recommended criteria of r = 0.30 to r = 0.70, except for one of the items that had an item‐to‐total correlation of 0.71, slightly exceeding the maximum recommended item‐to‐total correlation. The three factors had Cronbach's alphas of 0.86, 0.83 and 0.79, respectively. The three factors together explained 61.7% of the scale items variance. Each item of the scale had a strong factor loading ranging from 0.52 to 0.81. All measures of model fit exceeded the recommended criteria, indicating that the 15‐item ASAS‐R had a very good fit (χ²/d.f. = 1.97, GFI = 0.94, AGFI = 0.92, CFI = 0.96, TLI = 0.95, RMSEA = 0.05, RMR = 0.05 and the PCLOSE = 0.48).

Conclusions  The 15‐item three‐factor ASAS‐R is a short, reliable and valid instrument to measure self‐care agency among individuals from the general population, but further psychometric evaluation is needed among individuals with chronic diseases, especially those with diabetes mellitus.

The increasing availability of medical evidence in clinical practice was expected to improve the quality of care. However, this has not been realized. A possible explanation is that quality of care is a complex concept and needs a wider scope. Starting from the Donabedian triangle of structure, process and outcome, a framework for the analysis of quality of care is presented. The need for three types of evidence is identified and discussed: medical, contextual and policy evidence. Although the body of medical evidence is increasing, it has major flaws and gaps hampering its applicability in primary care. There is also a need to focus on the context of the medical encounter, which has been shown to influence outcome, but is still not well researched. Finally, evidence on costs, cost utility and equity needs to be considered. Taking these different aspects of evidence into account, an agenda for research in primary care is set. The analytical framework may provide new insights in the quest for improving quality of health care.

Objectives  The aim of this study was to explore patients' beliefs about medicines by administering the German version of the Beliefs about Medicines Questionnaire (BMQ) in a primary care setting among chronically ill patients and to examine its psychometric properties. The BMQ assesses patients' beliefs about their individual prescribed medication as well as their beliefs about medicines in general.

Methods  A cross‐sectional survey of 485 chronically ill patients was performed. The German version of the BMQ was evaluated in terms of internal consistency, validity and scale structure. To assess validity the Medication Adherence Report Scale (MARS‐D) and the Satisfaction with Information about Medicines Scale (SIMS‐D) were applied.

Results  The BMQ showed good internal consistency (Cronbach's α 0.79–0.83). Patients' belief about the specific necessity of their medicines correlated positively with the MARS‐D (ρ = 0.202; P < 0.01). There were significant correlations in the predicted direction between the MARS‐D and all the BMQ subscales with the exception of the General‐Overuse subscale (ρ = −0.06; P = 0.30). Relationship to the SIMS‐D was comparable to the original study. Factor analysis corroborated the scale structure.

Conclusions  The BMQ is a suitable instrument to measure patients' beliefs in medicines in German primary care settings. Most patients in our sample had positive beliefs concerning the necessity of their medication. Their levels of concern were associated with higher non‐adherence.

Patients are interested in receiving accurate diagnostic and prognostic information. Models and reasoning about diagnoses have been extensively investigated from a foundational perspective; however, for all its importance, prognosis has yet to receive a comparable degree of philosophical and methodological attention, and this may be due to the difficulties inherent in accurate prognostics. In the light of these considerations, we discuss a considerable body of critical thinking on the topic of prognostication and its strict relations with diagnostic reasoning, pointing out the distinction between nosographic and pathophysiological types of diagnosis and prognosis, underlying the importance of the explication and explanation processes. We then distinguish between various forms of hypothetical reasoning applied to reach diagnostic and prognostic judgments, comparing them with specific forms of abductive reasoning. The main thesis is that creative abduction regarding clinical hypotheses in diagnostic process is very unlikely to occur, whereas this seems to be often the case for prognostic judgments. The reasons behind this distinction are due to the different types of uncertainty involved in diagnostic and prognostic judgments.

When a randomized controlled trial is not feasible, investigators typically turn to matching techniques as an alternative approach to evaluate the effectiveness of health care interventions. Matching studies are designed to minimize imbalances on measured pre‐intervention characteristics, thereby reducing bias in estimates of treatment effects. Generally, a matching ratio up to 4:1 (control to treatment) elicits the lowest bias. However, when matching techniques are used in prospective studies, investigators try to maximize the number of controls matched to each treated individual to increase the likelihood that a sufficient sample size will remain after attrition. In this paper, we describe a systematic approach to managing the trade‐off between minimizing bias and maximizing matched sample size. Our approach includes the following three steps: (1) run the desired matching algorithm, starting with 1:1 (one control to one treated individual) matching and iterating until the maximum desired number of potential controls per treated subject is reached; (2) for each iteration, test for covariate balance; and (3) generate numeric summaries and graphical plots of the balance statistics across all iterations in order to determine the optimal solution. We demonstrate the implementation of this approach with data from a medical home pilot programme and with a simulation study of populations of 100 000 in which 1000 individuals receive the intervention. We advocate undertaking this methodical approach in matching studies to ensure that the optimal matching solution is identified. Doing so will raise the overall quality of the literature and increase the likelihood of identifying effective interventions.