Journal of Evaluation in Clinical Practice

Background  Recent estimates concerning the prevalence of autism spectrum disorder (ASD) suggest that at least one in 200 children is affected. This group of children and families have important service needs. The involvement of parents in implementing intervention strategies designed to help their autistic children has long been accepted as helpful. The potential benefits are increased skills and reduced stress for parents as well as children.

Methods  This research review focused on interventions for children aged 1–6 years, and was carried out using systematic methodology: a comprehensive search of psychological, educational and biomedical databases, as well as bibliographies and reference lists of key articles, contact with experts in the field, and hand search of key journals. Only studies which involved a concurrent element of control were included.

Results  The review found very few studies that had adequate research design from which to draw conclusions about the effectiveness of parent‐implemented early intervention. Both randomized and controlled studies tended to suggest that parent training leads to improved child communicative behaviour, increased maternal knowledge of autism, enhanced maternal communication style and parent child interaction, and reduced maternal depression.

Conclusion  It seems that parent training can successfully contribute to intervention for young children with ASD. However, the review highlights the need for improved research in this area.

Statistical tests of heterogeneity and bias, in particular publication bias, are very popular in meta‐analyses. These tests use statistical approaches whose limitations are often not recognized. Moreover, it is often implied with inappropriate confidence that these tests can provide reliable answers to questions that in essence are not of statistical nature. Statistical heterogeneity is only a correlate of clinical and pragmatic heterogeneity and the correlation may sometimes be weak. Similarly, statistical signals may hint to bias, but seen in isolation they cannot fully prove or disprove bias in general, let alone specific causes of bias, such as publication bias in particular. Both false‐positive and false‐negative signals of heterogeneity and bias can be common and their prevalence may be anticipated based on some rational considerations. Here I discuss the major common challenges and flaws that emerge in using and interpreting statistical tests of heterogeneity and bias in meta‐analyses. I discuss misinterpretations that can occur at the level of statistical inference, clinical/pragmatic inference and specific cause attribution. Suggestions are made on how to avoid these flaws, use these tests properly and learn from them.

Evidence from various sources indicates that a substantial number of hospitalized patients suffer treatment‐caused injuries. Most of these injuries result from errors. Yet physicians and other health professionals have been reluctant to admit and address the problem of errors, both because of feelings of guilt and from the desire to avoid punishment or disapproval by colleagues. Research in cognitive psychology and human factors has shown that most errors result from defects in the systems in which we work. These are failures in the design of processes, tasks, training, and conditions of work that make errors more likely. Meaningful reduction of errors requires correction of these systems failures. Barriers to reduction of errors include the complexity of health care systems, difficulties in information access, tolerance of stylistic practices, and fear of punishment that inhibits reporting. Most institutions also lack effective methods for detecting and quantifying errors. Significant improvements in error reduction will require major commitments by organizational leadership and the recognition that errors are evidence of deficiencies in systems, not deficiencies in people.

When a randomized controlled trial is not feasible, investigators typically turn to matching techniques as an alternative approach to evaluate the effectiveness of health care interventions. Matching studies are designed to minimize imbalances on measured pre‐intervention characteristics, thereby reducing bias in estimates of treatment effects. Generally, a matching ratio up to 4:1 (control to treatment) elicits the lowest bias. However, when matching techniques are used in prospective studies, investigators try to maximize the number of controls matched to each treated individual to increase the likelihood that a sufficient sample size will remain after attrition. In this paper, we describe a systematic approach to managing the trade‐off between minimizing bias and maximizing matched sample size. Our approach includes the following three steps: (1) run the desired matching algorithm, starting with 1:1 (one control to one treated individual) matching and iterating until the maximum desired number of potential controls per treated subject is reached; (2) for each iteration, test for covariate balance; and (3) generate numeric summaries and graphical plots of the balance statistics across all iterations in order to determine the optimal solution. We demonstrate the implementation of this approach with data from a medical home pilot programme and with a simulation study of populations of 100 000 in which 1000 individuals receive the intervention. We advocate undertaking this methodical approach in matching studies to ensure that the optimal matching solution is identified. Doing so will raise the overall quality of the literature and increase the likelihood of identifying effective interventions.

The objectives of this study were to assess the impact of major joint replacements in reducing pain and disability and to describe the burden of pain and disability that could be avoided by ordering the queues with respect to severity of disease. A secondary goal was to compare the uses of a general health status measure, the Short Form Health Survey (SF‐36), and a disease‐specific measure, the Western Ontario McMaster Osteoarthritis Index (WOMAC), for accomplishing the objectives.

The results are based on interviews with 209 patients before and after they had surgery. Only 15.9% of the patients had surgery within 3 months' waiting time, 19.2% waited 4–6 months, 30.7% waited 7–9 months, and the remaining 34.1 % waited a year or more. The waiting times were unrelated to the seventy of pain or disability reported in the initial interview.

Following surgery, there were large reductions in the WOMAC scores for pain, stiffness and difficulty in functioning. The SF‐36 showed substantial improvements in relief from pain and in physical functioning, and reductions in role limitation due to physical problems, but not for scores related to mental health. The WOMAC scores were more responsive to the benefits of surgery than the SF‐36 scores.

Queuing systems keyed on burden of symptoms could reduce the burden of pain and disability suffered by patients awaiting surgery. The improvements from hip and knee replacements suggest that equitable access for these procedures should be a priority in Ontario.

Rationale, aims and objectives  Self‐care agency is a fundamental concept in nursing and health care research. Having self‐care agency enhances an individual's health‐promoting behaviours and/or specific capabilities for chronic disease self‐management. The purpose of this study was to continue the development and psychometric testing of the Appraisal of Self‐Care Agency – Revised (ASAS‐R).

Methods  A cross‐sectional methodological design was used to examine the reliability, validity and factor structure of the ASAS‐R among individuals from the general population. The sample consisted of 629 adults who were randomly selected to conduct an exploratory factor analysis (EFA; n = 240) and a confirmatory factor analysis (CFA; n = 389) of the scale. A demographic questionnaire, the ASAS‐R and the Health‐Promoting Lifestyle Profile (HPLP‐II) were the measures used to collect the data.

Results  The final 15‐item three‐factor ASAS‐R had an overall Cronbach's alpha (α) of 0.89. The three factors extracted, rotated and scored in this study were labelled: having power for self‐care, developing power for self‐care and lacking power for self‐care. All inter‐items and item‐to‐total correlations met recommended criteria of r = 0.30 to r = 0.70, except for one of the items that had an item‐to‐total correlation of 0.71, slightly exceeding the maximum recommended item‐to‐total correlation. The three factors had Cronbach's alphas of 0.86, 0.83 and 0.79, respectively. The three factors together explained 61.7% of the scale items variance. Each item of the scale had a strong factor loading ranging from 0.52 to 0.81. All measures of model fit exceeded the recommended criteria, indicating that the 15‐item ASAS‐R had a very good fit (χ²/d.f. = 1.97, GFI = 0.94, AGFI = 0.92, CFI = 0.96, TLI = 0.95, RMSEA = 0.05, RMR = 0.05 and the PCLOSE = 0.48).

Conclusions  The 15‐item three‐factor ASAS‐R is a short, reliable and valid instrument to measure self‐care agency among individuals from the general population, but further psychometric evaluation is needed among individuals with chronic diseases, especially those with diabetes mellitus.

Objectives  The aim of this study was to explore patients' beliefs about medicines by administering the German version of the Beliefs about Medicines Questionnaire (BMQ) in a primary care setting among chronically ill patients and to examine its psychometric properties. The BMQ assesses patients' beliefs about their individual prescribed medication as well as their beliefs about medicines in general.

Methods  A cross‐sectional survey of 485 chronically ill patients was performed. The German version of the BMQ was evaluated in terms of internal consistency, validity and scale structure. To assess validity the Medication Adherence Report Scale (MARS‐D) and the Satisfaction with Information about Medicines Scale (SIMS‐D) were applied.

Results  The BMQ showed good internal consistency (Cronbach's α 0.79–0.83). Patients' belief about the specific necessity of their medicines correlated positively with the MARS‐D (ρ = 0.202; P < 0.01). There were significant correlations in the predicted direction between the MARS‐D and all the BMQ subscales with the exception of the General‐Overuse subscale (ρ = −0.06; P = 0.30). Relationship to the SIMS‐D was comparable to the original study. Factor analysis corroborated the scale structure.

Conclusions  The BMQ is a suitable instrument to measure patients' beliefs in medicines in German primary care settings. Most patients in our sample had positive beliefs concerning the necessity of their medication. Their levels of concern were associated with higher non‐adherence.

Rationale and objectives  Age‐related effects on health service utilization are not well understood. Most previous studies have examined only a single specific health care service or disease condition or have focused exclusively on economic variables. We aim to measure age‐related change in health care utilization among the elderly.

Methods  A population‐based retrospective cohort study was conducted using linked data from four administrative databases (OHIP, ODB, CIHI and RPDB). All Ontario residents over the age of 65 years and eligible for public health coverage were included in the analysis (approximately 1.6 million residents). Main outcome measures include utilization indicators for family physician visits, specialist physician visits, Emergency Department visits, drugs, lab claims, X‐rays, inpatient admissions, CT scans and MRI scans.

Results  The mean number of utilization events for Ontarians aged 65+ years for the 1‐year study period was 70 events (women = 76, men = 63). The overall absolute difference between the 65–69 age group and the 85+ age group was 155% (women = 162%, men = 130%), or 76 more events per person in the older group (women = 82, men = 61). Women averaged more events per person than men, as well as greater percentage differences by age. Drugs and diagnostics account for the majority of events. Only MRI and specialist visits were not higher among the older age groups.

Conclusions  At the population level, overall health care utilization would appear to increase significantly with age. It is unclear whether increasing health care utilization prevents morbidity, decreases mortality, or improves quality of life.