BMC Medical Research Methodology

The external validity of the randomized controlled trial (RCT) refers to the extent to which the results of the RCT apply to the relevant, non-trial population and is impacted by its eligibility criteria, its organization, and its delivery of the intervention. Here, we compared the outcomes of mortality and hospitalization between an RCT and a cohort study that concurrently enrolled HIV-exposed uninfected (HEU) newborns in Botswana. The Mpepu Study (the RCT) was a clinical trial which determined that co-trimoxazole (CTX) provided no survival benefit for HEUs, allowing both arms of the RCT to be used. The Maikaelelo study (the cohort study) was a prospective observational study that enrolled HEU newborns with telephone follow-up and no in-person visits. Rates of death and hospitalization in the pooled population, were modeled using cox-proportional hazards models for time to death or time to first hospitalization, with study setting (RCT vs. cohort study) as an independent variable. The causal effect of study setting on morbidity and mortality was obtained through a treatment effects approach. In total, 4,010 infants were included; 1,306 were enrolled into the cohort study and 2,704 were enrolled into the RCT. No significant differences in mortality were observed between the two study settings (HR: 1.28, 95% CI: 0.76, 2.13), but RCT participants had a lower risk of hospitalization (HR: 0.72, 95% CI: 0.58, 0.89) that decreased with age. However, RCT participants had a higher risk of hospitalization within the first six months of life. The causal risk difference in hospitalizations attributable to the RCT setting was -0.03 (95% CI: -0.05, -0.01). Children in an RCT with rigorous application of national standard of care guidelines experienced a significantly lower risk of hospitalization than children participating in a cohort study that did not alter clinical care. Future research is needed to further investigate outcome disparities when real-world results fail to mirror those achieved in a clinical trial. Trial registration The Mpepu Trial was funded by the U.S. National Institutes of Health (No. NCT01229761) and the Maikaelelo Study was funded primarily by the U.S. Centers for Disease Control and Prevention (32AI007433-21).

Fatigue is the most frequent symptom reported by patients with chronic illnesses. As a subjective experience, fatigue is commonly assessed with patient-reported outcome measures (PROMs). Currently, there are more than 40 generic and disease-specific PROMs for assessing fatigue in use today. The interpretation of changes in PROM scores may be enhanced by estimates of the so-called minimal important difference (MID). MIDs are not fixed attributes of PROMs but rather vary in relation to estimation method, clinical and demographic characteristics of the study group, etc. The purpose of this paper is to compile published MIDs for fatigue PROMs, spanning diagnostic/patient groups and estimation methods, and to provide information relevant for appraising their appropriateness for use in specific clinical trials and in monitoring fatigue in defined patient groups in routine clinical practice. A systematic search of three databases (Scopus, CINAHL and Cochrane) for studies published between January 2000 to April 2015 using fatigue and variations of the term MID, e.g. MCID, MIC, etc. Two authors screened search hits and extracted data independently. Data regarding MIDs, anchors used and study designs were compiled in tables. Included studies (n = 41) reported 60 studies or substudies estimating MID for 28 fatigue scales, subscales or single item measures in a variety of diagnostic groups and study designs. All studies used anchor-based methods, 21/60 measures also included distribution-based methods and 17/60 used triangulation of methods. Both similarities and dissimilarities were seen within the MIDs. Magnitudes of published MIDs for fatigue PROMs vary considerably. Information about the derivation of fatigue MIDs is needed to evaluate their applicability and suitability for use in clinical practice and research.

Questionnaires are valuable data collection instruments in public health research, and can serve to pre-screen respondents for suitability in future studies. Survey non-response leads to reduced effective sample sizes and can decrease representativeness of the study population, so high response rates are needed to minimize the risk of bias. Here we present results on the success of different postal questionnaire strategies at effecting response, and the effectiveness of these strategies at recruiting participants for a field study on the effects of aircraft noise on sleep. In total, we mailed 17 rounds of 240 questionnaires (total n = 4080) to randomly selected households around Atlanta International Airport. Different mailing rounds were varied in the length of the questionnaire (11, 26 or 55 questions), survey incentive (gift card or $2 cash), number of follow-up waves (0, 2 or 3), incentive for participating in a 5-night in-home sleep study ($100, $150 or $200), and address personalization. We received completed questionnaires from 407 respondents (response rate 11.4%). Personalizing the address, enclosing a $2 cash incentive with the initial questionnaire mailing and repeated follow-up mailings were effective at increasing response rate. Despite the increased expense of these approaches in terms of each household mailed, the higher response rates meant that they were more cost-effective overall for obtaining an equivalent number of responses. Interest in participating in the field study decreased with age, but was unaffected by the mailing strategies or cash incentives for field study participation. The likelihood that a respondent would participate in the field study was unaffected by survey incentive, survey length, number of follow-up waves, field study incentive, age or sex. Pre-issued cash incentives and sending follow-up waves could maximize the representativeness and numbers of people from which to recruit, and may be an effective strategy for improving recruitment into field studies.

Clinician surveys provide critical information about many facets of health care, but are often challenging to implement. Our objective was to assess use by participants and non-participants of a prepaid gift card incentive that could be later reclaimed by the researchers if unused. Clinicians were recruited to participate in a mailed or online survey as part of a study to characterize women’s primary health care provider attitudes towards breast and cervical cancer screening guidelines and practices (n = 177). An up-front incentive of a $50 gift card to a popular online retailer was included with the study invitation. Clinicians were informed that the gift card would expire if it went unused after 4 months. Outcome measures included use of gift cards by participants and non-participants and comparison of hypothetical costs of different incentive strategies. 63.5 % of clinicians who responded to the survey used the gift card, and only one provider who didn’t participate used the gift card (1.6 %). Many of those who participated did not redeem their gift cards (36.5 % of respondents). The price of the incentives actually claimed totaled $3700, which was less than half of the initial outlay. Since some of the respondents did not redeem their gift cards, the cost of incentives was less than it might have been if we had provided a conditional incentive of $50 to responders after they had completed the survey. Redeemable online gift card codes may provide an effective way to motivate clinicians to participate in surveys.

Phase III randomized controlled trials (RCT) typically exclude certain patient subgroups, thereby potentially jeopardizing estimation of a drug’s effects when prescribed to wider populations and under routine care (“effectiveness”). Conversely, enrolling heterogeneous populations in RCTs can increase endpoint variability and compromise detection of a drug’s effect. We developed the “RCT augmentation” method to quantitatively support RCT design in the identification of exclusion criteria to relax to address both of these considerations. In the present manuscript, we describe the method and a case study in schizophrenia. We applied typical RCT exclusion criteria in a real-world dataset (cohort) of schizophrenia patients to define the “RCT population” subgroup, and assessed the impact of re-including each of the following patient subgroups: (1) illness duration 1–3 years; (2) suicide attempt; (3) alcohol abuse; (4) substance abuse; and (5) private practice management. Predictive models were built using data from different “augmented RCT populations” (i.e., subgroups where patients with one or two of such characteristics were re-included) to estimate the absolute effectiveness of the two most prevalent antipsychotics against real-world results from the entire cohort. Concurrently, the impact on RCT results of relaxing exclusion criteria was evaluated by calculating the comparative efficacy of those two antipsychotics in virtual RCTs drawing on different “augmented RCT populations”. Data from the “RCT population”, which was defined with typical exclusion criteria, allowed for a prediction of effectiveness with a bias < 2% and mean squared error (MSE) = 5.8–6.8%. Compared to this typical RCT, RCTs using augmented populations provided improved effectiveness predictions (bias < 2%, MSE = 5.3–6.7%), while returning more variable comparative effects. The impact of augmentation depended on the exclusion criterion relaxed. Furthermore, half of the benefit of relaxing each criterion was gained from re-including the first 10–20% of patients with the corresponding real-world characteristic. Simulating the inclusion of real-world subpopulations into an RCT before running it allows for quantification of the impact of each re-inclusion upon effect detection (statistical power) and generalizability of trial results, thereby explicating this trade-off and enabling a controlled increase in population heterogeneity in the RCT design.

Chained equations imputation is widely used in medical research. It uses a set of conditional models, so is more flexible than joint modelling imputation for the imputation of different types of variables (e.g. binary, ordinal or unordered categorical). However, chained equations imputation does not correspond to drawing from a joint distribution when the conditional models are incompatible. Concurrently with our work, other authors have shown the equivalence of the two imputation methods in finite samples. Taking a different approach, we prove, in finite samples, sufficient conditions for chained equations and joint modelling to yield imputations from the same predictive distribution. Further, we apply this proof in four specific cases and conduct a simulation study which explores the consequences when the conditional models are compatible but the conditions otherwise are not satisfied. We provide an additional “non-informative margins” condition which, together with compatibility, is sufficient. We show that the non-informative margins condition is not satisfied, despite compatible conditional models, in a situation as simple as two continuous variables and one binary variable. Our simulation study demonstrates that as a consequence of this violation order effects can occur; that is, systematic differences depending upon the ordering of the variables in the chained equations algorithm. However, the order effects appear to be small, especially when associations between variables are weak. Since chained equations is typically used in medical research for datasets with different types of variables, researchers must be aware that order effects are likely to be ubiquitous, but our results suggest they may be small enough to be negligible.