BMC Medical Research Methodology

We investigated the feasibility of using a machine learning tool’s relevance predictions to expedite title and abstract screening. We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning. For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3–82) hours) and reliability (median (range) proportion missed records, 1 (0–14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2–18) hours and 3 (1–10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0–22)%. Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means.

Mediation analysis methodology underwent many advancements throughout the years, with the most recent and important advancement being the development of causal mediation analysis based on the counterfactual framework. However, a previous review showed that for experimental studies the uptake of causal mediation analysis remains low. The aim of this paper is to review the methodological characteristics of mediation analyses performed in observational epidemiologic studies published between 2015 and 2019 and to provide recommendations for the application of mediation analysis in future studies. We searched the MEDLINE and EMBASE databases for observational epidemiologic studies published between 2015 and 2019 in which mediation analysis was applied as one of the primary analysis methods. Information was extracted on the characteristics of the mediation model and the applied mediation analysis method. We included 174 studies, most of which applied traditional mediation analysis methods (n = 123, 70.7%). Causal mediation analysis was not often used to analyze more complicated mediation models, such as multiple mediator models. Most studies adjusted their analyses for measured confounders, but did not perform sensitivity analyses for unmeasured confounders and did not assess the presence of an exposure-mediator interaction. To ensure a causal interpretation of the effect estimates in the mediation model, we recommend that researchers use causal mediation analysis and assess the plausibility of the causal assumptions. The uptake of causal mediation analysis can be enhanced through tutorial papers that demonstrate the application of causal mediation analysis, and through the development of software packages that facilitate the causal mediation analysis of relatively complicated mediation models.

In randomized clinical trials or observational studies, it is common to collect biomarker values longitudinally on a cohort of individuals. The investigators may be interested in grouping individuals that share similar changes of biomarker values and use these groups for diagnosis or therapeutic purposes. However, most classical model-based classification methods rely mainly on empirical models such as splines or polynomials and do not reflect the physiological processes. A model-based classification method was developed for longitudinal biomarker measurements through a pharmacokinetic model that describes biomarker changes over time. The method is illustrated using data on human Chorionic Gonadotrophic Hormone measurements after curettage of hydatidiform moles. The resulting classification was linked to the evolution toward gestational trophoblastic neoplasia and may be used as a tool for early diagnosis. The diagnostic accuracy of the pharmacokinetic model was more reproducible than the one of a purely mathematical model that did not take into account the biological processes. The use of pharmacokinetic models in model-based classification approaches can lead to clinically useful classifications.

One potential concern with using mailed surveys containing trauma-related content is the possibility of re-traumatizing survivors without a trained mental health professional present. Prior research provides insufficient guidance regarding the prevalence and magnitude of this risk because the psychological harms of trauma-related surveys have typically been estimated using single post-test observations. Post-test observations cannot quantify magnitude of change in participants’ emotional states and may over or under estimate associations between participants’ characteristics (risk factors) and post-survey upset. We conducted two pre- and post-test studies in samples of former applicants for posttraumatic stress disorder disability benefits: 191 males who served during Gulf War I plus 639 male and 921 female Veterans who served sometime between 1955 and 1998. We used two 9-point items from the Self-Assessment Manikins to measure participants’ valence (sadness/happiness) and arousal (tenseness/calmness) before and after they completed mailed surveys asking about trauma-related symptoms or experiences. We examined the following potential predictors for post-survey sadness and tenseness: screening positive for posttraumatic stress disorder, having a serious mental illness, and history of military sexual assault or combat. After the survey, across the groups, 29.3–41.8% were sadder, 45.3–52.2% had no change in valence, and 12.9–22.5% were happier; 31.7–40.2% were tenser, 40.6–48.2% had no change in arousal, and 17.3–24.0% were calmer. The mean increase in sadness or tenseness post-survey was less than one point in all groups (SD’s < 1.7). Cohen’s d ranged from 0.07 to 0.30. Most hypothesized predictors were associated with greater baseline sadness or tenseness, but not necessarily with larger post-survey changes. Women with a history of military sexual assault had the largest net post-survey changes in sadness (mean = 0.7, SD = 1.4) and tenseness (mean = 0.6, SD = 1.6). While a substantial minority of Veterans reported more sadness or tenseness post-survey, the net change in affect was small. Most hypothesized risk factors were actually associated with higher baseline sadness or tenseness scores. When receiving unsolicited, trauma-related surveys by mail, separate protections for Veterans with the risk factors studied here do not seem necessary.

There is sufficient evidence that monetary incentives are effective in increasing survey response rates in the general population as well as with physicians. The objective of this study was to assess the impact of a monetary incentive intended for administrative assistants on the survey response rate of physicians in leadership positions. This was an ancillary study to a national survey of chairs of academic Departments of Medicine in the United States about measuring faculty productivity. We randomized survey participants to receive or not receive a $5 gift card enclosed in the survey package. The cover letter explained that the gift card was intended for the administrative assistants as a “thank you for their time.” We compared the response rates between the 2 study arms using the Chi-square test. Out of 152 participants to whom survey packages were mailed to, a total of 78 responses were received (51 % response rate). The response rates were 59 % in the incentive arm and 46 % in the no incentive arm. The relative effect of the incentive compared to no monetary incentive was borderline statistically significant (relative risk (RR) = 1.36, 95 % confidence interval (CI) 0.99 to 1.87; p = 0.055). Monetary incentives intended for administrative assistants likely increase the response rate of physicians in leadership positions.

Meta-analyses of analgesic medicines for low back pain often rescale measures of pain intensity to use mean difference (MD) instead of standardised mean difference for pooled estimates. Although this improves clinical interpretability, it is not clear whether this method is justified. Our study evaluated the justification for this method. We identified randomised clinical trials of analgesic medicines for adults with low back pain that used two scales with different ranges to measure the same construct of pain intensity. We transformed all data to a 0–100 scale, then compared between-group estimates across pairs of scales with different ranges. Twelve trials were included. Overall, differences in means between pain intensity measures that were rescaled to a common 0–100 scale appeared to be small and randomly distributed. For one study that measured pain intensity on a 0–100 scale and a 0–10 scale; when rescaled to 0–100, the difference in MD between the scales was 0.8 points out of 100. For three studies that measured pain intensity on a 0–10 scale and 0–3 scale; when rescaled to 0–100, the average difference in MD between the scales was 0.2 points out of 100 (range 5.5 points lower to 2.7 points higher). For two studies that measured pain intensity on a 0–100 scale and a 0–3 scale; when rescaled to 0–100, the average difference in MD between the scales was 0.7 points out of 100 (range 6.2 points lower to 12.1 points higher). Finally, for six studies that measured pain intensity on a 0–100 scale and a 0–4 scale; when rescaled to 0–100, the average difference in MD between the scales was 0.7 points (range 5.4 points lower to 8.3 points higher). Rescaling pain intensity measures may be justified in meta-analyses of analgesic medicines for low back pain. Systematic reviewers may consider this method to improve clinical interpretability and enable more data to be included. Open Science Framework (osf.io/8rq7f).

A number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. However, none of these have been evaluated empirically in a series of published systematic reviews. Using published systematic reviews with network meta-analysis (NMA) from a wide range of health-related fields, we evaluated comparatively the most frequently described Bayesian modelling strategies for MOD in terms of log odds ratio (log OR), between-trial variance, inconsistency factor (i.e. difference between direct and indirect estimates for a comparison), surface under the cumulative ranking (SUCRA) and rankings. We extended the Bayesian random-effects NMA model to incorporate the informative missingness odds ratio (IMOR) parameter, and applied the node-splitting approach to investigate inconsistency locally. We considered both pattern-mixture and selection models, different structures for prior distribution of log IMOR, and different scenarios for MOD. To illustrate level of agreement between different strategies and scenarios, we used Bland-Altman plots. Addressing MOD using extreme scenarios and ignoring the uncertainty about the scenarios led to systematically different and more precise log ORs compared to modelling MOD under the missing at random (MAR) assumption. Hierarchical structure of log IMORs led to lower between-trial variance, especially in the case of substantial MOD. Assuming common-within-network or trial-specific log IMORs yielded similar posterior results for all NMA estimates, whereas intervention-specific structure systematically inflated uncertainty around log ORs and SUCRAs. Pattern-mixture model agreed with selection model, particularly under the trial-specific structure; however, selection model systematically reduced precision around log IMORs. Overall, different strategies and scenarios mostly had good agreement in the case of low MOD. Addressing MOD using extreme scenarios and/or ignoring the uncertainty about the scenarios may negatively affect NMA estimates. Modelling MOD via the IMOR parameter can ensure bias-adjusted estimates and offer valuable insights into missingness mechanisms. The researcher should seek an expert opinion in order to decide on the structure of log IMOR that best aligns to the condition and interventions studied and to define a proper prior distribution for log IMOR. Our findings also apply to pairwise meta-analyses.

Epidemiological studies of incidence play an essential role in quantifying disease burden, resource planning, and informing public health policies. A variety of measures for estimating cancer incidence have been used. Appropriate reporting of incidence calculations is essential to enable clear interpretation. This review uses colorectal cancer (CRC) as an exemplar to summarize and describe variation in commonly employed incidence measures and evaluate the quality of reporting incidence methods. We searched four databases for CRC incidence studies published between January 2010 and May 2020. Two independent reviewers screened all titles and abstracts. Eligible studies were population-based cancer registry studies evaluating CRC incidence. We extracted data on study characteristics and author-defined criteria for assessing the quality of reporting incidence. We used descriptive statistics to summarize the information. This review retrieved 165 relevant articles. The age-standardized incidence rate (ASR) (80%) was the most commonly reported incidence measure, and the 2000 U.S. standard population the most commonly used reference population (39%). Slightly more than half (54%) of the studies reported CRC incidence stratified by anatomical site. The quality of reporting incidence methods was suboptimal. Of all included studies: 45 (27%) failed to report the classification system used to define CRC; 63 (38%) did not report CRC codes; and only 20 (12%) documented excluding certain CRC cases from the numerator. Concerning the denominator estimation: 61% of studies failed to state the source of population data; 24 (15%) indicated census years; 10 (6%) reported the method used to estimate yearly population counts; and only 5 (3%) explicitly explained the population size estimation procedure to calculate the overall average incidence rate. Thirty-three (20%) studies reported the confidence interval for incidence, and only 7 (4%) documented methods for dealing with missing data. This review identified variations in incidence calculation and inadequate reporting of methods. We outlined recommendations to optimize incidence estimation and reporting practices. There is a need to establish clear guidelines for incidence reporting to facilitate assessment of the validity and interpretation of reported incidence.

Temperature variability (TV) is associated with increased mortality risks. However, the independent impacts of interday and intraday are still unknown. We proposed a new method to decompose TV into interday TV and intraday TV through algebra derivation. Intraday TV was defined as the weighted average standard deviation (SD) of minimum temperature and maximum temperature on each day. Interday TV was defined as the weighted SD of daily mean temperatures during the exposure period. We then performed an illustrative analysis using data on daily mortality and temperature in France in 2019–2021. The novel interday and intraday TV indices were good proxies for existing indicators, inlcluding diurnal temperature range (DTR) and temperature change between neighbouring days (TCN). In the illustrative analyses, interday and intraday TVs showed differentiated mortality risks. Mortality burden related to TV was mainly explained by the intraday component, accounting for an attributable fraction (AF) of 1.81% (95% CI: 0.64%, 2.97%) of total mortality, more than twice the AF of interday TV (0.86%, 95% CI: 0.47%, 1.24%). This study proposed a novel method for identifying and isolating the different components of temperature variability and offered a comprehensive way to investigate their health impacts.