BMC Medical Research Methodology

Despite growing support for mixed methods approaches we still have little systematic knowledge about the consequences of combining surveys and focus groups. While the methodological aspects of questionnaire surveys have been researched extensively, the characteristics of focus group methodology are understudied. We suggest and discuss whether the focus group setting, as compared to questionnaire surveys, encourages participants to exaggerate views in a negative direction. Based on an example from our own research, where we conducted a survey as a follow up of a focus group study, and with reference to theoretical approaches and empirical evidence from the literature concerning survey respondent behaviour and small group dynamics, we discuss the possibility that a discrepancy in findings between the focus groups and the questionnaire reflects characteristics of the two different research methods. In contrast to the survey, the focus group study indicated that doctors were generally negative to clinical guidelines. We were not convinced that this difference in results was due to methodological flaws in either of the studies, and discuss instead how this difference may have been the result of a general methodological phenomenon. Based on studies of how survey questionnaires influence responses, it appears reasonable to claim that surveys are more likely to find exaggerated positive views. Conversely, there are some indications in the literature that focus groups may result in complaints and overly negative attitudes, but this is still an open question. We suggest that while problematic issues tend to be under-communicated in questionnaire surveys, they may be overstated in focus groups. We argue for the importance of increasing our understanding of focus group methodology, for example by reporting interesting discrepancies in mixed methods studies. In addition, more experimental research on focus groups should be conducted to advance the methodology and to test our hypothesis.

In the presence of dependent censoring even after stratification of baseline covariates, the Kaplan–Meier estimator provides an inconsistent estimate of risk. To account for dependent censoring, time-varying covariates can be used along with two statistical methods: the inverse probability of censoring weighted (IPCW) Kaplan–Meier estimator and the parametric g-formula estimator. The consistency of the IPCW Kaplan–Meier estimator depends on the correctness of the model specification of censoring hazard, whereas that of the parametric g-formula estimator depends on the correctness of the models for event hazard and time-varying covariates. We combined the IPCW Kaplan–Meier estimator and the parametric g-formula estimator into a doubly robust estimator that can adjust for dependent censoring. The estimator is theoretically more robust to model misspecification than the IPCW Kaplan–Meier estimator and the parametric g-formula estimator. We conducted simulation studies with a time-varying covariate that affected both time-to-event and censoring under correct and incorrect models for censoring, event, and time-varying covariates. We applied our proposed estimator to a large clinical trial data with censoring before the end of follow-up. Simulation studies demonstrated that our proposed estimator is doubly robust, namely it is consistent if either the model for the IPCW Kaplan–Meier estimator or the models for the parametric g-formula estimator, but not necessarily both, is correctly specified. Simulation studies and data application demonstrated that our estimator can be more efficient than the IPCW Kaplan–Meier estimator. The proposed estimator is useful for estimation of risk if censoring is affected by time-varying risk factors.

Following publication of the first worked example of the “best fit” method of evidence synthesis for the systematic review of qualitative evidence in this journal, the originators of the method identified a need to specify more fully some aspects of this particular derivative of framework synthesis. We therefore present a second such worked example in which all techniques are defined and explained, and their appropriateness is assessed. Specified features of the method include the development of new techniques to identify theories in a systematic manner; the creation of an a priori framework for the synthesis; and the “testing” of the synthesis. An innovative combination of existing methods of quality assessment, analysis and synthesis is used to complete the process. This second worked example was a qualitative evidence synthesis of employees’ views of workplace smoking cessation interventions, in which the “best fit” method was found to be practical and fit for purpose. The method is suited to producing context-specific conceptual models for describing or explaining the decision-making and health behaviours of patients and other groups. It offers a pragmatic means of conducting rapid qualitative evidence synthesis and generating programme theories relating to intervention effectiveness, which might be of relevance both to researchers and policy-makers.

Efforts to improve informed consent have led to calls for providing information a reasonable person would want to have, in a way that facilitates understanding of the reasons why one might or might not want to participate. At the same time, advances in large-scale genomic research have expanded both the opportunities and the risks for participants, families, and communities. To advance the use of effective consent materials that reflect this landscape, we used empirical data to develop model consent language, as well as brief questions to assist people in thinking about their own values relative to participation. We conducted in-person interviews to gather preliminary input on these materials from a diverse sample (n = 32) of the general population in Nashville, Tennessee. We asked them to highlight information they found especially reassuring or concerning, their hypothetical willingness to participate, and their opinions about the values questions. Consent information most often highlighted as reassuring included the purpose of the biobank, the existence and composition of a multidisciplinary oversight committee, the importance of participants’ privacy and efforts to protect it, and controlled access to a scientific database. Information most often highlighted as concerning included the deposition of data in a publicly accessible database, the risk of unintended access to data, the potential for non-research use of data, and use of medical record information in general. Seventy-five percent of participants indicated initial willingness to participate in the hypothetical biobank; this decreased to 66% as participants more closely considered the information over the course of the interview. A large majority rated the values questions as helpful. These results are consistent with other research on public perspectives on biobanking and genomic cohort studies, suggesting that our model language effectively captures commonly expressed reasons for and against participation. Our study enriches this literature by connecting specific consent form disclosures with qualitative data regarding what participants found especially reassuring or concerning and why. Interventions that facilitate individuals’ closer engagement with consent information may result in participation decisions more closely aligned with their values.

The use of preference-based measures in the evaluation of health outcomes has extended considerably over the last decade. Their alleged advantage over other types of general instruments in the evaluation of health related quality of life (HRQOL), supposedly lies in the fact that preference measures incorporate values or utilities that reflects the value of social preferences through health states. The objective of this study was to determine whether the use of social preference weights or utilities makes any real difference when calculating scores for the Euroqol (EQ5-D) questionnaire, a HRQOL preference-based measure. Responses to the EQ5-D of a sample of 10,972 patients from 10 countries enrolled in an observational study of the treatment of schizophrenia in Europe were used for this purpose. Two different methods of scoring the EQ-5D where compared: 'weighting the items' of the questionnaire through the UK official weight coefficients, and 'non-weighting the items'. Pearson's, Spearman's, and two-way mixed parametric intraclass correlation coefficients were used to estimate the association of the scores obtained in both ways. The association between weighted and unweighted Euroqol scores was extremely high (Pearson's r = 0.91), as was the association between their ranks (Spearman's ρ = 0.93). The intraclass correlation coefficient obtained (0.89) also suggested that the concordance between the score distributions was prominent. A non-weighted approach to score the EQ5-D is enough to explain a high proportion of variance in scores obtained through the use of utilities. The differential contribution of weights based on population preference values is therefore minimal and, in our opinion, negligible.

A good response rate has been considered as a proof of a study’s quality. Decreasing participation and its potential impact on the internal validity of the study are of growing interest. Our objective was to assess factors associated with contact and response to a postal survey in a epidemiological study of the long-term outcome of IVF couples. The DAIFI study is a retrospective cohort including 6,507 couples who began an IVF program in 2000-2002 in one of the eight participating French IVF centers. Medical data on all 6,507 couples were obtained from IVF center databases, and information on long-term outcome was available only for participants in the postal survey (n = 2,321). Logistic regressions were used to assess firstly factors associated with contact and secondly factors associated with response to the postal questionnaire among contacted couples. Sixty-two percent of the 6,507 couples were contacted and 58% of these responded to the postal questionnaire. Contacted couples were more likely to have had a child during IVF treatment than non-contactable couples, and the same was true of respondents compared with non-respondents. Demographic and medical characteristics were both associated with probability of contact and probability of response. After adjustment, having a live birth during IVF treatment remained associated with both probabilities, and more strongly with probability of response. Having a child during IVF treatment was a major factor impacting on participation rate. Non-response as well as non-contact were linked to the outcome of interest, i.e. long-term parenthood success of infertile couples. Our study illustrates that an a priori hypothesis may be too simplistic and may underestimate potential bias. In the context of growing use of analytical methods that take attrition into account (such as multiple imputation), we need to better understand the mechanisms that underlie attrition in order to choose the most appropriate method.

Increasing attention is being given to assessing treatment effect heterogeneity among individuals belonging to qualitatively different latent subgroups. Inference routinely proceeds by first partitioning the individuals into subgroups, then estimating the subgroup-specific average treatment effects. However, because the subgroups are only latently associated with the observed variables, the actual individual subgroup memberships are rarely known with certainty in practice and thus have to be imputed. Ignoring the uncertainty in the imputed memberships precludes misclassification errors, potentially leading to biased results and incorrect conclusions. We propose a strategy for assessing the sensitivity of inference to classification uncertainty when using such classify-analyze approaches for subgroup effect analyses. We exploit each individual’s typically nonzero predictive or posterior subgroup membership probabilities to gauge the stability of the resultant subgroup-specific average causal effects estimates over different, carefully selected subsets of the individuals. Because the membership probabilities are subject to sampling variability, we propose Monte Carlo confidence intervals that explicitly acknowledge the imprecision in the estimated subgroup memberships via perturbations using a parametric bootstrap. The proposal is widely applicable and avoids stringent causal or structural assumptions that existing bias-adjustment or bias-correction methods rely on. Using two different publicly available real-world datasets, we illustrate how the proposed strategy supplements existing latent subgroup effect analyses to shed light on the potential impact of classification uncertainty on inference. First, individuals are partitioned into latent subgroups based on their medical and health history. Then within each fixed latent subgroup, the average treatment effect is assessed using an augmented inverse propensity score weighted estimator. Finally, utilizing the proposed sensitivity analysis reveals different subgroup-specific effects that are mostly insensitive to potential misclassification. Our proposed sensitivity analysis is straightforward to implement, provides both graphical and numerical summaries, and readily permits assessing the sensitivity of any machine learning-based causal effect estimator to classification uncertainty. We recommend making such sensitivity analyses more routine in latent subgroup effect analyses.

Internet has been broadly employed as a facilitator for epidemiological surveys, as a way to provide a more economical and practical alternative to traditional survey modes. A current trend in survey research is to combine Web-based surveys with other survey modes by offering the participant the possibility of choosing his/her preferred response method (i.e. mixed-mode approach). However, studies have also demonstrated that the use of different survey modes may produce different responses to the same questions, posing potential challenges on the use of mixed-mode approaches. In this paper, we have implemented a statistical comparison between mixed-mode survey responses collected via mail (i.e. paper) and Web methods obtained from a cross-sectional study in non-urban areas of Denmark. Responses provided by mail and Web participants were compared in terms of: 1) the impact of reminder letters in increasing response rates; 2) differences in socio-demographic characteristics between response groups; 3) changes on the likelihood of reporting health symptoms and negative attitudes towards environmental stressors. Comparisons were mainly performed by two sample t-test, Pearson’s Chi-squared test and multinomial logistic regression models. Among 3104 contacted households, 1066 residents decided to participate on the study. Out of those, 971 selected to respond via mail, whereas 275 preferred the Web method. The majority of socio-demographic characteristics between these two groups of respondents were shown to be statistically different. The use of mailed surveys increased the likelihood of reporting health symptoms and negative attitudes towards environmental stressors, even after controlling for demographic characteristics. Furthermore, the use of reminder letters had a higher positive impact in increasing responses of Web surveys when compared to mail surveys. Our main findings suggest that the use of mail and Web surveys may produce different responses to the same questions posed to participants, but, at the same time, may reach different groups of respondents, given that the overall characteristics of both groups considerably differ. Therefore, the tradeoff between using mixed-mode survey as a way to increase response rate and obtaining undesirable measurement changes may be attentively considered in future survey studies.

Research on risk factors for neuropsychiatric adverse events (NAEs) in smoking cessation with pharmacotherapy is scarce. We aimed to identify predictors and develop a prediction model for risk of NAEs in smoking cessation with medications using Bayesian regularization. Bayesian regularization was implemented by applying two shrinkage priors, Horseshoe and Laplace, to generalized linear mixed models on data from 1203 patients treated with nicotine patch, varenicline or placebo. Two predictor models were considered to separate summary scores and item scores in the psychosocial instruments. The summary score model had 19 predictors or 26 dummy variables and the item score model 51 predictors or 58 dummy variables. A total of 18 models were investigated. An item score model with Horseshoe prior and 7 degrees of freedom was selected as the final model upon model comparison and assessment. At baseline, smokers reporting more abnormal dreams or nightmares had 16% greater odds of experiencing NAEs during treatment (regularized odds ratio (rOR) = 1.16, 95% credible interval (CrI) = 0.95 – 1.56, posterior probability P(rOR > 1) = 0.90) while those with more severe sleep problems had 9% greater odds (rOR = 1.09, 95% CrI = 0.95 – 1.37, P(rOR > 1) = 0.85). The prouder a person felt one week before baseline resulted in 13% smaller odds of having NAEs (rOR = 0.87, 95% CrI = 0.71 – 1.02, P(rOR < 1) = 0.94). Odds of NAEs were comparable across treatment groups. The final model did not perform well in the test set. Worse sleep-related symptoms reported at baseline resulted in 85%—90% probability of being more likely to experience NAEs during smoking cessation with pharmacotherapy. Treatment for sleep disturbance should be incorporated in smoking cessation program for smokers with sleep disturbance at baseline. Bayesian regularization with Horseshoe prior permits including more predictors in a regression model when there is a low number of events per variable.