Organizational Research Methods

Mixed methods research is becoming an increasingly popular approach in several areas, and it has long been called for as an approach for providing a better understanding of research problems. However, there have been no assessments as to whether such research, which may be timely and expensive, has more impact on the field. The main purpose of this article is to determine whether the use of a mixed methods approach is a predictor of article impact. The analysis is based on articles published in the Strategic Management Journal from 1980 to 2006. The findings show that mixed methods articles tend to receive more citations than monomethod articles do. The average citations received per year and the cumulating sum of citations are both higher for articles reporting studies using mixed methods than for monomethod research designs. Furthermore, a content analysis of the mixed methods articles identified shows that there are different types of studies based on several characteristics (purpose, priority, implementation, and design). All types of mixed methods articles tend to have a higher number of citations than the group of monomethod studies.

Group-level constructs are often derived from individual-level data. This procedure requires a composition model, specifying how the lower level data can be combined to compose the higher level construct. Two common composition methods are direct consensus composition, where items refer to the individual, and referent-shift consensus composition, where items refer to the group. The use and selection of composition methods is subject to a number of problems, calling for more systematic work on the empirical properties of and distinction between constructs composed by different methods. To facilitate and encourage such work, the authors present a methodological framework for addressing the distinction between and the baseline psychometric quality of composed group constructs, illustrated by an empirical example in the group job-design domain. The framework primarily represents a developmental tool with applications in multilevel theory building and scale construction, but also in meta-analysis or secondary analysis, and more general, the validation of group constructs.

In management research, theory testing confronts a paradox described by Meehl in which designing studies with greater methodological rigor puts theories at less risk of falsification. This paradox exists because most management theories make predictions that are merely directional, such as stating that two variables will be positively or negatively related. As methodological rigor increases, the probability that an estimated effect will differ from zero likewise increases, and the likelihood of finding support for a directional prediction boils down to a coin toss. This paradox can be resolved by developing theories with greater precision, such that their propositions predict something more meaningful than deviations from zero. This article evaluates the precision of theories in management research, offers guidelines for making theories more precise, and discusses ways to overcome barriers to the pursuit of theoretical precision.

Việc thiết lập tính bất biến đo lường giữa các nhóm là một điều kiện tiên quyết hợp lý để tiến hành so sánh liên nhóm chính xác (ví dụ như kiểm định sự khác biệt trung bình nhóm, sự bất biến của các ước tính tham số cấu trúc), tuy nhiên tính bất biến đo lường hiếm khi được kiểm tra trong nghiên cứu tổ chức. Trong bài báo này, các tác giả (a) làm rõ tầm quan trọng của việc thực hiện các kiểm định tính bất biến đo lường giữa các nhóm, (b) xem xét các thực hành khuyến nghị cho việc thực hiện các kiểm định tính bất biến đo lường, (c) điểm lại ứng dụng của các kiểm định tính bất biến đo lường trong ứng dụng thực tiễn, (d) thảo luận về các vấn đề liên quan đến kiểm định các khía cạnh khác nhau của tính bất biến đo lường, (e) trình bày một ví dụ thực nghiệm về phân tích tính bất biến đo lường theo thời gian, và (f) đề xuất một mô hình tích hợp cho việc thực hiện các dãy kiểm định tính bất biến đo lường.

We aim to develop a meaningful single-source reference for management and organization scholars interested in using bibliometric methods for mapping research specialties. Such methods introduce a measure of objectivity into the evaluation of scientific literature and hold the potential to increase rigor and mitigate researcher bias in reviews of scientific literature by aggregating the opinions of multiple scholars working in the field. We introduce the bibliometric methods of citation analysis, co-citation analysis, bibliographical coupling, co-author analysis, and co-word analysis and present a workflow for conducting bibliometric studies with guidelines for researchers. We envision that bibliometric methods will complement meta-analysis and qualitative structured literature reviews as a method for reviewing and evaluating scientific literature. To demonstrate bibliometric methods, we performed a citation and co-citation analysis to map the intellectual structure of the Organizational Research Methods journal.

Because of the importance of mediation studies, researchers have been continuously searching for the best statistical test for mediation effect. The approaches that have been most commonly employed include those that use zero-order and partial correlation, hierarchical regression models, and structural equation modeling (SEM). This study extends MacKinnon and colleagues (MacKinnon, Lockwood, Hoffmann, West, & Sheets, 2002; MacKinnon, Lockwood, & Williams, 2004, MacKinnon, Warsi, & Dwyer, 1995) works by conducting a simulation that examines the distribution of mediation and suppression effects of latent variables with SEM, and the properties of confidence intervals developed from eight different methods. Results show that SEM provides unbiased estimates of mediation and suppression effects, and that the bias-corrected bootstrap confidence intervals perform best in testing for mediation and suppression effects. Steps to implement the recommended procedures with Amos are presented.

This article addresses Rönkkö and Evermann’s criticisms of the partial least squares (PLS) approach to structural equation modeling. We contend that the alleged shortcomings of PLS are not due to problems with the technique, but instead to three problems with Rönkkö and Evermann’s study: (a) the adherence to the common factor model, (b) a very limited simulation designs, and (c) overstretched generalizations of their findings. Whereas Rönkkö and Evermann claim to be dispelling myths about PLS, they have in reality created new myths that we, in turn, debunk. By examining their claims, our article contributes to reestablishing a constructive discussion of the PLS method and its properties. We show that PLS does offer advantages for exploratory research and that it is a viable estimator for composite factor models. This can pose an interesting alternative if the common factor model does not hold. Therefore, we can conclude that PLS should continue to be used as an important statistical tool for management and organizational research, as well as other social science disciplines.

Previous research has recommended several measures of effect size for studies with repeated measurements in both treatment and control groups. Three alternate effect size estimates were compared in terms of bias, precision, and robustness to heterogeneity of variance. The results favored an effect size based on the mean pre-post change in the treatment group minus the mean pre-post change in the control group, divided by the pooled pretest standard deviation.

Lodahl and Kejner’s Job Involvement (JI) measure has been and continues to be heavily used despite known measurement deficiencies. Using a convergent evidence approach, the authors examine the psychometric properties of that scale and offer a refined version that accurately taps the JI construct. Based on a combination of five methodologies (qualitative content analysis, classical item analyses, item response theory analyses, partial confirmatory factor analyses, and discriminant validity analysis) applied to five samples, results indicate that numerous items function inadequately as indicators of JI, whereas a core of items have superior item statistics and conceptually match the definition of JI. The advantages of using a convergent evidence approach are discussed.

Many researchers who use same-source data face concerns about common method variance (CMV). Although post hoc statistical detection and correction techniques for CMV have been proposed, there is a lack of empirical evidence regarding their efficacy. Because of disagreement among scholars regarding the likelihood and nature of CMV in self-report data, the current study evaluates three post hoc strategies and the strategy of doing nothing within three sets of assumptions about CMV: that CMV does not exist, that CMV exists and has equal effects across constructs, and that CMV exists and has unequal effects across constructs. The implications of using each strategy within each of the three assumptions are examined empirically using 691,200 simulated data sets varying factors such as the amount of true variance and the amount and nature of CMV modeled. Based on analyses of these data, potential benefits and likely risks of using the different techniques are detailed.