Replicating software engineering experiments: a poisoned chalice or the Holy Grail

Amato, 1993, Pons and Fleischmann Redux, Science, 260, 895, 10.1126/science.260.5110.895

Basili, 1986, Experimentation in software engineering, IEEE Transactions in Software Engineering, 12, 733, 10.1109/TSE.1986.6312975

Basili, 1996, The empirical investigation of perspective-based reading, Empirical Software Engineering, 1, 133, 10.1007/BF00368702

Brewer, 1989

Broad, 1986

Brooks, 1980, Studying programmer behaviour experimentally: the problems of proper methodology, Communications of the ACM, 23, 207, 10.1145/358841.358847

Campbell, 1963

Close, 1990

Cohen, 1977

Collins, 1985

Curtis, 1990, Measurement and experimentation in software engineering, Proceedings of the IEEE, 68, 1144, 10.1109/PROC.1980.11813

Daly, 1994, Verification of results in software maintenance through external replication, 10.1109/ICSM.1994.336790

Dunsmore, 2003, The development and evaluation of three diverse techniques for OO code inspection, IEEE Transactions on Software Engineering, 677, 10.1109/TSE.2003.1223643

Fusaro, 1997, A replicated experiment to assess requirements, Inspection Techniques, 2, 39

Hayes, 1999, Research synthesis in software engineering: a case for meta-analysis, 10.1109/METRIC.1999.809735

Hedges, 1985

Hooker, 1994, Needed: an empirical science of algorithms, Operations Research, 42, 201, 10.1287/opre.42.2.201

Hunter, 1997, Cumulative research knowledge and social policy formulation: the critical role of meta-analysis, Psychology, Public Policy and Law, 2, 324, 10.1037/1076-8971.2.2.324

Kramer, 1999, Effect sizes and significance levels in small sample research

Laitenberger, 2000, An experimental comparison of reading techniques for defect detection in UML design documents, The Journal of Systems and Software, 53, 183, 10.1016/S0164-1212(00)00052-2

Laitenberger, 2001, An internally replicated Quasi-Experimental Comparison of Checklist and Perspective based Reading of Code Documents, IEEE Transactions on Software Engineering, 387, 10.1109/32.922713

Lott, 1996, Repeatable software engineering experiments for comparing defect-detection techniques, Journal of Empirical Software Engineering, 1, 241, 10.1007/BF00127447

McGrath, 1982

Miller, 2004, Statistical significance testing—a Panacea for software technology experiments?, Journal of Systems and Software, 73, 183, 10.1016/j.jss.2003.12.019

Miller, 1998, Further experiences with scenarios and checklists, Empirical Software Engineering, 3, 37, 10.1023/A:1009735805377

Nye, 1986

Porter, 1995, Comparing detection methods for software requirements inspections: a replicated experiment, IEEE Transactions on Software Engineering, 21, 563, 10.1109/32.391380

Rosenthal, 1984

Rosenthal, 1991, Replication in behavioural research

Scanlan, 1989, Structured flowcharts outperform pseudocode: an experimental comparison, IEEE Software, 6, 28, 10.1109/52.35587

Shneiderman, 1977, Experimental investigations of the utility of detailed flowcharts in programming, Communications of the ACM, 20, 373, 10.1145/359605.359610

Thelin, 2003, An experimental comparison of usage-based reading and checklist-based reading, IEEE Transactions of Software Engineering, 29, 687, 10.1109/TSE.2003.1223644

Wolf, 1986

Wood, 1999, Multi-method research: an empirical investigation of object-oriented technology, Journal of Systems and Software, 48, 13, 10.1016/S0164-1212(99)00042-4