The quality of experimental designs in mobile learning research: A systemic review and self-improvement tool

Ahmed, 2013, Abductive science inquiry using mobile devices in the classroom, Computers & Education, 63, 62, 10.1016/j.compedu.2012.11.017 American Psychological Association, 2013 Aron, 1999 Aron, 2012 Asabere, 2012, The necessity and relevance of mobile computer-supported collaborative learning (mCSCL): A review, International Journal of Science and Technology, 2, 413 Barron, 2014, Debating curricular strategies for teaching statistics and research methods: What does the current evidence suggest?, Teaching of Psychology, 41, 187, 10.1177/0098628314537967 Baydas, 2015, Educational technology research trends from 2002 to 2014, Scientometrics, 105, 709, 10.1007/s11192-015-1693-4 Bhagat, 2018, To use augmented reality or not in formative assessment: A comparative study, Interactive Learning Environments Bloom, 1999, Using cluster random assignment to measure program impacts: Statistical implications for the evaluation of education programs, Evaluation Review, 23, 445, 10.1177/0193841X9902300405 Boeker, 2013, Game-based e-learning is more effective than a conventional instructional method: A randomized controlled trial with third-year medical students, PLoS One, 8, 10.1371/journal.pone.0082328 Boland, 2017 Borman, 2009, The use of randomized trials to inform education policy, 129 Boruch, 1997 Boruch, 2017, Randomized controlled trials, 15 Burkhardt, 2003, Improving educational research: Toward a more useful, more influential, and better-funded enterprise, Educational Researcher, 32, 3, 10.3102/0013189X032009003 Burston, 2015, Twenty years of MALL project implementation: A meta-analysis of learning outcomes, ReCALL, 27, 4, 10.1017/S0958344014000159 Campbell, 1963 Campbell, 1966 Cascio, 2017, Using critical thinking drills to teach and assess proficiency in methodological and statistical thinking, Teaching of Psychology, 44, 250, 10.1177/0098628317712753 Chambless, 2001, Empirically supported psychological interventions: Controversies and evidence, Annual Review of Psychology, 52, 685, 10.1146/annurev.psych.52.1.685 Chang, 2018, Trends and research issues of mobile learning studies in nursing education: A review of academic publications from 1971 to 2016, Computers & Education, 116, 28, 10.1016/j.compedu.2017.09.001 Chang, 2016, Using mobile devices to enhance the interactive learning for spatial geometry, Interactive Learning Environments, 24, 916, 10.1080/10494820.2014.948458 Chao, 2016, Integration of mobile AR technology in performance assessment, Journal of Educational Technology & Society, 19, 239 Chen, 2008, Effects of short-term memory and content representation type on mobile language learning, Language, Learning and Technology, 12, 93 Cheung, 2012, How features of educational technology applications affect student reading outcomes: A meta-analysis, Educational Research Review, 7, 198, 10.1016/j.edurev.2012.05.002 Cheung, 2013, The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis, Educational Research Review, 9, 88, 10.1016/j.edurev.2013.01.001 Chinnery, 2006, Emerging technologies going to the MALL: Mobile assisted language learning, Language, Learning and Technology, 10, 9 Choi, 2015, Comparative analysis of the use of mobile microblogging and non-mobile online message board for group collaboration, International Journal of Electronic Commerce, 19, 112, 10.1080/10864415.2015.1029360 Coe Cohen, 1988 Cohen, 1992, A power primer, Psychological Bulletin, 112, 155, 10.1037/0033-2909.112.1.155 Cohen, 1994, The earth is round (p<.05), American Psychologist, 49, 997, 10.1037/0003-066X.49.12.997 Collins, 2007, Teaching evidence‐based practice: Implications for psychology, Journal of Clinical Psychology, 63, 657, 10.1002/jclp.20378 Connolly, 2018, The trials of evidence-based practice in education: A systematic review of randomised controlled trials in education research 1980-2016, Educational Research, 60, 276, 10.1080/00131881.2018.1493353 Connor, 2019, Building word knowledge, learning strategies, and metacognition with the Word-Knowledge e-Book, Computers & Education, 128, 284, 10.1016/j.compedu.2018.09.016 Cook, 2000, Toward a practical theory of external validity, Vol. 1, 3 Cook, 2015, CEC's standards for classifying the evidence base of practices in special education, Remedial and Special Education, 36, 220, 10.1177/0741932514557271 Crompton, 2017, The use of mobile learning in PK-12 education: A systematic review, Computers & Education, 110, 51, 10.1016/j.compedu.2017.03.013 Dashtestani, 2016, Moving bravely towards mobile learning: Iranian students' use of mobile devices for learning English as a foreign language, Computer Assisted Language Learning, 29, 815, 10.1080/09588221.2015.1069360 Duman, 2015, Research trends in mobile assisted language learning from 2000 to 2012, ReCALL, 27, 197, 10.1017/S0958344014000287 D'Agostino, 2005, Measuring learning outcomes: Reliability and validity issues, 113 Evans, 2008, The effectiveness of m-learning in the form of podcast revision lectures in higher education, Computers & Education, 50, 491, 10.1016/j.compedu.2007.09.016 Faul, 2009, Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses, Behavior Research Methods, 41, 1149, 10.3758/BRM.41.4.1149 Faul, 2007, G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences, Behavior Research Methods, 39, 175, 10.3758/BF03193146 Flanigan, 2018, What college instructors can do about student cyber-slacking, Educational Psychology Review, 30, 585, 10.1007/s10648-017-9418-2 Fleischer, 2012, What is our current understanding of one-to-one computer projects: A systematic narrative research review, Educational Research Review, 7, 107, 10.1016/j.edurev.2011.11.004 Fraenkel, 2011 Gao, 2016, Effects of mode of target task selection on learning about plants in a mobile learning environment: Effortful manual selection versus effortless QR-code selection, Journal of Educational Psychology, 108, 694, 10.1037/edu0000080 Garcia-Cabot, 2015, An empirical study on m-learning adaptation: Learning performance and learning contexts, Computers & Education, 82, 450, 10.1016/j.compedu.2014.12.007 Gebru, 2012, Effect of clickers versus online homework on students' long-term retention of general chemistry course material, Chemistry Education: Research and Practice, 13, 325 Gentry, 2008, PDAs as cognitive aids for people with multiple sclerosis, American Journal of Occupational Therapy, 62, 18, 10.5014/ajot.62.1.18 Glass, 1972, Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance, Review of Educational Research, 42, 237, 10.3102/00346543042003237 Gresham, 2004, Social skills training for children and youth with emotional and behavioral disorders: Validity considerations and future directions, Behavioral Disorders, 30, 32, 10.1177/019874290403000101 Higgins, 2011 Hirsh-Pasek, 2018, Living in pasteur's quadrant: How conversational duets spark language at home and in the community, Discourse Processes, 55, 338, 10.1080/0163853X.2018.1442114 Hong, 2017, An exploration of students’ science learning interest related to their cognitive anxiety, cognitive load, self-confidence and learning progress using inquiry-based learning with an iPad, Research in Science Education, 47, 1193, 10.1007/s11165-016-9541-y Howell, 2002 Hsu, 2013, Mobile computer-supported collaborative learning: A review of experimental research, British Journal of Educational Technology, 44, E111, 10.1111/bjet.12002 Hsu, 2017, Effects of a structured resource-based web issue-quest approach on students' learning performances in computer programming courses, Journal of Educational Technology & Society, 20, 82 Huang, 2017, Mobile game-based learning with a mobile App: Motivational effects and learning performance, Journal of Advanced Computational Intelligence and Intelligent Informatics, 21, 963, 10.20965/jaciii.2017.p0963 Huck, 2008 Hwang, 2011, Research trends in mobile and ubiquitous learning: A review of publications in selected journals from 2001 to 2010, British Journal of Educational Technology, 42, E65, 10.1111/j.1467-8535.2011.01183.x Jimenez-Buedo, 2010, Why a trade-off? The relationship between the external and internal validity of experiments, Theoria, 25, 301, 10.1387/theoria.779 Jonsson, 2007, The use of scoring rubrics: Reliability, validity and educational consequences, Educational Research Review, 2, 130, 10.1016/j.edurev.2007.05.002 Juni, 2001, Assessing the quality of controlled clinical trials, BMJ, 323, 42, 10.1136/bmj.323.7303.42 Karaman, 2011, Effects of audience response systems on student achievement and long-term retention, Social Behavior and Personality, 39, 1431, 10.2224/sbp.2011.39.10.1431 Kirk, 1995 Kirk, 1996, Practical significance: A concept whose time has come, Educational and Psychological Measurement, 56, 746, 10.1177/0013164496056005002 Klopfer, 2002, Environmental detectives: PDAs as a window into a virtual simulated world, 95 LaCosse, 2017, An active-learning approach to fostering understanding of research methods in large classes, Teaching of Psychology, 44, 117, 10.1177/0098628317692614 Lakens, 2013, Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs, Frontiers in Psychology, 4, 863, 10.3389/fpsyg.2013.00863 Lan, 2016, Mobile seamless technology enhanced CSL oral communication, Journal of Educational Technology & Society, 19, 335 Levin, 2004, Random thoughts on the (in) credibility of educational-psychological intervention research, Educational Psychologist, 39, 173, 10.1207/s15326985ep3903_3 Lin, 2014, Learning English reading in a mobile-assisted extensive reading program, Computers & Education, 78, 48, 10.1016/j.compedu.2014.05.004 Lin, 2017, Effects of spell checkers on English as a second language students' incidental spelling learning: A cognitive load perspective, Reading and Writing, 30, 1501, 10.1007/s11145-017-9734-4 Lin, 2019, Chinese undergraduate students' perceptions of mobile learning: Conceptions, learning profiles, and approaches, Journal of Computer Assisted Learning, 10.1111/jcal.12333 Lin, 2016, The effects of mobile learning on students' oral performance in Mandarin Chinese and their attitudes, International Journal of Mobile Learning and Organisation, 10, 78, 10.1504/IJMLO.2016.076191 Liou, 2016, Beyond the flipped classroom: A highly interactive cloud-classroom (HIC) embedded into basic materials science courses, Journal of Science Education and Technology, 25, 460, 10.1007/s10956-016-9606-8 Lipsey, 1993, The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis, American Psychologist, 48, 1181, 10.1037/0003-066X.48.12.1181 Liu, 2016, Identifying learning features and models for context-aware ubiquitous learning with phenomenological research method, International Journal of Mobile Learning and Organisation, 10, 238, 10.1504/IJMLO.2016.079501 Liu, 2019, The modality effect in a mobile learning environment: Learning from spoken text and real objects, British Journal of Educational Technology, 50, 574, 10.1111/bjet.12605 Liu, 2009, Outdoor natural science learning with an RFID-supported immersive ubiquitous learning environment, Journal of Educational Technology & Society, 12, 161 Malouf, 2016, Anatomy of an evidence base, Educational Researcher, 45, 454, 10.3102/0013189X16678417 Mayer, 2005, The failure of educational research to impact educational practice: Six obstacles to educational reform, 67 Moher, 2001, The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials, Lancet, 357, 1191, 10.1016/S0140-6736(00)04337-3 Morris, 2012, Can a tablet device alter undergraduate science students' study behavior and use of technology?, Advances in Physiology Education, 36, 97, 10.1152/advan.00104.2011 National Research Council, 2002 Pagano, 2007 Patchan, 2016, Using tablet computers to teach preschool children to write letters: Exploring the impact of extrinsic and intrinsic feedback, Computers & Education, 102, 128, 10.1016/j.compedu.2016.07.007 Paule-Ruiz, 2017, Music learning in preschool with mobile devices, Behaviour & Information Technology, 36, 95 Pimmer, 2016, Mobile and ubiquitous learning in higher education settings. A systematic review of empirical studies, Computers in Human Behavior, 63, 490, 10.1016/j.chb.2016.05.057 Raudenbush, 1997, Statistical analysis and optimal design for cluster randomized trials, Psychological Methods, 2, 173, 10.1037/1082-989X.2.2.173 Raudenbush, 2002, Vol. 1 Roschelle, 2010, Scaffolding group explanation and feedback with handheld technology: Impact on students' mathematics learning, Educational Technology Research & Development, 58, 399, 10.1007/s11423-009-9142-9 Sana, 2013, Laptop multitasking hinders classroom learning for both users and nearby peers, Computers & Education, 62, 24, 10.1016/j.compedu.2012.10.003 Sandberg, 2011, Mobile English learning: An evidence-based study with fifth graders, Computers & Education, 57, 1334, 10.1016/j.compedu.2011.01.015 Schraw, 2008, A need for quality research in e-learning, 1 Shadish, 2002 Sharples, 2000, The design of personal mobile technologies for lifelong learning, Computers & Education, 34, 177, 10.1016/S0360-1315(99)00044-5 Shih, 2011, An investigation of attitudes of students and teachers about participating in a context-aware ubiquitous learning activity, British Journal of Educational Technology, 42, 373, 10.1111/j.1467-8535.2009.01020.x Slavin, 2002, Evidence-based education policies: Transforming educational practice and research, Educational Researcher, 31, 15, 10.3102/0013189X031007015 Slavin, 2003, A researcher's guide to scientifically based research, Educational Leadership, 60, 12 Song, 2014, Methodological issues in mobile computer-supported collaborative learning (mCSCL): What methods, what to measure and when to measure?, Educational Technology & Society, 17, 33 Song, 2010, Critical issues and common pitfalls in designing and conducting impact studies in education: Lessons learned from the what works clearinghouse (phase I), Educational Evaluation and Policy Analysis, 32, 351, 10.3102/0162373710373389 Stoloff, 2004, Seven goals for effective program assessment, 29 Stone, 2003, Editorial: Modification to “instructions to authors”, Health Psychology, 22, 331, 10.1037/0278-6133.22.4.331 Suárez, 2018, A review of the types of mobile activities in mobile inquiry-based learning, Computers & Education, 118, 38, 10.1016/j.compedu.2017.11.004 Sung, 2010, Designing an electronic guidebook for learning engagement in a museum of history, Computers in Human Behavior, 26, 74, 10.1016/j.chb.2009.08.004 Sung, 2016, The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis, Computers & Education, 94, 252, 10.1016/j.compedu.2015.11.008 Sung, 2015, How effective are mobile devices for language learning? A meta-analysis, Educational research review, 16, 68, 10.1016/j.edurev.2015.09.001 Sung, 2017, The effects of mobile-computer-supported collaborative learning: Meta-analysis and critical synthesis, Review of Educational Research, 87, 768, 10.3102/0034654317704307 Tabachnick, 2007 Tamim, 2011, What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study, Review of Educational Research, 81, 4, 10.3102/0034654310393361 Tezer, 2018, The impact of using mobile-supported learning management systems in teaching web design on the academic success of students and their opinions on the course, Interactive Learning Environments, 26, 402, 10.1080/10494820.2017.1337037 Tsai, 2019, Preservice teachers' conceptions of teaching using mobile devices and the quality of technology integration in lesson plans, British Journal of Educational Technology, 50, 614, 10.1111/bjet.12613 Tu, 2018 Turkle, 2017 Valentine, 2008, A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: The study design and implementation assessment device (Study DIAD), Psychological Methods, 13, 130, 10.1037/1082-989X.13.2.130 Wang, 2011, Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system, Expert Systems with Applications, 38, 10831, 10.1016/j.eswa.2011.02.083 What Works Clearinghouse, 2014 What Works Clearinghouse, 2017 Wong, 2011, What seams do we remove in mobile assisted seamless learning? A critical review of the literature, Computers & Education, 57, 2364, 10.1016/j.compedu.2011.06.007 Yang, 2013, The effectiveness of automatic text summarization in mobile learning contexts, Computers & Education, 68, 233, 10.1016/j.compedu.2013.05.012 Yen, 2012, The effects of image-based concept mapping on the learning outcomes and cognitive processes of mobile learners, British Journal of Educational Technology, 43, 307, 10.1111/j.1467-8535.2011.01189.x Yu, 2016, Academic achievements and satisfaction of the clicker-aided flipped business English writing class, Journal of Educational Technology & Society, 19, 298 Zheng, 2013, Digital writing and diversity: The effects of school laptop programs on literacy processes and outcomes, Journal of Educational Computing Research, 48, 267, 10.2190/EC.48.3.a Zheng, 2016, Learning in one-to-one laptop environments: A meta-analysis and research synthesis, Review of Educational Research, 86, 1052, 10.3102/0034654316628645 Zheng, 2016, Exploring the behavioral patterns of co-regulation in mobile computer-supported collaborative learning, Smart Learning Environments, 3, 1, 10.1186/s40561-016-0024-4 Zurita, 2004, Computer supported collaborative learning using wirelessly interconnected handheld computers, Computers & Education, 42, 289, 10.1016/j.compedu.2003.08.005