Scalable authentic assessment of collaborative work assignments in wikis
Tóm tắt
Wikis are appropriate tools for deploying authentic assessment experiences for learning and work scenarios in which a group of users are asked to develop a shared task. However, when the number of wiki users increases, the number of contributions can grow at a pace whereby accurately assessing them becomes a complex and non-scalable task. While different quantitative approaches have been shown to be scalable, they are usually coarse-grained and provide limited feedback about the assessment. This work proposes a scalable assessment methodology for wiki-based tasks, based on qualitative self- and peer assessment of wiki contributions. The methodology is implemented using a software tool and is applied as part of an undergraduate course, complementing a quantitative assessment approach. Positive evidence on the scalability of the method and how it implements a more fine-grained qualitative assessment than the regular quantitative approach is found, providing indicators for assessing both individual and group generic skills.
Tài liệu tham khảo
Arevalillo-Herráez, M., Perez-Muñoz, R., & Ezbakhe, Y. (2010). A wiki based system to produce high quality teaching materials. In 5th Iberian Conference on Information Systems and Technologies (CISTI), (pp. 1–4).
Aronsson, L. (2002). Operation of a large scale, general purpose wiki website. In Proceedings of the 6th International ICCC/IFIP Conference on Electronic Publishing. ELPUB, (pp. 27–37). Karlovy Vary: VWF Berlin.
Ashford-Rowe, K., Herrington, J., & Brown, C. (2014). Establishing the critical elements that determine authentic assessment. Assessment & Evaluation in Higher Education, 39(2), 205–222 https://doi.org/10.1080/02602938.2013.819566.
Balderas, A., de-la-Fuente-Valentin, L., Ortega-Gomez, M., Dodero, J. M., & Burgos, D. (2018). Learning management systems activity records for students’ assessment of generic skills. IEEE Access, 6, 15958–15968 https://doi.org/10.1109/ACCESS.2018.2816987.
Benlloch, J. V., Benet, G., Blanc, S., Gil, D., Busquets, J. V., Gil, P., … Albaladejo, J. (2012). Análisis de la implantación de la asignatura Tecnología de Computadores en el Grado de Ingeniería Informática. In 10th Congreso de Tecnologías Aplicadas en la Enseñanza de la Electrónica, (pp. 358–363).
Bhopale, S. D. (2013). Cloud migration benefits and its challenges issue. IOSR Journal of Computer Engineering, 1(8), 40–45.
Biggs, J. (2015). Assessment in a constructively aligned system. In International Conference Assessment for Learning in Higher Education. Hong Kong: The University of Hong Kong. https://www.cetl.hku.hk/conf2015/assessment-in-a-constructively-aligned-system/.
Biggs, J. B., Tang, C. S., & Society for Research into Higher Education (2011). Teaching for quality learning at university: What the student does. Berkshire: McGraw-Hill/Society for Research into Higher Education/Open University Press.
Bocconi, S., & Trentin, G. (2012). Wiki supporting formal and informal learning. New York: Nova Science Publishers.
Bondi, A. B. (2000). Characteristics of scalability and their impact on performance. In Proceedings of the second international workshop on software and performance - WOSP ‘00 (pp. 195–203). New York: ACM Press. https://dl.acm.org/citation.cfm?id=350432
Boud, D., & Soler, R. (2015). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 0(0), 1–14.
Brodie, M. L., & Stonebraker, M. (1995). Migrating legacy systems: gateways, interfaces & the incremental approach. Massachusetts: Morgan Kaufmann Publishers.
Brown, S. (2014). Learning, teaching and assessment in higher education: global perspectives. Basingstoke: Palgrave Macmillan.
Brown, S., & Pickford, R. (2006). Assessing skills and practice. Abingdon: Routledge.
Cole, M. (2009). Using Wiki technology to support student engagement: Lessons from the trenches. Computers & Education, 52(1), 141–146.
Committee on the Foundations of Assessment, Pellegrino, J. W., Chudowsky, N., Glaser, R., Board on Testing and Assessment, Center for Education, … National Research Council (U.S.) (2001). Knowing what students Know: The science and Design of Educational Assessment. Washington: National Academies Press.
De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2011). Assessing collaboration in a wiki: the reliability of university students’ peer assessment. The Internet and Higher Education, 14(4), 201–206 https://doi.org/10.1016/j.iheduc.2011.07.003.
Díaz, O., & Puente, G. (2012). Wiki scaffolding: aligning wikis with the corporate strategy. Information Systems, 37(8), 737–752 https://doi.org/10.1016/j.is.2012.05.002.
Dillenbourg P. (1999) What do you mean by 'collaborative learning'?. In P. Dillenbourg (Ed) Collaborative Learning Cognitive and Computational Approaches, (pp.1–19). Oxford: Elsevier. https://telearn.archives-ouvertes.fr/hal-00190240/document.
Elgort, I., Smith, A. G., & Toland, J. (2008). Is wiki an effective platform for group course work? Australasian Journal of Educational Technology, 24(2), 195–210.
Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education: a meta-analysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287–322.
Florian-Gaviria, B., Glahn, C., & Fabregat Gesa, R. (2013). A software suite for efficient use of the European qualifications framework in online and blended courses. IEEE Transactions on Learning Technologies, 6(3), 283–296 https://doi.org/10.1109/TLT.2013.18.
Fountain, R. (2005). Wiki pedagogy. Dossiers Pratiques. Profetic.
Gielen, S., Dochy, F., & Onghena, P. (2011). An inventory of peer assessment diversity. Assessment & Evaluation in Higher Education, 36(2), 137–155 https://doi.org/10.1080/02602930903221444.
Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67–86.
Hatzipanagos, S., & Warburton, S. (2009). Feedback as dialogue: Exploring the links between formative assessment and social software in distance learning. Learning, Media and Technology, 34(1), 45–59.
Henkel, M. (2007). Can academic autonomy survive in the knowledge society? A perspective from Britain. Higher Education Research & Development, 26(1), 87–99.
Herrington, J., & Herrington, A. (2006). Authentic conditions for authentic assessment: aligning task and assessment: In A. Bunker & I. Vardi (Eds.), Proceedings of the 2006 Annual International Conference of the Higher Education Research and Development Society of Australasia Inc (HERDSA): Critical Visions: Thinking, Learning and Researching in Higher Education: Research and Development in Higher Education, 29, 141–151. Milperra: HERDSA.
Ibarra Saiz, M. S., Rodriguez Gomez, G., & Gomez Ruiz, M. A. (2012). Benefits of peer assessment and strategies for its practice at university. Revista de Educación, 359, 206–231.
Ibarra-Sáiz, M. S., & Rodríguez-Gómez, G. (2017). EvalCOMIX® a web-based programme to support collaboration in assessment. In Smart Technology Applications in Business Environments, (pp. 249–275). IGI Global https://doi.org/10.4018/978-1-5225-2492-2.ch012.
Jackson, D. (2009). Wiki-tecture: The DRAPE artist residence and gallery. Journal of Architectural Education, 63(1), 97–106.
Jaksch, B., Kepp, S.-J., & Womser-Hacker, C. (2008). Integration of a wiki for collaborative knowledge development in an E-learning context for university teaching. In A. Holzinger (Ed.), HCI and Usability for Education and Work (Vol. 5298, pp. 77–96). New York: Springer Berlin Heidelberg.
Koper, R. (2014). Conditions for effective smart learning environments. Smart Learning Environments, 1(1), 5.
Lacuesta, R., Palacios, G., & Fernández, L. (2009). Active learning through problem based learning methodology in engineering education. In Frontiers in Education Conference, 2009. FIE’09. 39th IEEE, (pp. 1–6).
Leuf, B., & Cunningham, W. (2001). The wiki way: quick collaboration on the web. Boston: Addison-Wesley Professional.
Llorens, A., Llinàs-Audet, X., Ras, A., & Chiaramonte, L. (2013). The ICT skills gap in Spain: industry expectations versus university preparation. Computer Applications in Engineering Education, 21(2), 256–264 https://doi.org/10.1002/cae.20467.
Lykourentzou, I., Papadaki, K., Vergados, D. J., Polemi, D., & Loumos, V. (2010). CorpWiki: A self-regulating wiki to promote corporate collective intelligence through expert peer matching. Information Sciences, 180(1), 18–38.
Macdonald, J. (2003). Assessing online collaborative learning: process and product. Computers & Education, 40(4), 377–391 https://doi.org/10.1016/S0360-1315(02)00168-9.
Minocha, S., Petre, M., & Roberts, D. (2008). Using wikis to simulate distributed requirements development in a software engineering course. International Journal of Engineering Education, 24(4), 689–704.
Mitchell, P. (2006). Wikis in education (Ed. Jane K).
Moskaliuk, J., Kimmerle, J., & Cress, U. (2012). Collaborative knowledge building with wikis: The impact of redundancy and polarity. Computers & Education, 58(4), 1049–1057.
Mueller, J. (2005). The authentic assessment toolbox: enhancing student learning through online faculty development. Journal of Online Learning and Teaching, 1(1),1–7.
Oates, B. J. (2006). Action research. In Researching information systems and computing, (pp. 154–172). London: Sage Publications.
Onah, D. F. O., Sinclair, J., & Boyatt, R. (2014). Dropout rates of massive open online courses: behavioural patterns. In 6th international conference on education and new learning technologies (EDULEARN’14), (pp. 5825–5834).
Ortega, F., González-Barahona, J. M., & Robles, G. (2007). The top-ten wikipedias-a quantitative analysis using wikixray. In ICSOFT (ISDM/EHST/DC), (pp. 46–53).
Ortega Valiente, J., & Reinoso Peinado, A. J. (2011). New educational approach based on the use of wiki platforms in university environments. In Next Generation Web Services Practices (NWeSP), 2011 7th International Conference on, (pp. 280–284).
Palomo-Duarte, M., Dodero, J. M., García-Domínguez, A., Neira-Ayuso, P., Sales-Montes, N., Medina-Bulo, I., … Balderas, A. (2014). Scalability of assessments of wiki-based learning experiences in higher education. Computers in Human Behavior, 31(1) https://doi.org/10.1016/j.chb.2013.07.033.
Parker, K., & Chao, J. (2007). "Wiki as a teaching tool." Interdisciplinary journal of knowledge and learning objects. Interdisciplinary Journal of E-Learning and Learning Objects, 3(1), 57–72. Informing Science Institute.
Pearce, M., Zeadally, S., & Hunt, R. (2013). Virtualization. ACM Computing Surveys, 45(2), 1–39 https://doi.org/10.1145/2431211.2431216.
Robson, C., & McCartan, K. (2016). Real world research: a resource for users of social research methods in applied settings, (4th ed., ). Chichester: Wiley.
Rodríguez-Gómez, G., & Ibarra-Sáiz, M. S. (2015). Assessment as learning and empowerment: towards sustainable learning in higher education, (pp. 1–20). Cham: Springer https://doi.org/10.1007/978-3-319-10804-9_1.
Runeson, P., & Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, 14(2), 131–164 https://doi.org/10.1007/s10664-008-9102-8.
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
Sousa, F., Aparicio, M., & Costa, C. J. (2010). Organizational wiki as a knowledge management tool. In Proceedings of the 28th ACM international conference on Design of Communication, (pp. 33–39). New York: ACM.
Spencer, L. M., & Spencer, S. M. (1993). Competence at work: models for superior performance. New York: Wiley.
Su, F., & Beaumont, C. (2010). Evaluating the use of a wiki for collaborative learning. Innovations in Education and Teaching International, 47(4), 417–431.
Trentin, G. (2009). Using a wiki to evaluate individual contribution to a collaborative learning project. Journal of Computer Assisted Learning, 25(1), 43–55.
Viégas, F. B., Wattenberg, M., & Dave, K. (2004). Studying cooperation and conflict between authors with history flow visualizations. In Proceedings of the 2004 conference on human factors in computing systems - CHI ‘04, (pp. 575–582). New York: ACM Press https://doi.org/10.1145/985692.985765.
Wattenberg, M., Viégas, F., & Hollenbach, K. (2007). Visualizing activity on Wikipedia with chromograms. In C. Baranauskas, P. Palanque, J. Abascal, & S. Barbosa (Eds.), Human-Computer Interaction – INTERACT 2007 (Vol. 4663, pp. 272–287). New York: Springer Berlin Heidelberg.
Xiao, Y., & Lucking, R. (2008). The impact of two types of peer assessment on students’ performance and satisfaction within a Wiki environment. The Internet and Higher Education, 11(3), 186–193.