Online Automatic Marking of Diagrams
Tóm tắt
This article describes the way in which the Open University is expanding its online quiz engine to enable students to draw free-form diagrams in response to a quiz question. The new quiz engine automatically marks a student’s diagram and provides feedback either instantaneously when used in formative mode or later when used in summative mode. The new quiz engine is hosted within the Open University’s quiz module, OpenMark, which interfaces to the Moodle VLE which is responsible for rendering both the question and the feedback in the user’s browser. The article outlines the general nature of the diagrams that our system has been designed to deal with. This is followed by an example of the use of the new question type as implemented within our e-assessment system. The article also briefly summarizes the results of experiments carried out on the accuracy of the automatic marker and reports on feedback from students who have used the new diagramming question type.
Tài liệu tham khảo
Batmaz F, Stone R, Hinde C (2009) Personalised feedback with semi-automatic assessment tool for conceptual database model. In: Proceedings of 10th annual conference of information and computer sciences. Higher Education Academy Subject Centre, UK, pp 115–120
Bunke H (2000) Graph matching: theoretical foundations, algorithms, and applications. Proc Vis Interface 2000:82–88
Butcher P (2012) eAssessment at the Open University with open source software. http://labspace.open.ac.uk/course/view.php?id=3484. Accessed 27 Mar 2012
Fan S, Tanimoto S (2007) A framework for automated diagram assessment in online learning. In: Proceedings of 7th IEEE international conference on advanced learning technologies (ICALT 2007), IEEE Computer Society, Niigata, Japan, pp 51–53
Haji Ali N, Shukur Z, Idris S (2007) Assessment system for UML class diagram using notations extraction. IJCNS Int J Comput Sci Netw Secur 7(8):181–187
Higgins CA, Bligh B (2006) Formative computer based assessment in diagram based domains. In: Proceedings of 11th annual conference on innovation and technology in computer science education, Bologna, Italy, pp 98–102
Hinde CJ, Batmaz F, Stone R (2008) A web-based semi-automatic assessment tool for conceptual database model. HEA development fund 2008/9. http://www.ics.heacademy.ac.uk/projects/development-fund/fund_details.php?id=125. Accessed 11 Dec 2012
Hoggarth G, Lockyer M (1998) An automated student diagram assessment system. In: Proceedings of 6th conference on innovation and technology in computer science education ITICSE, pp 122–124
Jayal A, Shepperd M (2008) The problem of labels in e-assessment of diagrams. ACM J Educ Resour Comput 8(4):Article 12, 2008
Laurillard D (2008) Digital technologies and their role in achieving our ambitions for education. Institute of Education, London
Moodle (2012) http://www.moodle.org. Accessed 21 Apr 2012
Open University (2012) http://www.open.ac.uk. Accessed 11 Dec 2012
OpenMark (2012) http://www.open.ac.uk/openmarkexamples. Accessed 11 Dec 2012
Roscoe RD, Kugler D, Crossley SA, Weston JL, McNamara DS (2012) Developing pedagogically-guided threshold algorithms for intelligent automated essay feedback. In: Proceedings of 25th international Florida artificial intelligence research society conference. pp 466–471
Smith N, Pete T, Kevin W (2010) Diagram interpretation and e-learning systems. In: Goel AK, Jamnik M, Narayanan NH (eds) Diagrams 2010, LNAI 6170. Springer, Heidelberg, pp 331–333
Soler J, Poch J, Barrabes E, Juher D, Ripoll J (2010) A tool for the continuous assessment and improvement of the student’s skills in a mathematics course. In: Proceedings of international symposium on technology of information and communication in education for engineering and industry, TICE pp 105–110
Stone RG, Batmaz F, Hinde CJ et al (2009) Drawing and marking graph diagrams. Italics 8(2):45–53
Streeter L, Bernstein J, Foltz P, DeLand D (2011) Pearson’s automated scoring of writing, speaking, and mathematics. http://www.pearsonassessments.com/hai/images/automatedscoring/downloads/PearsonsAutomatedScoringofWritingSpeakingandMathematics.pdf. Accessed 19 Nov 2012
Thomas PG, Waugh K, Smith N (2007a) Computer assisted assessment of diagrams. In: Proceedings of the 12th annual conference on innovation and technology in computer science education (ITiCSE), June 25–29, Dundee, Scotland, 2007, pp 25–29
Thomas PG, Waugh K, Smith N (2007b) Learning and automatically assessing graph-based diagrams. In: Wheeler S, Whitton N (eds) Beyond control: learning technology for the social network generation. Research Proceedings of the 14th Association for Learning Technology Conference (ALT-C, 4–6 September, Nottingham, UK, 2007), pp 61–74
Thomas P, Waugh K, Smith N (2007c) Tools for supporting the teaching and learning of data modelling. In: Proceedings ED-MEDIA, Vancouver, Canada, 25–29 June 2007
Thomas P, Smith N, Waugh K (2008) Automatically assessing graph-based diagrams. J Learn Media Technol 33(3):249–267
Thomas PG, Smith N, Waugh K (2009) The role of labels in the automatic assessment of graph-based diagrams. In: Proceedings of 23rd ICDE world conference on open and distance learning (Icdl 2009). Maastricht, Holland, 8–11 June 2009
Thomas PG, Waugh K, Smith N (2011) Generalised diagramming tools with automatic marking. Italics 10(1):22–33
Tselonis C, Sargeant J (2007) Domain specific formative feedback through domain-independent diagram matching. In: Proceedings of 11th international computer assisted assessment conference, Loughborough. http://www.caaconference.co.uk/pastConferences/2007/proceedings/index.asp
Tselonis C et al (2005) Diagram matching for human-computer collaborative assessment. In: Proceedings of 9th international computer assisted assessment conference, Loughborough. http://www.caaconference.co.uk/pastConferences/2005/proceedings/index.asp
Tsintsifas A (2002) A framework for the computer based assessment of diagram-based coursework. PhD Thesis, Computer Science Department, University of Nottingham, Nottingham