Exploiting on-the-fly interpretation to design technical documents in a mobile context
Tóm tắt
Pen-based interaction is well adapted for writing down information in a mobile context. However, there is a lack of software taking advantage of this interaction process to design technical documents in constrained environments. This is because sketch interpretation is a complex research problem and good performances are required to design industrial software. The first contribution of this article is to show how “on-the-fly” interpretation is a viable approach to design robust and efficient sketch interpretation systems. With this process, the system interprets each stroke after its drawing and produces a visual feedback to the user. The interaction with the user is then strongly taken into account to get pertinent information and better interpret the document. Moreover, on-the-fly interpretation opens a large panel of possibilities to design accurate documents, such as contextual help, complex layout reproduction, etc. Of course, such a process has several requirements to be usable in practice, for example to avoid disturbing the user with inconsistent feedback. The second contribution of this paper is a set of strategies, based on grammatical modeling and pattern recognition, to face these needs. We present two industrial pen-based software that are based on the principles we present in this article. One of them, dedicated to architectural floor-plan design, as not been presented previously.
Tài liệu tham khảo
Almaksour A, Anquetil E, Quiniou S, Cheriet M (2010) Personalizable pen-based interface using lifelong learning. In: Proceedings of the international conference on frontiers in handwriting recognition (ICFHR’10)
Alvarado C, Davis R (2004) Sketchread: a multi-domain sketch recognition engine. In: Proceedings of the 17th annual ACM symposium on user interface software and technology (UIST’04), pp 23–32
Anstice J, Bell T, Cockburn A, Setchell M (1996) The design of a pen-based musical input system. In: Proceedings of the sixth Australian conference on computer-human interaction (OzChi’96), pp 260–267
Blostein D, Lank E, Rose A, Zanibbi R (2001) User interfaces for on-line diagram recognition. In: Selected papers from the fourth international workshop on graphics recognition algorithms and applications. LNCS, vol 2390. Springer, Berlin, pp 93–103
Caetano A, Goulart N, Fonseca M, Jorge J (2002) Javasketchit: Issues in sketching the look of user interfaces. In: Proceedings of the AAAI spring symposium on sketch understanding (SSSU’02)
Chen Q, Grundy J, Hosking J (2003) An e-whiteboard application to support early design-stage sketching of uml diagrams. In: Proceedings of the 2003 IEEE symposium on human centric computing languages and environments, pp 219–226
Chok SS, Marriott K (2003) Automatic generation of intelligent diagram editors. ACM Trans Comput-Hum Interact 10(3):244–276
Costagliola G, Deufemia V, Risi M (2005) Sketch grammars: A formalism for describing and recognizing diagrammatic sketch languages. In: Proceedings of the eighth international conference on document analysis and recognition (ICDAR’05), Seoul, Korea, pp 1226–1231
Coyette A, Kieffer S, Vanderdonckt J (2007) Multi-fidelity prototyping of user interfaces. In: INTERACT 2007. LNCS, vol 4662. Springer, Berlin, pp 150–164. Part I
Donaldson AF, Williamson A (2005) Pen-based unput of uml activity diagrams for business process modelling. In: Proceedings of the first workshop on improving and assessing pen-based input techniques
Evodia (2010) http://www.evodia.fr
Feng G, Viard-Gaudin C, Sun Z (2009) On-line hand-drawn electric circuit diagram recognition using 2d dynamic programming. Pattern Recognit 42:3215–3223
Forsberg A, Dieterich M, Zeleznik R (1998) The music notepad. In: ACM symposium on user interface software and technology, San Francisco, CA, USA, pp 203–210
Fujiyama K, Iizuka K, Tanaka J (1999) Vic: Cmg input system using example figures. In: Proceedings of the international symposium on future software technology, Nanjing, China, pp 67–72
Gennari L, Kara LB, Stahovich TF, Shimada K (2005) Combining geometry and domain knowledge to interpret hand-drawn diagrams. Comput Graph 29(4):547–562
Gross MD (1996) The electronic cocktail napkin—a computational environment for working with design diagrams. Des Stud 17:53–69
Hall A, Pomm C, Widmayer P (2007) A combinatorial approach to multi-domain sketch recognition. In: EUROGRAPHICS workshop on sketch-based interfaces and modelling (SBIM’07)
Hammond T, Davis R (2005) Ladder, a sketching language for user interface developers. Comput Graph 29(4):518–532
Jorge JA, Glinert EP (1995) Online parsing of visual languages using adjacency grammars. In: Proceedings of the eleventh international IEEE symposium on visual languages. IEEE Computer Society, Washington, pp 250–257
Kara LB, Stahovich TF (2004) Hierarchical parsing and recognition of hand-sketched diagrams. In: Proceedings of the 17th annual ACM symposium on user interface software and technology (UIST’04), pp 13–22
Kim DH, Kim MJ (2006) A curvature estimation for pen input segmentation in sketch-based modeling. Comput Aided Des 36(3):238–248
Kurtoglu T, Stahovich TF (2002) Interpreting schematic sketches using physical reasoning. In: Proceedings of the AAAI spring symposium on sketch understanding (SSSU’02), Palo Alto, CA, USA, pp 78–85
Landay JA, Myers BA (2001) Sketching interfaces: Toward more human interface design. Computer 34(3):56–64
Lank E, Thorley JS, Chen S, Blostein D (2001) On-line recognition of uml diagrams. In: Proceedings of the sixth international conference on document analysis and recognition (ICDAR’01), Seattle, USA, pp 356–360
Macé S, Anquetil E (2009) Eager interpretation of on-line hand-drawn structured documents: the Dali methodology. Pattern Recognit 42(12):3202–3214
Macé S, Anquetil E, Bossis B (2007) Pen-based interaction for intuitive music composition and editing. In: Shen J, Shepherd J, Cui B, Liu L (eds) Intelligent music information systems: tools and methodologies, Idea Group, pp 261–288
Mankoff J, Hudson SE, Abowd GD (2000) Oops: a toolkit supporting mediation techniques for resolving ambiguity in recognition-based interfaces. Comput Graph 24(6):819–834
Marriott K (1994) Constrain multiset grammars. In: Proceedings of the IEEE symposium on visual languages (VL’94), St Louis, MO, USA, pp 118–125
Marriott K, Meyer B, Wittenburg K (1998) A survey of visual language specification and recognition. In: Marriott K, Meyer B (eds) Theory of visual languages. Springer, Berlin
Mas J, Sánchez G, Lladós J, Lamiroy B (2007) An incremental on-line parsing algorithm for recognizing sketching diagrams. In: Proceedings of the ninth international conference on document analysis and recognition (ICDAR’07), Curitiba, Brazil, pp 452–456
Mas J, Lladós J, Sánchez G, Jorge JA (2010) A syntactic approach based on distortion-tolerant adjacency grammars and a spatial-directed parser to interpret sketched diagrams. Pattern Recognit 43:4148–4164
Miyao H, Maruyama M (2007) An online handwritten music symbol recognition system. Int J Doc Anal Recognit 9(1):49–58
Mouchère H, Anquetil E (2006) A unified strategy to deal with different natures of reject. In: Proceedings of the international conference on pattern recognition (ICPR’06), Hong Kong, China, pp 792–795
Nakagawa M, Machii K, Kato N, Souya T (1993) Lazy recognition as a principle of pen interfaces. In: Proceedings of the conference companion on human factors in computing systems (INTERACT’93 and CHI’93), Amsterdam, The Netherlands, pp 89–90
Plimmer B, Apperley M (2003) Software to sketch interface designs. In: Proceedings of the ninth international conference on human-computer interaction (INTERACT’03), Zürich, Switzerland, pp 73–80
Sezgin TM, Davis R (2007) Sketch interpretation using multiscale models of temporal patterns. IEEE Comput Graph Appl 27(1):28–37
Sezgin TM, Stahovich T, Davis R (2001) Sketch based interfaces: Early processing for sketch understanding. In: Proceedings of international conference on computer graphics and interactive techniques (ICCGIT’01), San Diego, California, USA, pp 37–44
Shilman M, Pasula H, Russell S, Newton R (2002) Statistical visual language models for ink parsing. In: AAAI spring 2002 symposium on sketch understanding
Thimbleby W (2004) A novel pen-based calculator and its evaluation. In: Proceedings of the third Nordic conference on human-computer interaction, Tempere, Finland, pp 445–448
Valois JP, Cote M, Cheriet M (2001) Online recognition of sketched electrical diagrams. In: Proceedings of the sixth international conference on document analysis and recognition (ICDAR’01), Seattle, USA, pp 460–464
Zadeh LA (1994) Soft computing and fuzzy logic. IEEE Softw 11(6):48–56
Zhao R (1993) Incremental recognition in gesture-based and syntax-directed diagram editors. In: Proceedings of the INTERACT ’93 and CHI ’93 conference on human factors in computing systems (CHI’93), Amsterdam, The Netherlands, pp 95–100