Evaluating unsupervised thesaurus-based labeling of audiovisual content in an archive production environment
Tóm tắt
In this paper we report on a two-stage evaluation of unsupervised labeling of audiovisual content using collateral text data sources to investigate how such an approach can provide acceptable results for given requirements with respect to archival quality, authority and service levels to external users. We conclude that with parameter settings that are optimized using a rigorous evaluation of precision and accuracy, the quality of automatic term-suggestion is sufficiently high. We furthermore provide an analysis of the term extraction after being taken into production, where we focus on performance variation with respect to term types and television programs. Having implemented the procedure in our production work-flow allows us to gradually develop the system further and to also assess the effect of the transformation from manual to automatic annotation from an end-user perspective. Additional future work will be on deploying different information sources including annotations based on multimodal video analysis such as speaker recognition and computer vision.
Tài liệu tham khảo
Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: A Nucleus for a Web of Open Data. Springer (2007)
Berners-Lee, T.: Linked data-design issues (2006)
Bizer, C., Heath, T., Berners-Lee, T.: Linked data—the story so far. Int. J. Semantic Web Inf. Syst. 5(3), 1–22 (2009). doi:10.4018/jswis.2009081901
de Boer, V., Hildebrand, M., Aroyo, L., De Leenheer, P., Dijkshoorn, C., Tesfa, B., Schreiber, G.: Nichesourcing: harnessing the power of crowds of experts. In: Knowledge Engineering and Knowledge Management, pp. 16–20. Springer (2012)
Bontcheva, K., Tablan, V., Maynard, D., Cunningham, H.: Evolving gate to meet new challenges in language engineering. Nat. Lang. Eng. 10, 349–373 (2004). doi:10.1017/S1351324904003468. http://journals.cambridge.org/article_S1351324904003468
Declerck, T., Kuper, J., Saggion, H., Samiotou, A., Wittenburg, P., Contreras, J.: Contribution of nlp to the content indexing of multimedia documents. In: Enser, P., Kompatsiaris, Y., Connor, N., Smeaton, A., Smeulders, A (eds.) Image and Video Retrieval, Lecture Notes in Computer Science, vol. 3115, pp. 610–618. Springer Berlin Heidelberg (2004). doi:10.1007/978-3-540-27814-6_71
Dietterich, T.G.: Ensemble methods in machine learning. In: Multiple classifier systems, pp. 1–15. Springer (2000)
Fellbaum, C.: WordNet. Wiley Online Library (1998)
Gazendam, L., Malaisé, V., Schreiber, G., Brugman, H., et al.: Deriving semantic annotations of an audiovisual program from contextual texts. In: Proceedings of First International workshop on Semantic Web Annotations for Multimedia (SWAMM 2006), vol 23 (2006)
Gazendam, L., Wartena, C., Malaisé, V., Schreiber, G., de Jong, A., Brugman, H.: Automatic annotation suggestions for audiovisual archives: Evaluation aspects. Interdisciplinary Sci. Rev. 34(2–3), 172–188 (2009). doi:10.1179/174327909X441090
Huurnink, B., Hollink, L., Van Den Heuvel, W., De Rijke, M.: Search behavior of media professionals at an audiovisual archive: a transaction log analysis. J. Am. Soc. Inf. Sci. Technol. 61(6), 1180–1197 (2010)
Iivonen, M.: Consistency in the selection of search concepts and search terms. Inf. Process. Manage. 31(2), 173–190 (1995)
Kobilarov, G., Scott, T., Raimond, Y., Oliver, S., Sizemore, C., Smethurst, M., Bizer, C., Lee, R.: Media meets semantic web—how the bbc uses dbpedia and linked data to make connections. In: The Semantic Web: Research and Applications, pp. 723–737. Springer (2009)
Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. (1932)
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The stanford corenlp natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)
Maynard, D., Ananiadou, S.: Acquiring contextual information for term disambiguation. In: Proc. of 1st Workshop Computational Terminology, Computerm98. Citeseer (1998)
Oomen, J., Ordelman, R.: Accessing audiovisual heritage: a roadmap for collaborative innovation. MultiMedia IEEE 18(4), 4–10 (2011)
Ordelman, R., Heeren, W., Huijbregts, M., de Jong, F., Hiemstra, D.: Towards affordable disclosure of spoken heritage archives. J. Digit. Inf. 10(6) (2009)
Schaffert, S., Bauer, C., Kurz, T., Dorschel, F., Glachs, D., Fernandez, M.: The linked media framework: Integrating and interlinking enterprise media content and data. In: Proceedings of the 8th International Conference on Semantic Systems, pp. 25–32. ACM (2012)
T. Tommasi, R. Aly, K. McGuinness, K. Chatfield, R. Arandjelovic, O. Parkhi, R. Ordelman, A. Zisserman, T.T.: Beyond metadata: searching your archive based on its audio-visual content. In: IBC 2014. Amsterdam, The Netherlands (2014). doi:10.1049/ib.2014.0003
Zhang, Y., Li, Y.: A user-centered functional metadata evaluation of moving image collections. J. Am. Soc. Inf. Sci. Technol. 59(8), 1331–1346 (2008)