Evaluation of SPARQL-compliant semantic search user interfaces
Tóm tắt
A regular user of a semantic search system frequently posses no knowledge about the SPARQL language nor about the ontology of a given knowledge base, especially when it provides domain-unspecific data obtained from heterogeneous sources. Nevertheless, he/she should be provided with tools enabling both intuitive and effective exploration of RDF-compliant knowledge bases. Natural language querying is one of the solutions that have been proposed so far as means for making knowledge bases more user-friendly. However, the results of natural language querying usually have lower precision and recall than analogical results of graph-based querying. In the paper, we introduce an evaluation methodology based on the 2011 QALD workshop queries that allows to measure the accuracy of a semantic search system as well as the complexity of the query formulation process. The obtained results confirm the intuition that graph-based querying, although assuring comparatively high accuracy of the results, is usually still too difficult for regular users. On the other hand, on the basis of results obtained for an experimental search system referred to as Semantic Focused Crawler, we claim that enhancing a SPARQL-compliant graph-based system by an entity-type recommendation feature may reduce the number of query elements necessary to formulate a query without compromising the quality of the results.
Tài liệu tham khảo
Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web. Sci. Am. 284(5), 34–43 (2001)
Bizer, C., Auer, S., Kobilarov, G., Lehmann, J., Becker, C., Hellmann, S.: DBpedia—querying wikipedia like a database and an interlinking-hub in the web of data. In: Querying Wikipedia Like a Database (4/4/2009) FU Berlin, Universitt Leipzig (2009)
Carroll, J., Dickinson, I., Dollin, C., Reynolds, D., Seaborne, A., Wilkinson, K.: Jena: implementing the semantic web recommendations. Tech. rep., Hewlett Packard (2003)
Damljanovic, D., Agatonovic, M., Cunningham, H.: Natural language interfaces to ontologies: Combining syntactic analysis and ontology-based lookup through the user interaction. In: Aroyo, L., Antoniou, G., Hyvnen, E., ten Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC (1). Lecture Notes in Computer Science, vol. 6088, pp. 106–120. Springer, Berlin (2010)
Elbedweihy, K., Wrigley, S.N., Ciravegna, F.: Evaluating semantic search query approaches with expert and casual users. In: Proceedings of the 11th International Conference on The Semantic Web (ISWC’12), vol. Part II, pp. 274–286. Springer, Berlin (2012)
Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. In: Losada, D.E., Fernndez-Luna, J.M. (eds.) ECIR. Lecture Notes in Computer Science, vol. 3408, pp. 345–359. Springer, Berlin (2005)
Han, L., Finin, T., Joshi, A.: GoRelations: an intuitive query system for DBpedia. In: Pan, J.Z., Chen, H., Kim, H.G., Li, J., Wu, Z., Horrocks, I., Mizoguchi, R., Wu, Z. (eds.) JIST. Lecture Notes in Computer Science, vol. 7185, pp. 334–341. Springer, Berlin (2011)
Kasneci, G., Suchanek, F.M., Ifrim, G., Ramanath, M., Weikum, G.: NAGA: searching and ranking knowledge. In: 24th International Conference on Data Engineering (ICDE’08). IEEE, IEEE Press, Cancun (2008)
Lopez, V., Fernndez, M., Motta, E., Stieler, N.: Poweraqua: supporting users in querying and exploring the semantic web. Semantic Web 3(3), 249–265 (2012)
Russell, A., Smart, P.R., Braines, D., Shadbolt, N.R.: Nitelight: A graphical tool for semantic query construction. In: Semantic Web User Interaction Workshop (SWUI’08) (2008), event Dates: 5th April 2008
Styperek, A., Ciesielczyk, M., Szwabe, A.: Sparql compliant semantic search engine with an intuitive user interface. In: Nguyen, N., Attachoo, B., Trawiski, B., Somboonviwat, K. (eds.) Intelligent Information and Database Systems. Lecture Notes in Computer Science, vol. 8397, pp. 201–210. Springer International Publishing, NY (2014)
Suchanek, F.M., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge unifying WordNet and Wikipedia. In: Proceedings of the 16th International World Wide Web Conference (WWW’07), pp. 697–706. Banff (2007)
The Apache Software Foundation: Apache Lucene Core. http://lucene.apache.org/core/ (2014). Accessed 5 Apr 2015
Tran, T., Math, T., Haase, P.: Usability of keyword-driven schema-agnostic search. In: Aroyo, L., Antoniou, G., Hyvnen, E., Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) The Semantic Web: Research and Applications. Lecture Notes in Computer Science, vol. 6089, pp. 349–364. Springer, Berlin (2010)
Wang, C., Xiong, M., Zhou, Q., Yu, Y.: Panto: a portable natural language interface to ontologies. In: Franconi, E., Kifer, M., May, W. (eds.) ESWC. Lecture Notes in Computer Science, vol. 4519, pp. 473–487. Springer, Berlin (2007)
Wrigley, S.N., Reinhard, D., Elbedweihy, K., Bernstein, A., Ciravegna, F.: Methodology and campaign design for the evaluation of semantic search tools. In: Proceedings of the 3rd International Semantic Search Workshop (SEMSEARCH’10), pp. 10:1–10:10. ACM, New York (2010)
Zhou, Q., Wang, C., Xiong, M., Wang, H., Yu, Y.: Spark: adapting keyword query to semantic search. In: Aberer, K., Choi, K.S., Noy, N., Allemang, D., Lee, K.I., Nixon, L., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., Cudr-Mauroux, P. (eds.) The Semantic Web. Lecture Notes in Computer Science, vol. 4825, pp. 694–707. Springer, Berlin (2007)