A system for identifying named entities in biomedical text: how results from two evaluations reflect on both the system and the evaluations
Tóm tắt
Abstract We present a maximum entropy‐based system for identifying named entities (NEs) in biomedical abstracts and present its performance in the only two biomedical named entity recognition (NER) comparative evaluations that have been held to date, namely BioCreative and Coling BioNLP. Our system obtained an exact match F‐score of 83.2% in the BioCreative evaluation and 70.1% in the BioNLP evaluation. We discuss our system in detail, including its rich use of local features, attention to correct boundary identification, innovative use of external knowledge resources, including parsing and web searches, and rapid adaptation to new NE sets. We also discuss in depth problems with data annotation in the evaluations which caused the final performance to be lower than optimal. Copyright © 2005 John Wiley & Sons, Ltd.
Từ khóa
#automatic named entity recognition #manual annotation of biomedical textsTài liệu tham khảo
Blaschke C, Hirschman L, Yeh A (eds). 2004. Proceedings of the BioCreative Workshop, Granada; http://www.pdg.cnb.uam.es/BioLINK/workshop_BioCreative_04/handout/.
Brants T. 2000. TnT—a statistical part‐of‐speech tagger. In Proceedings of the Sixth Applied Natural Language Processing Conference ANLP‐2000, Seattle, WA 6: 224–231.
Collier N, Kim J, Tateisi Y, Ohta T, Tsuruoka Y (eds). 2004. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, Geneva, Switzerland. http://www.genesis.ch/∼natlang/JNLPBAO4.
Collier N, Nobata C, Tsujii J. 2000. Extracting the names of genes and gene products with a hidden Markov model. In Proceedings of the 18th International Conference on Computational Linguistics (Coling 2000), Saarbruecken, Germany; pp. 201–207.
Curran JR, Clark S. 2003. Language independent NER using a maximum entropy tagger. In Proceedings of the 7th Conference on Natural Language Learning (CoNLL‐03), Edmonton, Canada; 164–167.
Demetriou G, Gaizauskas R. 2003. Corpus resources for development and evaluation of a biological text mining system. In Proceedings of the Third Meeting of the Special Interest Group on Text Mining, Brisbane, Australia; http://www.pdg.cnb.uam.es/BioLink/SpecialInterestTextMining/PRESENTATIONS/rob_g.ppt.
Dingare S, Finkel J, Nissim M, Manning C, Alex B. 2004. Exploring the boundaries: Gene and protein identification in biomedical text. In Proceedings of the BioCreative Workshop, Granada; http://www.pdg.cnb.uam.es/BioLINK/workshop_BioCrea‐tive_04/handout/.
Finkel J, Dingare S, Nguyen H, Nissim M, Manning C. 2004. From syntax to the web. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, at CoLing 2004, Geneva, Switzerland; pp. 88–91.
Fukuda K. 1998. Toward information extraction: Identifying protein names from biological papers. In Proceedings of the Pacific Symposium on Biocomputing 705–716.
Grefenstette G. 1999. TheWWW as a resource for example‐based MT tasks. In Proceedings of ASLIB'99, Translating and the Computer 21, London.
Hirschman L. 2003. Using biological resources to bootstrap text mining. Presentation to the Massachusetts Biotechnology Council Informatics Committee; http://www.e‐biosci.org/sept/Hirs‐chman.pdf.
Kazama J, Makino T, Ohta Y, Tsujii J. 2002. Biomedical name recognition: Tuning support vector machines for biomedical named entity recognition. In Proceedings of the ACL 2002 Workshop on Natural Language Processing in the Biomedical Domain 1–8.
Keller F, Lapata M. 2003. Using the web to obtain frequencies for unseen bigrams. Comput Linguist 29(3): 459–484.
Kilgarriff A. 1997. Putting frequencies in the dictionary. Int J Lexicogr 10(2): 135–155.
Klein D, Manning C. 2003. Accurate unlexicalized parsing. Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL, 2003), Sapporo, Japan, pp. 423–430.
Klein D, Smarr J, Nguyen H, Manning CD. 2003. Named entity recognition with character‐level models. Proceedings of the 7th Conference on Natural Language Learning (CoNLL 2003), Edmonton, Canada; pp. 180–183.
Koichi T, Collier N. 2003. Bio‐medical entity extraction using support vector machines. In Proceedings of the Workshop on Natural Language Processing in Biomedicine, held as part of the 41st Annual Meeting of the Association for Computational Linguistics (ACL, 2003), Sappoco, Japan, July 7–12, 2003.
Markert K, Nissim M, Modjeska N. 2003. Using the web for nominal anaphora resolution. In Proceedings of the Workshop on the Computational Treatment of Anaphora held as part of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL, 2003), Budapest, Hungary, April 12–17, 2003. Dale R, van Deemter K, Mitkov R (eds) 39–46.
Marsh E, Perzanowski D. 1998. MUC‐7 evaluation of IE tech‐nology: Overview of results. In Message Understanding Conference Proceedings 7; http://www.itl.nist.gov/iaui/894.02/related_projects/muc/proceedings/muc_7_proceedings/marsh_slides.pdf.
McCallum A, Freitag D, Pereira F. 2000. Maximum entropy markov models for information extraction and segmentation. In Proceedings of the 17th International Conference on Machine Learning.
Mikheev A, Moens M, Grover C. 1999. Named entity recognition without gazetteers. In Proceedings of the 9th Conference of the European Chapter of the Association for Computational Linguistics (EACL '99), Bergen, Norway, June 8–12, 1999, pp. 1–8.
Nobata C, Collier N, Tsujii J. 1999. Automatic term identification and classification in biology texts. In Proceedings of the 5th Natural Language Processing Pacific Rim Symposium (NLPRS '99), Beijing, China, November 5–7, 1999; pp. 369–374.
Ohta T, Tateisi Y, Mima H, Tsujii J. 2002. GENIA corpus: an annotated research abstract corpus in molecular biology domain. In Proceedings of the Human Language Technology Conference, San Diego, CA, USA, March 24–27, 2002.
Sang EFTK, De Meulder F. 2003. Introduction to the CoNLL‐2003 shared task: language‐independent named entity recognition. In Proceedings of CoNLL‐2003 142–147.
Schwartz A, Hearst M. 2003. A simple algorithm for identifying abbreviation definitions in biomedical text. In Pacific Symposium on Biocomputing, Kauai.
Shen D, Zhang J, Su GZJ, CTan. 2003. Effective adaptation of hidden Markov model‐based named entity recognizer for biomedical domain. In Proceedings of the Workshop on Natural Language Processing in Biomedicine, held as part of the 41st Annual Meeting of the Association for Computational Linguistics (ACL, 2003), Sapporo, Japan, July 7–12, 2003.