Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation

R Costanza, 1998, The value of the world’s ecosystem services and natural capital, Ecol Econ, 25, 3, 10.1016/S0921-8009(98)00020-2

Cesar H, Burke L, Pet-Soede L. The economics of worldwide coral reef degradation. Cesar Environ Econ Consult. 2003;

O Hoegh-Guldberg, 2007, Coral reefs under rapid climate change and ocean acidification, Science, 318, 1737, 10.1126/science.1152509

TA Gardner, 2003, Long-term region-wide declines in Caribbean corals, Science, 301, 958, 10.1126/science.1086050

JF Bruno, 2007, Regional Decline of Coral Cover in the Indo-Pacific: Timing, Extent, and Subregional Comparisons, PLoS One, 2, e711, 10.1371/journal.pone.0000711

R Van Hooidonk, 2013, Temporary refugia for coral reefs in a warming world, Nat Clim Change, 3, 508, 10.1038/nclimate1829

TP Hughes, 2003, Climate change, human impacts, and the resilience of coral reefs, Science, 301, 929, 10.1126/science.1085046

DR Bellwood, 2004, Confronting the coral reef crisis, Nature, 429, 827, 10.1038/nature02691

JS Jaffe, 2001, Underwater optical imaging: status and prospects, Oceanogr, 14, 66, 10.5670/oceanog.2001.24

E Pante, 2012, Getting to the Point: Accuracy of Point Count in Monitoring Ecosystem Change, J Mar Biol, 2012, 10.1155/2012/802875

ACR Gleason, 2010, Landscape video mosaic from a mesophotic coral reef, Coral Reefs, 29, 253, 10.1007/s00338-009-0544-2

JH Carleton, 1995, Quantitative video sampling of coral reef benthos: large-scale application, Coral Reefs, 14, 35, 10.1007/BF00304070

R Ninio, 2003, Estimating cover of benthic organisms from underwater video images: variability associated with multiple observers, Mar Ecol Prog Ser, 265, 107, 10.3354/meps265107

H Singh, 2004, Imaging coral I: imaging coral habitats with the SeaBED AUV, Subsurf Sens Technol Appl, 5, 25, 10.1023/B:SSTA.0000018445.25977.f3

JW Nicholson, 2008, The present state of autonomous underwater vehicle (AUV) applications and technologies, J Mar Technol Society, 42, 44, 10.4031/002533208786861272

A Eleftheriou, 2013

M González-Rivero, 2014, The Catlin Seaview Survey–kilometre-scale seascape assessment, and monitoring of coral reef ecosystems, Aquat Conserv, 24, 184, 10.1002/aqc.2505

PP Molloy, 2013, How much sampling does it take to detect trends in coral-reef habitat using photoquadrat surveys?, Aquat Conserv, 23, 820, 10.1002/aqc.2372

KE Kohler, 2006, Coral Point Count with Excel extensions (CPCe): A Visual Basic program for the determination of coral and substrate coverage using random point count methodology, Comput Geosci, 32, 1259, 10.1016/j.cageo.2005.11.009

V Trygonis, 2012, photoQuad: A dedicated seabed image processing software, and a comparative error analysis of four photoquadrat methods, J Exp Mar Bio Ecol, 424, 99, 10.1016/j.jembe.2012.04.018

Porter JW, Kosmynin V, Patterson KL, Porter KG, Jaap WC, Wheaton JL, et al. Detection of coral reef change by the Florida Keys coral reef monitoring project. The Everglades, Florida Bay, and coral reefs of the Florida Keys: An ecosystem sourcebook. 2002; 749–769.

MSA Marcos, 2005, Classification of coral reef images from underwater video using neural networks, Optics Express, 13, 8766, 10.1364/OPEX.13.008766

MD Stokes, 2009, Automated processing of coral reef benthic images, Limnol Oceanogr Meth, 7, 157, 10.4319/lom.2009.7.157

O Beijbom, 2012, Automated annotation of coral reef survey images, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 1, 1170

Bewley M, Douillard B, Nourani-Vatani N, Friedman A, Pizarro O, Williams S. Automated species detection: An experimental approach to kelp detection from sea-floor AUV images. Proc Australas Conf Rob Autom. 2012;

ASM Shihavuddin, 2013, Image-Based Coral Reef Classification and Thematic Mapping, Remote Sens, 5, 1809, 10.3390/rs5041809

SR Phinn, 2012, Multi-scale, object-based image analysis for mapping geomorphic and ecological zones on coral reefs, Int J Remote Sens, 33, 3768, 10.1080/01431161.2011.633122

R-E Fan, 2008, LIBLINEAR: A library for large linear classification, J Mach Learn Res, 9, 1871

F Perronnin, 2007, Fisher kernels on visual vocabularies for image categorization, Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, 1, 1

Chatfield K, Lempitsky V, Vedaldi A, Zisserman A. The devil is in the details: an evaluation of recent feature encoding methods. Proc Br Mach Vis Conf. 2011;

Branson S, Beijbom O, Belongie S. Efficient large-scale structured learning. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2013. pp. 1806–1813.

J Carletta, 1996, Assessing agreement on classification tasks: the kappa statistic, Comput Linguist, 22, 249

R Rifkin, 2004, In defense of one-vs-all classification, J Mach Learn Res, 5, 101

DD Lewis, 1994, Heterogenous Uncertainty Sampling for Supervised Learning, Proc Int Conf Mach Learn, 94, 148

A Solow, 2001, Estimating the taxonomic composition of a sample when individuals are classified with error, Mar Ecol Prog Ser, 216, 309, 10.3354/meps216309

DJ Hopkins, 2010, A method of automated nonparametric content analysis for social science, Am J Pol Sci, 54, 229, 10.1111/j.1540-5907.2009.00428.x

P Good, 2005, parametric and bootstrap tests of hypotheses

O Allouche, 2006, Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS), J Appl Ecol, 43, 1223, 10.1111/j.1365-2664.2006.01214.x

S Manel, 2001, Evaluating presence–absence models in ecology: the need to account for prevalence, J Appl Ecol, 38, 921, 10.1046/j.1365-2664.2001.00647.x

RA Monserud, 1992, Comparing global vegetation maps with the Kappa statistic, Ecol Model, 62, 275, 10.1016/0304-3800(92)90003-W

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 159–174.

JL Fleiss, 1981, The measurement of interrater agreement, Stat Methods Rates Prop, 2, 212

I Miller, 1999, Validity and reproducibility of benthic cover estimates made during broadscale surveys of coral reefs by manta tow, Coral Reefs, 18, 353, 10.1007/s003380050212

DG Zawada, 2014, Fluorescence-Based Classification of Caribbean Coral Reef Organisms and Substrates, PLoS One, 9, e84570, 10.1371/journal.pone.0084570

T Treibitz, 2015, Wide Field-of-View Fluorescence Imaging of Coral Reefs, Sci Rep, 5, 10.1038/srep07694

EJ Buskey, 2006, Use of the FlowCAM for semi-automated recognition and enumeration of red tide cells (Karenia brevis) in natural plankton samples, Harmful Algae, 5, 685, 10.1016/j.hal.2006.02.003

HM Sosik, 2007, Automated taxonomic classification of phytoplankton sampled with imaging-in-flow cytometry, Limnol Oceanogr Methods, 5, 204, 10.4319/lom.2007.5.204

RJ Olson, 2007, A submersible imaging-in-flow instrument to analyze nano-and microplankton: Imaging FlowCytobot, Limnol Oceanogr Methods, 5, 195, 10.4319/lom.2007.5.195

Beijbom O. Cost-Effective Sampling for Pairs of Annotators. arXiv preprint arXiv:14107074. 2014;

SJ Pan, 2010, A survey on transfer learning, IEEE Trans Knowl Data Eng, 22, 1345, 10.1109/TKDE.2009.191

Beijbom O. Domain Adaptations for Computer Vision Applications. arXiv preprint arXiv:12114860. 2012;