Data challenges of time domain astronomy
Tóm tắt
Astronomy has been at the forefront of the development of the techniques and methodologies of data intensive science for over a decade with large sky surveys and distributed efforts such as the Virtual Observatory. However, it faces a new data deluge with the next generation of synoptic sky surveys which are opening up the time domain for discovery and exploration. This brings both new scientific opportunities and fresh challenges, in terms of data rates from robotic telescopes and exponential complexity in linked data, but also for data mining algorithms used in classification and decision making. In this paper, we describe how an informatics-based approach—part of the so-called “fourth paradigm” of scientific discovery—is emerging to deal with these. We review our experiences with the Palomar-Quest and Catalina Real-Time Transient Sky Surveys; in particular, addressing the issue of the heterogeneity of data associated with transient astronomical events (and other sensor networks) and how to manage and analyze it.
Tài liệu tham khảo
Gray, J., Szalay, A.: 2020 computing: science in an exponential world. Nature 440, 23 (2006)
York, D.G., et al. (the SDSS team): The Sloan Digital Sky Survey: technical summary. Astron. J. 120, 1579 (2000)
Djorgovski, S.G., Gal, R., Odewahn, S., de Carvalho, R., Brunner, R., Longo, G., Scaramella, R.: The Palomar Digital Sky Survey (DPOSS). In: Wide Field Surveys in Cosmology, p. 89. Editions Frontieres, Gif sur Yvette (1998)
Skrutskie, M., et al. (the 2MASS team): The Two Micron All Sky Survey (2MASS). Astron. J. 131, 1163 (2006)
Djorgovski, S.G., Mahabal, A.A., Drake, A.J., Graham, M.J., Donalek, C.: Sky Surveys, Planets, Stars, and Stellar Systems. Springer, Berlin (2012, in press)
Brunner, R.J., Djorgovski, S.G., Szalay, A.S.: Virtual Observatories of the Future. Astronomical Society of the Pacific, San Francisco (2001)
National Research Council: New Worlds, New Horizons in Astronomy and Astrophysics, Decadal Survey of Astronomy and Astrophysics Comm. The National Academies Press, Washington (2010)
Large Synoptic Sky Survey. http://lsst.org
Square Kilometer Array. http://skatelescope.org
Drake, A.J., et al.: First results from the Catalina Real-Time Transient Survey. Astrophys. J. 696, 870 (2009)
Djorgovski, S.G., et al.: The Catalina Real-Time Transient Survey (CRTS), The First Year of MAXI: Monitoring Variable X-ray Sources. JAXA Special Publ., Tokyo (2011)
Mahabal, A.A., et al.: Discovery, classification, and scientific exploration of transient events from the Catalina Real-Time Transient Survey. Bull. Astron. Soc. India 39, 387–408 (2011)
Djorgovski, S.G., Mahabal, A., Drake, A., Graham, M., Donalek, C., Williams, R.: Exploring the time domain with Synoptic Sky Surveys. In: Proc. IAU Symp. 285, New Horizons in Time Domain Astronomy, p. 141. Cambridge University Press, Cambridge (2012)
Djorgovski, S.G., et al.: Some pattern recognition challenges in data-intensive astronomy. In: Proc. 18th Intl. Conf. on Pattern Recognition, p. 856. IEEE Press, New York (2006)
Djorgovski, S.G., Donalek, C., Mahabal, A., Moghaddam, B., Turmon, M., Graham, M., Drake, A., Sharma, N., Chen, Y.: Towards an automated classification of transient events in Synoptic Sky Surveys. In: Proc. CIDU 2011 Conf. (ASA), p. 174 (2011)
Donalek, C., Mahabal, A., Djorgovski, S.G., Marney, S., Drake, A., Graham, M., Glikman, E., Williams, R.: New approaches to object classification in Synoptic Sky Surveys. In: AIP Conf. Proc., vol. 1082, p. 252 (2008)
SciDB. http://www.scidb.org
Cornwell, T.J., van Diepen, G.: Scaling mount exaflop: from the pathfinders to the square kilometre array. In: Proc. SPIE, vol. 7016 (2008)
International Virtual Observatory Alliance. http://www.ivoa.org
Ochsenbein, F., et al.: IVOA recommendation: VOTable format definition version 1.2 (2011). arXiv:1110.0524
Graham, M.J., Morris, D., Rixon, G.: IVOA recommendation: VOSpace specification version 1.15 (2011). arXiv:1110.0508
Seaman, R., et al.: IVOA recommendation: sky event reporting metadata version 2.0 (2011). arXiv:1110.0523
Djorgovski, S.G., et al.: The Palomar-Quest Digital Synoptic Sky Survey. Astron. Nachr. 329, 263 (2008)
Williams, R.D., Djorgovski, S.G., Drake, A.J., Graham, M.J., Mahabal, A.: Skyalert: real-time astronomy for you and your robots. In: Astronomical Data Analysis Software and Systems XVIII, p. 115. Astronomical Society of the Pacific, San Francisco (2009)
Philip, N.S., Mahabal, A., Abraham, S., Williams, R., Djorgovski, S.G., Drake, A., Donald, C., Graham, M.: Classification by boosting differences in input vectors. In: Proc. International Workshop on Stellar Spectra Libraries, Ast. Soc. of India Conf. Ser. (ASICS) (2012, in press)
Budavari, T.: Probabilistic cross-identification of cosmic events. Astrophys. J. 736, 155–159 (2011)
Palomar-Quest Synoptic Sky Survey data release 1. http://www.astro.caltech.edu/~aam/pqdr1/
Kunszt, P.Z., Szalay, A.S., Thakar, A.R.: The hierarchical triangular mesh, mining the sky. In: Proceedings of the MPA/ESO/MPE Workshop Held at Garching, Germany, July 31–August 4 2000, p. 631. Springer, Berlin (2001).
Gorski, K.M., et al.: HEALPix: a framework for high-resolution discretization and fast analysis of data distributed on the sphere. Astrophys. J. 622, 759 (2005)
Gray, J., Nieto-Santisteban, M., Szalay, A.S.: The Zones algorithm for finding points-near-a-point or cross-matching spatial datasets (2006). arXiv:cs/0701171
Koposov, S., Bartunov, O.: Q3C, Quad Tree Cube—the new sky-indexing concept for huge astronomical catalogues and its realization for main astronomical queries (cone search and Xmatch) in open source database PostgreSQL. In: Astronomical Data Analysis Software and Systems XV, p. 735. Astronomical Society of the Pacific, San Francisco (2006)
Richards, J.W., et al.: On machine-learned classification of variable stars with sparse and noisy time-series data. Astrophys. J. 733, 10 (2011)
Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2012)
Graczyk, D., Eyer, L.: The light curve statistical moments analysis: the identification of eclipsing binaries. Acta Astron. 60, 109 (2010)
Edelson, R.A., Krolik, J.H.: The discrete correlation function—a new method for analyzing unevenly sampled variability data. Astrophys. J. 333, 646 (1988)
Simonetti, J.H., Cordes, J.M., Heeschen, D.S.: Flicker of extragalactic radio sources at two frequencies. Astrophys. J. 296, 46 (1985)
Jackson, B., et al.: An algorithm for the optimal partitioning of data on an interval. IEEE Signal Process. Lett. 12, 105 (2005)