The LOCKSS peer-to-peer digital preservation system

ACM Transactions on Computer Systems - Tập 23 Số 1 - Trang 2-50 - 2005
Petros Maniatis1, Mema Roussopoulos2, TJ Giuli3, David S. H. Rosenthal4, Mary Baker5
1Intel Res., Berkeley, CA#TAB#
2Harvard University, Cambridge, Ma.#TAB#
3Stanford University, Stanford, CA
4Stanford University Libraries, Palo Alto, CA#TAB#
5HP Labs., Palo Alto, CA#TAB#

Tóm tắt

The LOCKSS project has developed and deployed in a world-wide test a peer-to-peer system for preserving access to journals and other archival information published on the Web. It consists of a large number of independent, low-cost, persistent Web caches that cooperate to detect and repair damage to their content by voting in “opinion polls.” Based on this experience, we present a design for and simulations of a novel protocol for voting in systems of this kind. It incorporates rate limitation and intrusion detection to ensure that even some very powerful adversaries attacking over many years have only a small probability of causing irrecoverable damage before being detected.

Từ khóa


Tài liệu tham khảo

Abadi , M. , Burrows , M. , Manasse , M. , and Wobber , T . 2003. Moderately hard, memory-bound functions . In Proceedings of the 10th Annual Network and Distributed System Security Symposium ( San Diego, CA).]] Abadi, M., Burrows, M., Manasse, M., and Wobber, T. 2003. Moderately hard, memory-bound functions. In Proceedings of the 10th Annual Network and Distributed System Security Symposium (San Diego, CA).]]

Anderson , R. J. 1996 . The Eternity Service . In Proceedings of the 1st International Conference on the Theory and Applications of Cryptology (PRAGOCRYPT 1996, Prague, Czech Republic).]] Anderson, R. J. 1996. The Eternity Service. In Proceedings of the 1st International Conference on the Theory and Applications of Cryptology (PRAGOCRYPT 1996, Prague, Czech Republic).]]

ARL---Association of Research Libraries. 2001. ARL Statistics 2000-01. Available online at http://www.arl.org/stats/arlstat/01pub/intro.html.]] ARL---Association of Research Libraries. 2001. ARL Statistics 2000-01. Available online at http://www.arl.org/stats/arlstat/01pub/intro.html.]]

Back A. 2002. Hashcash---a denial of service counter measure. Available online at http://www.hashcash.org/hashcash.pdf.]] Back A. 2002. Hashcash---a denial of service counter measure. Available online at http://www.hashcash.org/hashcash.pdf.]]

10.1145/312203.312207

Castro , M. and Liskov , B . 1999. Practical Byzantine fault tolerance . In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation. USENIX Association ( New Orleans, LA). 173--186.]] Castro, M. and Liskov, B. 1999. Practical Byzantine fault tolerance. In Proceedings of the 3rd Symposium on Operating Systems Design and Implementation. USENIX Association (New Orleans, LA). 173--186.]]

Chen , Y. , Edler , J. , Goldberg , A. , Gottlieb , A. , Sobti , S. , and Yianilos , P . 1999. A prototype implementation of archival intermemory . In Proceedings of the International Conference on Digital Libraries ( Berkeley, CA). 28--37.]] 10.1145/313238.313249 Chen, Y., Edler, J., Goldberg, A., Gottlieb, A., Sobti, S., and Yianilos, P. 1999. A prototype implementation of archival intermemory. In Proceedings of the International Conference on Digital Libraries (Berkeley, CA). 28--37.]] 10.1145/313238.313249

Chor , B. and Dwork , C. 1989 . Randomization in Byzantine agreement . Advan. Comput. Res. 5 , 443 -- 497 .]] Chor, B. and Dwork, C. 1989. Randomization in Byzantine agreement. Advan. Comput. Res. 5, 443--497.]]

Clarke , I. , Sandberg , O. , Wiley, B. , and Hong, T. W. 2000 . Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability , H. Federrath, Ed. Lecture Notes in Computer Science, vol. 2009 . Springer , Berlin, Germany, 46--66.]] Clarke, I., Sandberg, O., Wiley, B., and Hong, T. W. 2000. Freenet: A distributed anonymous information storage and retrieval system. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability, H. Federrath, Ed. Lecture Notes in Computer Science, vol. 2009. Springer, Berlin, Germany, 46--66.]]

ConservationOnline. 2003. Electronic storage media. Available online at http://palimpsest.stanford.edu/bytopic/electronic-records/electronic-storage-media/.]] ConservationOnline. 2003. Electronic storage media. Available online at http://palimpsest.stanford.edu/bytopic/electronic-records/electronic-storage-media/.]]

Dabek , F. , Kaashoek , M. F. , Karger , D. , Morris , R. , and Stoica , I . 2001. Wide-area Cooperative storage with CFS . In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (Chateau Lake Louise, Banff, AB, Canada). 202--215 .]] 10.1145/50 2034 .502054 Dabek, F., Kaashoek, M. F., Karger, D., Morris, R., and Stoica, I. 2001. Wide-area Cooperative storage with CFS. In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (Chateau Lake Louise, Banff, AB, Canada). 202--215.]] 10.1145/502034.502054

Davis , D. 1996 . Compliance defects in public-key cryptography . In Proceedings of the 6th USENIX Security Symposium ( San Jose, CA). 171--178.]] Davis, D. 1996. Compliance defects in public-key cryptography. In Proceedings of the 6th USENIX Security Symposium (San Jose, CA). 171--178.]]

Dean , D. and Stubblefield , A . 2001. Using client puzzles to protect TLS . In Proceedings of the 10th USENIX Security Symposium ( Washington, DC).]] Dean, D. and Stubblefield, A. 2001. Using client puzzles to protect TLS. In Proceedings of the 10th USENIX Security Symposium (Washington, DC).]]

Diffie , W. 2003 . Perspective: Decrypting the secret to strong security. Available online at http://news.com.com/2010-1071-980462.html.]] Diffie, W. 2003. Perspective: Decrypting the secret to strong security. Available online at http://news.com.com/2010-1071-980462.html.]]

10.1109/TIT.1976.1055638

Dingledine , R. , Freedman , M. J. , and Molnar , D . 2000. The Free Haven Project: Distributed anonymous storage service . In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability, H. Federrath, Ed. Lecture Notes in Computer Science , vol. 2009 . Springer, Berlin, Germany, 67--95.]] Dingledine, R., Freedman, M. J., and Molnar, D. 2000. The Free Haven Project: Distributed anonymous storage service. In Proceedings of the Workshop on Design Issues in Anonymity and Unobservability, H. Federrath, Ed. Lecture Notes in Computer Science, vol. 2009. Springer, Berlin, Germany, 67--95.]]

Douceur , J. 2002 . The Sybil attack . In Proceedings of the 1st International Workshop on Peer-to-Peer Systems ( Boston, MA). 251--260.]] Douceur, J. 2002. The Sybil attack. In Proceedings of the 1st International Workshop on Peer-to-Peer Systems (Boston, MA). 251--260.]]

Dwork , C. , Goldberg , A. , and Naor , M . 2003. On memory-bound functions for fighting spam . In Proceedings of the 23rd Annual International Cryptology Conference ( Santa Barbara, CA).]] Dwork, C., Goldberg, A., and Naor, M. 2003. On memory-bound functions for fighting spam. In Proceedings of the 23rd Annual International Cryptology Conference (Santa Barbara, CA).]]

Dwork , C. and Naor , M . 1992. Pricing via processing . In Proceedings of the 12nd Annual International Cryptology Conference ( Santa Barbara, CA). 139--147.]] Dwork, C. and Naor, M. 1992. Pricing via processing. In Proceedings of the 12nd Annual International Cryptology Conference (Santa Barbara, CA). 139--147.]]

Erdos M. and Cantor S. 2002. Shibboleth Architecture DRAFT v05. Available online at http://shibboleth.internet2.edu/docs/draft-internet2-shibboleth-arch-v05.pdf. Work in progress.]] Erdos M. and Cantor S. 2002. Shibboleth Architecture DRAFT v05. Available online at http://shibboleth.internet2.edu/docs/draft-internet2-shibboleth-arch-v05.pdf. Work in progress.]]

10.1162/105864001300122476

Fullmer C. 2002. Storage and multimedia: The facts and more. Available online at http://www.cse.ucsc.edu/classes/cmpe003/Fall02/L11_ch6.pps.]] Fullmer C. 2002. Storage and multimedia: The facts and more. Available online at http://www.cse.ucsc.edu/classes/cmpe003/Fall02/L11_ch6.pps.]]

Giuli , T. and Baker , M . 2002 . Narses: A scalable, flow-based network simulator. Tech. rep. arXiv:cs.PF/0211024. Computer Science Department , Stanford University , Stanford, CA .]] Giuli, T. and Baker, M. 2002. Narses: A scalable, flow-based network simulator. Tech. rep. arXiv:cs.PF/0211024. Computer Science Department, Stanford University, Stanford, CA.]]

Horlings J. 2003. Cd-r's binnen twee jaar onleesbaar. Available online at http://www.pc-active.nl/toonArtikel.asp?artikelID=508. http://www.cdfreaks.com/news/7751.]] Horlings J. 2003. Cd-r's binnen twee jaar onleesbaar. Available online at http://www.pc-active.nl/toonArtikel.asp?artikelID=508. http://www.cdfreaks.com/news/7751.]]

10.5210/fm.v8i5.1053

Kubiatowicz , J. , Bindel , D. , Chen , Y. , Czerwinski , S. , Eaton , P. , Geels , D. , Gummadi , R. , Rhea , S. , Weatherspoon , H. , Weimer , W. , Wells , C. , and Zhao , B . 2000. OceanStore: An architecture for global-scale persistent storage . In Proceedings of the 9th International Conference on Architectural Support for Programming Languages and Operating Systems ( Cambridge, MA). 190--201.]] 10.1145/378993.379239 Kubiatowicz, J., Bindel, D., Chen, Y., Czerwinski, S., Eaton, P., Geels, D., Gummadi, R., Rhea, S., Weatherspoon, H., Weimer, W., Wells, C., and Zhao, B. 2000. OceanStore: An architecture for global-scale persistent storage. In Proceedings of the 9th International Conference on Architectural Support for Programming Languages and Operating Systems (Cambridge, MA). 190--201.]] 10.1145/378993.379239

10.1007/s004460050050

Maniatis , P. , Giuli , T. , Roussopoulos , M. , Rosenthal , D. S. H. , and Baker , M . 2004. Impeding attrition attacks in P2P systems . In Proceedings of the 11th ACM SIGOPS European Workshop ( Leuven, Belgium).]] 10.1145/1133572.1133601 Maniatis, P., Giuli, T., Roussopoulos, M., Rosenthal, D. S. H., and Baker, M. 2004. Impeding attrition attacks in P2P systems. In Proceedings of the 11th ACM SIGOPS European Workshop (Leuven, Belgium).]] 10.1145/1133572.1133601

Maniatis , P. , Roussopoulos , M. , Giuli , T. , Rosenthal , D. S. H. , Baker , M. , and Muliadi , Y . 2003. Preserving peer replicas by rate-limited sampled voting . In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles ( Bolton Landing, NY). 44--59.]] 10.1145/945445.945451 Maniatis, P., Roussopoulos, M., Giuli, T., Rosenthal, D. S. H., Baker, M., and Muliadi, Y. 2003. Preserving peer replicas by rate-limited sampled voting. In Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles (Bolton Landing, NY). 44--59.]] 10.1145/945445.945451

Manns B. and Shahani C. J. 2003. Longevity of CD media research at the Library of Congress. Available online at http://www.loc.gov/preserv/study\%20of\%20CD\%20longevity.pdf.]] Manns B. and Shahani C. J. 2003. Longevity of CD media research at the Library of Congress. Available online at http://www.loc.gov/preserv/study\%20of\%20CD\%20longevity.pdf.]]

Michalakis , N. , Chiu , D.-M. , and Rosenthal , D. S. H. 2003. Long term data resilience using opinion polls . In Proceedings of the 22nd IEEE International Performance Computing and Communications Conference ( Phoenix, AZ).]] Michalakis, N., Chiu, D.-M., and Rosenthal, D. S. H. 2003. Long term data resilience using opinion polls. In Proceedings of the 22nd IEEE International Performance Computing and Communications Conference (Phoenix, AZ).]]

10.1108/07378839910267154

Moore , D. , Voelker , G. M. , and Savage , S . 2001. Inferring Internet denial-of-service activity . In Proceedings of the 10th USENIX Security Symposium.]] Moore, D., Voelker, G. M., and Savage, S. 2001. Inferring Internet denial-of-service activity. In Proceedings of the 10th USENIX Security Symposium.]]

Patterson , D. A. , Gibson , G. , and Katz , R. H . 1988. A case for redundant arrays of inexpensive disks (RAID) . In Proceedings of the ACM SIGMOD International Conference on Management of Data ( Chicago, IL). 109--116.]] 10.1145/50202.50214 Patterson, D. A., Gibson, G., and Katz, R. H. 1988. A case for redundant arrays of inexpensive disks (RAID). In Proceedings of the ACM SIGMOD International Conference on Management of Data (Chicago, IL). 109--116.]] 10.1145/50202.50214

Quirke C. 2002. Hard drive data corruption. Available online at http://users.iafrica.com/c/cq/cquirke/baddata.htm.]] Quirke C. 2002. Hard drive data corruption. Available online at http://users.iafrica.com/c/cq/cquirke/baddata.htm.]]

Reiter , M. 1994 . Secure agreement protocols: Reliable and atomic group multicast in Rampart . In Proceedings of the 2nd ACM Conference on Computer and Communications Security ( Fairfax, VA). 68--80.]] 10.1145/ 191177.191194 Reiter, M. 1994. Secure agreement protocols: Reliable and atomic group multicast in Rampart. In Proceedings of the 2nd ACM Conference on Computer and Communications Security (Fairfax, VA). 68--80.]] 10.1145/191177.191194

Rhea , S. , Geels , D. , Roscoe , T. , and Kubiatowicz , J . 2004. Handling Churn in a DHT . In Proceedings of the Usenix Annual Technical Conference ( Boston, MA).]] Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2004. Handling Churn in a DHT. In Proceedings of the Usenix Annual Technical Conference (Boston, MA).]]

Rosenthal , D. S. H. 2003 . A digital preservation network appliance based on OpenBSD . In Proceedings of BSDcon 2003 ( San Mateo, CA).]] Rosenthal, D. S. H. 2003. A digital preservation network appliance based on OpenBSD. In Proceedings of BSDcon 2003 (San Mateo, CA).]]

Rosenthal , D. S. H. and Reich , V . 2000. Permanent Web publishing . In Proceedings of the USENIX Annual Technical Conference, Freenix Track ( San Diego, CA). 129--140.]] Rosenthal, D. S. H. and Reich, V. 2000. Permanent Web publishing. In Proceedings of the USENIX Annual Technical Conference, Freenix Track (San Diego, CA). 129--140.]]

Rosenthal , D. S. H. , Roussopoulos , M. , Maniatis , P. , and Baker , M . 2003. Economic measures to resist attacks on a peer-to-peer network . In Proceedings of the Workshop on Economics of Peer-to-Peer Systems ( Berkeley, CA).]] Rosenthal, D. S. H., Roussopoulos, M., Maniatis, P., and Baker, M. 2003. Economic measures to resist attacks on a peer-to-peer network. In Proceedings of the Workshop on Economics of Peer-to-Peer Systems (Berkeley, CA).]]

Rowstron , A. and Druschel , P . 2001. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility . In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (Chateau Lake Louise, Banff, AB, Canada). 188--201 .]] 10.1145/50 2034 .502053 Rowstron, A. and Druschel, P. 2001. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. In Proceedings of the Eighteenth ACM Symposium on Operating Systems Principles (Chateau Lake Louise, Banff, AB, Canada). 188--201.]] 10.1145/502034.502053

Staniford , S. , Paxson , V. , and Weaver , N . 2002. How to Own the Internet in your spare time . In Proceedings of the 11th USENIX Security Symposium ( San Francisco, CA). 149--167.]] Staniford, S., Paxson, V., and Weaver, N. 2002. How to Own the Internet in your spare time. In Proceedings of the 11th USENIX Security Symposium (San Francisco, CA). 149--167.]]

Tenopir C. 2004. Online scholarly journals: How many? The Library Journal 2 (Feb.). Available online at http://www.libraryjournal.com/index.asp?layout=articlePrint&articleID=C%A374956.]] Tenopir C. 2004. Online scholarly journals: How many? The Library Journal 2 (Feb.). Available online at http://www.libraryjournal.com/index.asp?layout=articlePrint&articleID=C%A374956.]]

U. S. Government. 2004. Web design and usability guidelines. Available online at http://usability.gov/guidelines/softhard.html.]] U. S. Government. 2004. Web design and usability guidelines. Available online at http://usability.gov/guidelines/softhard.html.]]

Venema , W. 1996 . Murphy's Law and computer security . In Proceedings of the 6th USENIX Security Symposium ( San Jose, CA).]] Venema, W. 1996. Murphy's Law and computer security. In Proceedings of the 6th USENIX Security Symposium (San Jose, CA).]]

Waldman , M. and Mazières , D . 2001. Tangler: A censorship-resistant publishing system based on document entanglements . In Proceedings of the 8th ACM Conference on Computer and Communications Security ( Philadelphia, PA). 126--135.]] 10.1145/50 1983 .502002 Waldman, M. and Mazières, D. 2001. Tangler: A censorship-resistant publishing system based on document entanglements. In Proceedings of the 8th ACM Conference on Computer and Communications Security (Philadelphia, PA). 126--135.]] 10.1145/501983.502002

Williamson , M. 2002 . Throttling viruses: Restricting propagation to defeat malicious mobile code . In Proceedings of the 18th Annual Computer Security Applications Conference ( Las Vegas, NV).]] Williamson, M. 2002. Throttling viruses: Restricting propagation to defeat malicious mobile code. In Proceedings of the 18th Annual Computer Security Applications Conference (Las Vegas, NV).]]

Wingfield , N. 2002 . EBay's figurine scandal: Auction site merchant disappears with the goods . Wall Street J. , Feb 22.]] Wingfield, N. 2002. EBay's figurine scandal: Auction site merchant disappears with the goods. Wall Street J., Feb 22.]]