Screenshot instructions:
Windows
Mac
Red Hat Linux
Ubuntu
Click URL instructions:
Right-click on ad, choose "Copy Link", then paste here →
(This may not be possible with some types of ads)
|
From: Ferran Jorba <Ferran.Jorba@ua...> - 2007-01-25 15:34:47
|
Hello, I'm going to ask help for something I'd say it is a common situation nowadays mostly everywhere. My university is engaged in digitalisation of old material, like most do. So do some of my neighbour univesrities. Our libraries belong to a local consortium, like most libraries do. This digitalized material means, among other things, lots of fat TIFF files, totaling a huge amount of Gigabytes. I think most of you know that. There are plenty disk array vendors willing to sell you their solutions. I like specially Capricorn Tech (http://www.capricorn-tech.com/), due to their Archive.org pedigree, and Copan Systems (http://www.copansys.com/) for their MAID concept. But one of our most urgent problems is keeping those original TIFF (and their corresponding PDFs) safe beyond just storing them somewhere: I mean having more than one copy, doing backups, veryfiying checksums, automatically fixing the damaged files, maybe changing formats, etc. This second part is already invented, and it is called LOCKSS (http://www.locks.org). It is a software with anything I could ask for, and more. What I have been unable to find in the LOCKSS site is a configuration model where some libraries, in a local consortium, join together to keep jointly this material. Ok, I understand that LOCKSS is designed to keep the material of [external] publishers. But when I first learned about CLOCKSS (Controlled LOCKSS) I immediately thought that it would address the scenario we are facing in our consortium. However, I cannot find it in their web pages. May I ask how are you addressing this scenario? If there is a better forum for this question, I'd gladly ask it there again. Thanks, Ferran |
|
From: Joe Hourcle <oneiros@gr...> - 2007-01-25 16:33:32
|
On Thu, 25 Jan 2007, Ferran Jorba wrote: > Hello, > > I'm going to ask help for something I'd say it is a common situation > nowadays mostly everywhere. > > My university is engaged in digitalisation of old material, like most > do. So do some of my neighbour univesrities. Our libraries belong to > a local consortium, like most libraries do. This digitalized material > means, among other things, lots of fat TIFF files, totaling a huge Could you give us an idea of how big the archive is, and how fast it's expected to grow? For smaller repositories, we're currently using Apple XServe RAID. For stuff that's multiple terrabytes, we're using hardware from Pillar Data Systems <http://www.pillardata.com/>;. I know for our last big purchase, we had looked at Network Appliance <http://www.netapp.com/>;, but I wasn't involved in the purchase decision, so I don't know what the determining factors were (performance, features, support, price, etc.), or what other vendors were evaluated. [trimmed] > What I have been unable to find in the LOCKSS site is a configuration > model where some libraries, in a local consortium, join together to > keep jointly this material. Ok, I understand that LOCKSS is designed > to keep the material of [external] publishers. But when I first > learned about CLOCKSS (Controlled LOCKSS) I immediately thought that > it would address the scenario we are facing in our consortium. However, > I cannot find it in their web pages. http://www.lockss.org/clockss/Home (see the left-nav) http://www.lockss.org/clockss/FAQ Where does the initiative currently stand? The initiative, which began early in 2006, is implementing and evaluating both social and technical models over a two-year period. During this time the initiative will work to build a full-scale production system using a significant portion of the content of the publisher members. The work of the initiative is transparent and will be independently assessed, with all findings reported to the wider community. ----- Joe Hourcle |
|
From: Ferran Jorba <Ferran.Jorba@ua...> - 2007-01-26 14:57:20
|
Thank you all for your responses. [...] > Essentially, these projects (an others) are using private LOCKSS > networks to create a dark archive with the caches somewhat > geographically dispersed. Beth's reply about Auburn participating in a private LOCKSS network has given me hope. I've followed your suggestion and I've filled contacted the LOCKSS people at http://www.lockss.org/clockss/Talkback Answering some of your other questions, I still don't know the size, because I have (partial) information about my own library, but less from the others. Several Terabytes for sure, but again, I know that this is too vague. > Does this help? Sure it does. Thanks again, Ferran |
|
From: Han, Yan <hany@u.library.arizona.edu> - 2007-01-26 21:09:58
|
This is not an easy answer for your questions. my understanding of LOCKSS is that it does not work for straight TIFF = /PDFs. There are organizations who can take care of your problems. OCLC = is testing the idea of preservation. There is also a research project = going on with NDIIPP project, which has a consortium to do digital = preservation. (Emory U. is one of the partners).=20 Or you can just buy some hard drives/tapes and save multiple copies in = off-site storage. but in this case, you are responsible for the = migration of formats etc.=20 I like the idea of consortium preservation, but there are other issues = to be sorted out. Yan Han The University of Arizona Libraries=20 -----Original Message----- From: oss4lib-discuss-bounces@... on behalf of Ferran = Jorba Sent: Thu 1/25/2007 8:33 AM To: OSS4LIB (E-mail) Subject: [oss4lib-discuss] Storing and keeping safe those = hugedigitalisation files =20 Hello, I'm going to ask help for something I'd say it is a common situation nowadays mostly everywhere. My university is engaged in digitalisation of old material, like most do. So do some of my neighbour univesrities. Our libraries belong to a local consortium, like most libraries do. This digitalized material means, among other things, lots of fat TIFF files, totaling a huge amount of Gigabytes. I think most of you know that. There are plenty disk array vendors willing to sell you their solutions. I like specially Capricorn Tech (http://www.capricorn-tech.com/), due to their Archive.org pedigree, and Copan Systems (http://www.copansys.com/) for their MAID concept. But one of our most urgent problems is keeping those original TIFF (and their corresponding PDFs) safe beyond just storing them somewhere: I mean having more than one copy, doing backups, veryfiying checksums, automatically fixing the damaged files, maybe changing formats, etc. This second part is already invented, and it is called LOCKSS (http://www.locks.org). It is a software with anything I could ask for, and more. What I have been unable to find in the LOCKSS site is a configuration model where some libraries, in a local consortium, join together to keep jointly this material. Ok, I understand that LOCKSS is designed to keep the material of [external] publishers. But when I first learned about CLOCKSS (Controlled LOCKSS) I immediately thought that it would address the scenario we are facing in our consortium. However, I cannot find it in their web pages. May I ask how are you addressing this scenario? If there is a better forum for this question, I'd gladly ask it there again. Thanks, Ferran -------------------------------------------------------------------------= Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share = your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDEV _______________________________________________ oss4lib-discuss@... https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://oss4lib.org/ |
|
From: Stephen De Gabrielle <spdegabrielle@gm...> - 2007-01-27 05:36:24
Attachments:
Message as HTML
|
Hi I just thought I'd point out some useful resources; This message talks about making LOCKSS work with Digital repository software to create a private LOCKSS network. http://www.sfu.ca/~hgmorris/openaccess-archiving/msg00006.html Hopefully this helps, as the Repository software, is getting pretty good at supporting the digital preservation work you mention in you original email. Also the China Digital Museum Project paper 'Building a Distributed, Standards-based Repository Federation' talks a fair bit about how they handled replication and naming of metadata and content using the DSpace Digital Repository software. -- http://www.dlib.org/dlib/july06/tansley/07tansley.html The RLG has some nice work that you may find interesting: Attributes of Trusted Digital Repositories - http://www.rlg.org/en/page.php?Page_ID=583 Audit Checklist for Certifying Digital Repositories (Draft - but good enough for others[MAGDIR] to consider it as a model) -- http://www.rlg.org/en/page.php?Page_ID=20769 Cheers, Stephen De Gabrielle On 1/27/07, Han, Yan <hany@...> wrote: > > This is not an easy answer for your questions. > > my understanding of LOCKSS is that it does not work for straight TIFF > /PDFs. There are organizations who can take care of your problems. OCLC is > testing the idea of preservation. There is also a research project going on > with NDIIPP project, which has a consortium to do digital preservation. > (Emory U. is one of the partners). > > Or you can just buy some hard drives/tapes and save multiple copies in > off-site storage. but in this case, you are responsible for the migration of > formats etc. > > I like the idea of consortium preservation, but there are other issues to > be sorted out. > > Yan Han > The University of Arizona Libraries > > -----Original Message----- > From: oss4lib-discuss-bounces@... on behalf of Ferran > Jorba > Sent: Thu 1/25/2007 8:33 AM > To: OSS4LIB (E-mail) > Subject: [oss4lib-discuss] Storing and keeping safe those > hugedigitalisation files > > Hello, > > I'm going to ask help for something I'd say it is a common situation > nowadays mostly everywhere. > > My university is engaged in digitalisation of old material, like most > do. So do some of my neighbour univesrities. Our libraries belong to > a local consortium, like most libraries do. This digitalized material > means, among other things, lots of fat TIFF files, totaling a huge > amount of Gigabytes. I think most of you know that. > > There are plenty disk array vendors willing to sell you their > solutions. I like specially Capricorn Tech > (http://www.capricorn-tech.com/), due to their Archive.org pedigree, > and Copan Systems (http://www.copansys.com/) for their MAID concept. > > But one of our most urgent problems is keeping those original TIFF > (and their corresponding PDFs) safe beyond just storing them > somewhere: I mean having more than one copy, doing backups, veryfiying > checksums, automatically fixing the damaged files, maybe changing > formats, etc. This second part is already invented, and it is called > LOCKSS (http://www.locks.org). It is a software with anything I could > ask for, and more. > > What I have been unable to find in the LOCKSS site is a configuration > model where some libraries, in a local consortium, join together to > keep jointly this material. Ok, I understand that LOCKSS is designed > to keep the material of [external] publishers. But when I first > learned about CLOCKSS (Controlled LOCKSS) I immediately thought that > it would address the scenario we are facing in our consortium. However, > I cannot find it in their web pages. > > May I ask how are you addressing this scenario? If there is a better > forum for this question, I'd gladly ask it there again. > > Thanks, > > Ferran > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > > > oss4lib-discuss@... > > > https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss > see also http://oss4lib.org/ > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share > your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > > > oss4lib-discuss@... > > > https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss > see also http://oss4lib.org/ > -- -- Stephen De Gabrielle |
|
From: Ferran Jorba <Ferran.Jorba@ua...> - 2007-01-29 09:51:38
|
Hi Yan and Stephen, Thank both for your ideas and links. I'll keep an eye on them. Apparently there is a list called openaccess-archiving (found via a link Stephen sent) for this kind of issues, as statet in their announcement: http://www.sfu.ca/~hgmorris/openaccess-archiving/msg00000.html It seems to me also, from what I have learned since I started this thread, that LOCKSS needs a plug-ins for each kind of archive, maybe even for those "straight TIFF/PDFs" that Yan wrote. I'll try to learn a bit more installing a LOCKSS node myself. Again, thanks to all, Ferran |
|
From: Beth Nicol <nicollb@au...> - 2007-01-29 14:31:21
Attachments:
Message as HTML
|
I'm not sure what you mean by "it does not work for straight TIFF/PDF's" -- you must organize your files into Archival Units, and create a manifest page just as you would for a journal. I've harvested several GB's of tiff's using LOCKSS. Beth Nicol <nicollb@...> Information Technology Specialist Auburn University Libraries (334)844-1731 >>>"Han, Yan" <hany@...> 01/26/07 3:09 PM >>> This is not an easy answer for your questions. my understanding of LOCKSS is that it does not work for straight TIFF /PDFs. There are organizations who can take care of your problems. OCLC is testing the idea of preservation. There is also a research project going on with NDIIPP project, which has a consortium to do digital preservation. (Emory U. is one of the partners). Or you can just buy some hard drives/tapes and save multiple copies in off-site storage. but in this case, you are responsible for the migration of formats etc. I like the idea of consortium preservation, but there are other issues to be sorted out. Yan Han The University of Arizona Libraries -----Original Message----- From: oss4lib-discuss-bounces@... on behalf of Ferran Jorba Sent: Thu 1/25/2007 8:33 AM To: OSS4LIB (E-mail) Subject: [oss4lib-discuss] Storing and keeping safe those hugedigitalisation files Hello, I'm going to ask help for something I'd say it is a common situation nowadays mostly everywhere. My university is engaged in digitalisation of old material, like most do. So do some of my neighbour univesrities. Our libraries belong to a local consortium, like most libraries do. This digitalized material means, among other things, lots of fat TIFF files, totaling a huge amount of Gigabytes. I think most of you know that. There are plenty disk array vendors willing to sell you their solutions. I like specially Capricorn Tech (http://www.capricorn-tech.com/), due to their Archive.org pedigree, and Copan Systems (http://www.copansys.com/) for their MAID concept. But one of our most urgent problems is keeping those original TIFF (and their corresponding PDFs) safe beyond just storing them somewhere: I mean having more than one copy, doing backups, veryfiying checksums, automatically fixing the damaged files, maybe changing formats, etc. This second part is already invented, and it is called LOCKSS (http://www.locks.org). It is a software with anything I could ask for, and more. What I have been unable to find in the LOCKSS site is a configuration model where some libraries, in a local consortium, join together to keep jointly this material. Ok, I understand that LOCKSS is designed to keep the material of [external] publishers. But when I first learned about CLOCKSS (Controlled LOCKSS) I immediately thought that it would address the scenario we are facing in our consortium. However, I cannot find it in their web pages. May I ask how are you addressing this scenario? If there is a better forum for this question, I'd gladly ask it there again. Thanks, Ferran ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ( http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ) _______________________________________________ oss4lib-discuss@... https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://oss4lib.org/ ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ( http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV ) _______________________________________________ oss4lib-discuss@... https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://oss4lib.org/ |
|
From: Han, Yan <hany@u.library.arizona.edu> - 2007-01-29 18:15:32
Attachments:
Message as HTML
|
Beth, =20 Thanks for point out.=20 =20 The algorithem in LOCKSS is to use peers in the network to preserve/restore/repair files (by voting with the majority). My question is: if I have a MD5/SHA signiture, I know if the file is authenticated. Why do I need a vote?=20 =20 As your library is a member of MetaArchive, could you explain more about how you handle the digitial signiture? do you do any modification of the LOCKSS source code? what about the cost? (in this case, I assume that you are using PC as a storage unit. the cost should be lower). =20 =20 Yan=20 ________________________________ From: Beth Nicol [mailto:nicollb@...]=20 Sent: Monday, January 29, 2007 7:31 AM To: OSS4LIB (E-mail); Han, Yan; Ferran Jorba Subject: Re: [oss4lib-discuss] Storing and keeping safe thosehugedigitalisation files I'm not sure what you mean by "it does not work for straight TIFF/PDF's" -- you must organize your files into Archival Units, and create a manifest page just as you would for a journal. I've harvested several GB's of tiff's using LOCKSS. Beth Nicol <nicollb@...> Information Technology Specialist Auburn University Libraries (334)844-1731 >>>"Han, Yan" <hany@...> 01/26/07 3:09 PM >>> This is not an easy answer for your questions. my understanding of LOCKSS is that it does not work for straight TIFF /PDFs. There are organizations who can take care of your problems. OCLC is testing the idea of preservation. There is also a research project going on with NDIIPP project, which has a consortium to do digital preserv! ation. (Emory U. is one of the partners). Or you can just buy some hard drives/tapes and save multiple copies in off-site storage. but in this case, you are responsible for the migration of formats etc. I like the idea of consortium preservation, but there are other issues to be sorted out. Yan Han The University of Arizona Libraries -----Original Message----- From: oss4lib-discuss-bounces@... on behalf of Ferran Jorba Sent: Thu 1/25/2007 8:33 AM To: OSS4LIB (E-mail) Subject: [oss4lib-discuss] Storing and keeping safe those hugedigitalisation files Hello, I'm going to ask help for something I'd say it is a common situation nowadays mostly everywhere. My university is engaged in digitalisation of old material, like most do. So do some of my neighbour univesrities. Our libraries belong to a local consortium, like ! most libraries do. This digitalized material means, among other things, lots of fat TIFF files, totaling a huge amount of Gigabytes. I think most of you know that. There are plenty disk array vendors willing to sell you their solutions. I like specially Capricorn Tech (http://www.capricorn-tech.com/), due to their Archive.org pedigree, and Copan Systems (http://www.copansys.com/) for their MAID concept. But one of our most urgent problems is keeping those original TIFF (and their corresponding PDFs) safe beyond just storing them somewhere: I mean having more than one copy, doing backups, veryfiying checksums, automatically fixing the damaged files, maybe changing formats, etc. This second part is already invented, and it is called LOCKSS (http://www.locks.org). It is a software with anything I could ask for, and more. What I have been unable to find in the L! OCKSS site is a configuration model where some libraries, in a local consortium, join together to keep jointly this material. Ok, I understand that LOCKSS is designed to keep the material of [external] publishers. But when I first learned about CLOCKSS (Controlled LOCKSS) I immediately thought that it would address the scenario we are facing in our consortium. However, I cannot find it in their web pages. May I ask how are you addressing this scenario? If there is a better forum for this question, I'd gladly ask it there again. Thanks, Ferran ------------------------------------------------------------------------ - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash ! http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDE V _______________________________________________ oss4lib-discuss@... https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://oss4lib.org/ ------------------------------------------------------------------------ - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D= DEVDE V _______________________________________________ oss4lib-dis! cuss@... https://lists.sourceforge.net/lists/listinfo/oss4lib-discuss see also http://oss4lib.org/ |