Menu
▾
▴
[sword-app-tech] Package Registry [was: Documentation on packaging formats]
|
From: Richard J. <ri...@co...> - 2012-05-24 16:02:45
|
Hi Mark, > I understand the protocol, it's very clear, but what I am asking is, is there a list somewhere of package formats, and their internal structures (e.g., what metadata formats are allowed, how are relationships between multipart objects like a thesis and its two supplemental files described, etc.). No such thing exists (yet). We have talked about doing something like this for SWORD for a long time, but the problem is surprisingly hard, and some concerted effort and support would be needed to successfully pull it off. Conscious of the fact that support for a standard package format is a central problem of deposit which wasn't being addressed by SWORDv1, we have implemented a minimal possible package format in SWORDv2 which consists of: a) providing metadata as dcterms embedded in an atom:entry document b) providing a simple zip file as the packaged payload To deposit a "package" consisting of some dcterms metadata and a bunch of zipped files you can do: POST Atom Entry to Col-IRI - creates a new item with the metadata PUT SimpleZip to EM-IRI - sends the content to the media resource of the item In DSpace this will unpack all the files into the ORIGINAL bundle, and will xwalk the metadata as per the swordv2-server.cfg. Some servers may allow you to do this same operation in a single Multipart deposit, but from a technical point of view we have had a hard time getting multipart to behave consistently or uniformly, and overall server-side support is poor, so I recommend the above 2 stage deposit. > Marco used BagIt as an example, and my own institution would use BagIt as a way of moving groups of related files around and storing them together, but, to draw a parallel between SWORD and BagIt, both are just containers that are agnostic to the payloads they carry. METS is the same. Tools that define the container only handle half the process of data exchange; what we need are a set of profiles or other ways of specifying the details of how the payload is structured so that once it is removed from the container, other applications know how to reassemble it. Some caution required with these definitions: SWORD is not a container format, it's a transport protocol, most principally defined by its protocol operations (Section 6 in the spec). SWORD is more interested in the process of data exchange (what it means to create, to update, to delete), while BagIt is more focussed on the format of data exchange (albeit in a relatively loose way). > If such a list or registry doesn't exist, would there be interest in starting one, and describing the packaging formats the community is using (and willing to share) in a standardized way, as is done with METS profiles or DC application profiles? I would very much like to start one, though it could be quite hard to do generically. In the mean time, I have been working on a more practical idea for DSpace: For another project I wrote a DocX and XIF ingester for SWORDv2, which currently aren't incorporated in the main DSpace release, but are available in a separate project here: https://bitbucket.org/richardjones/sword-packagers I would like to migrate these to the swordapp project on github, and restructure the project so that it would be easy for others to contrib their own packager plugins. Probably each plugin would be its own maven project, and they could be included as modules in an overall "sword packagers" maven module. Open to other suggestions though. By doing something practical, rather than working on a more abstract package registry, we might learn enough about the problem space and the package formats which are in common usage to bootstrap a package registry off of it (or maybe we won't even need to). Cheers, Richard > ----- Original Message ----- >> Hi Mark, >> >> What you are after is the Service Document. The Sword spec describes >> a >> Service Document that the server should provide when requested, to >> advertise amongst other things, what package formats it supports. So >> the >> typical conversation would go... >> >> Client - "Please send me your Service Document" >> Server - "Here it is" >> Client - "Thanks. Now I know what you support I'm going to send you a >> package you will be happy with" >> Server - "Got it!" >> >> Sorry, got to rush but I'll try and find some links to examples >> later. >> >> Cheers, Robin. >> >> >> >> On 24/05/12 15:39, Mark Jordan wrote: >> > Marco, >> > >> > Thanks for the pointer to your blog, I'll take a look. I realize >> > that SWORD is just a transfer protocol, and that the packaging >> > formats are independent of the protocol (in the same way that >> > metadata formats are independent of OAI-PMH), but I am trying to >> > determine the degree to which the packing formats are >> > standardized. Specifically, I'm struggling to understand how a >> > client and a server can exchange content effectively in the >> > packaging format Foo (which they both agree via a protocol >> > handshake is the format they are exchanging) if they don't have a >> > shared understanding of the internal structure of the Foo format, >> > or, more accurately, if the services behind the client and the >> > server don't understand the internal structure of Foo. >> > >> > So, I'm really asking about what happens before the SWORD client >> > sends the package off to the server, and what happens after the >> > SWORD server hands the transferred package off to the repository. >> > If a publisher or other service provider asks "Does your >> > repository support SWORD?" I'd like to be able to tell them, "Yes, >> > in these formats that we all understand", instead of having to >> > explain to the publisher how I want the content packaged up, or >> > take the content in whatever format the publisher provides and >> > figure out how to get it into the target repository. >> > >> > Mark >> > >> > ----- Original Message ----- >> >> Hi Mark, >> >> >> >> I have been using SWORDv2 with DSpace. SWORD is just a transfer >> >> protocol, it doesn't really matter what type of package you send >> >> with it, as long as the receiving SWORD server understands how to >> >> handle it. >> >> >> >> I used DSpace METS SIP, simple zip files, and binary files because >> >> the DSpace SWORDv2 server implementation supported those packages. >> >> Then, we wanted to try BagIt, so I wrote a so called "ingester" to >> >> let the SWORDv2 DSpace server handle BagIt packages. And since I >> >> was >> >> at it, I also made one for DataBankBagIt packages, which are not >> >> in >> >> the SWORD documentation (and also have a different namespace, >> >> http://dataflow.ox.ac.uk/package/DataBankBagIt). You can define >> >> your >> >> own package format if you want to. >> >> >> >> You can read about our work on SWORD on the blog of our project, >> >> Sustainable Management of Digital Music Research Data: >> >> >> >> http://rdm.c4dm.eecs.qmul.ac.uk/sword-tools >> >> >> >> http://rdm.c4dm.eecs.qmul.ac.uk/datastage-and-dspace >> >> >> >> Good luck! >> >> Best regards >> >> Marco >> >> >> >> -------------------------------------------------- >> >> Marco Fabiani >> >> Postdoctoral Research Assistant >> >> Centre for Digital Music >> >> School of Electronic Engineering and Computer Science >> >> Queen Mary, University of London >> >> Mile End Road, London E1 4NS, UK >> >> >> >> On 23 May 2012, at 21:40, Mark Jordan wrote: >> >> >> >>> Hi, >> >>> >> >>> Sorry for this second n00b question to the list in less than a >> >>> few >> >>> weeks. >> >>> >> >>> Is there any public documentation on SWORD2 packaging formats? >> >>> The >> >>> profile uses the DSpace METS SIP and BagIt as examples. I assume >> >>> that the DSpace packaging format is the one described at >> >>> https://wiki.duraspace.org/display/DSPACE/DSpaceMETSSIPProfile, >> >>> but is this actually the case? Does a BagIt profile actually >> >>> exist >> >>> or is it just used as an example? >> >>> >> >>> Our most immediate use case is that we am exploring using SWORD2 >> >>> to >> >>> move theses from our thesis management system to our Drupal-based >> >>> IR. There is a SWORD1 server module for Drupal but not a SWORD2 >> >>> server. I'd like to make the SWORD2 server as generic as possible >> >>> in terms of deposit but am unclear on what the common packaging >> >>> formats are and how they are documented. >> >>> >> >>> Thanks, >> >>> >> >>> Mark >> >>> >> >>> >> >>> >> >>> ------------------------------------------------------------------------------ >> >>> Live Security Virtual Conference >> >>> Exclusive live event will cover all the ways today's security and >> >>> threat landscape has changed and how IT managers can respond. >> >>> Discussions >> >>> will include endpoint security, mobile security and the latest in >> >>> malware >> >>> threats. >> >>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> >>> _______________________________________________ >> >>> sword-app-tech mailing list >> >>> swo...@li... >> >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> >> > ------------------------------------------------------------------------------ >> > Live Security Virtual Conference >> > Exclusive live event will cover all the ways today's security and >> > threat landscape has changed and how IT managers can respond. >> > Discussions >> > will include endpoint security, mobile security and the latest in >> > malware >> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> > _______________________________________________ >> > sword-app-tech mailing list >> > swo...@li... >> > https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. >> Discussions >> will include endpoint security, mobile security and the latest in >> malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
