Menu
▾
▴
sword-app-tech — Mailing list for discussion of SWORD implemenation and development.
You can subscribe to this list here.
| 2008 |
Jan
(22) |
Feb
(8) |
Mar
(9) |
Apr
(4) |
May
(17) |
Jun
(29) |
Jul
(11) |
Aug
(13) |
Sep
(17) |
Oct
(14) |
Nov
(41) |
Dec
(8) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2009 |
Jan
(17) |
Feb
(26) |
Mar
(18) |
Apr
(1) |
May
(11) |
Jun
(20) |
Jul
(5) |
Aug
|
Sep
|
Oct
|
Nov
(3) |
Dec
|
| 2010 |
Jan
(23) |
Feb
(7) |
Mar
(9) |
Apr
(13) |
May
(5) |
Jun
|
Jul
(3) |
Aug
(5) |
Sep
|
Oct
(1) |
Nov
(3) |
Dec
|
| 2011 |
Jan
(3) |
Feb
|
Mar
(2) |
Apr
(1) |
May
|
Jun
(14) |
Jul
(22) |
Aug
(1) |
Sep
(2) |
Oct
(11) |
Nov
(11) |
Dec
(35) |
| 2012 |
Jan
(17) |
Feb
(12) |
Mar
(41) |
Apr
(40) |
May
(41) |
Jun
(27) |
Jul
(9) |
Aug
(1) |
Sep
|
Oct
(6) |
Nov
|
Dec
(11) |
| 2013 |
Jan
|
Feb
(4) |
Mar
(2) |
Apr
(8) |
May
(1) |
Jun
(18) |
Jul
(10) |
Aug
(16) |
Sep
(2) |
Oct
(1) |
Nov
(14) |
Dec
(11) |
| 2014 |
Jan
(7) |
Feb
(2) |
Mar
|
Apr
|
May
(8) |
Jun
(1) |
Jul
(7) |
Aug
(10) |
Sep
(8) |
Oct
(8) |
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
(2) |
Mar
(2) |
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2016 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(4) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(6) |
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
(4) |
| 2018 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Philip D. <phi...@ha...> - 2014-07-10 12:30:26
|
Well... I guess it's more complicated than I realized. Here are two concrete issues to look at: Allow disabling SSL certificate validation · Issue #9 · swordapp/python-client-sword2 - https://github.com/swordapp/python-client-sword2/issues/9 Add support for unicode characters · Issue #10 · swordapp/python-client-sword2 - https://github.com/swordapp/python-client-sword2/issues/10 Both were filed by https://github.com/rliebz on behalf of http://centerforopenscience.org but I'm not sure if any of those folks are on this mailing list or not. I'll forward this to make sure they're aware of this conversation. The history here is that https://github.com/swordapp/python-client-sword2/pull/7 is quite old (almost a year) and comes from https://github.com/pjbull/python-client-sword2 (which is used in https://github.com/dvn/swordpoc/tree/master/dvn_client which I encouraged the development of but is now considered somewhat legacy) which was subsequently forked into https://github.com/rliebz/python-client-sword2 (which is being used in https://github.com/rliebz/dvn-client-python which is actively maintained). I guess the main point is to please think about those two issues rather than looking at https://github.com/swordapp/python-client-sword2/pull/7 ( https://github.com/pjbull is no longer actively working on his fork). I don't actually know anything about https://github.com/swordapp/python-client-sword2/pull/6 (I don't know who https://github.com/sjwqmul is ). I hopes this helps! Phil On Thu, Jul 10, 2014 at 4:47 AM, Richard Jones <ri...@co...> wrote: > So that would cover both these pull requests, right: > > https://github.com/swordapp/python-client-sword2/pulls > > Will do! > > Cheers, > > Richard > > > > On 7 July 2014 18:53, Philip Durbin <phi...@ha...> wrote: >> >> Richard, if you could also please look at contributions at >> https://github.com/rliebz/python-client-sword2 I'd appreciate it. That code >> is being used in an integration between Open Science Framework (OSF) and >> Dataverse: >> https://github.com/CenterForOpenScience/openscienceframework.org/issues/112 >> >> >> On Mon, Jul 7, 2014 at 12:44 PM, Richard Jones <ri...@co...> >> wrote: >>> >>> Hi Holly, >>> >>> Thanks for bringing this up - good point. I have a tendency just to use >>> the tip of the master whenever I need this. I'm pretty tied up at the >>> moment, but I've put it on my todo list and I'll get to it as soon as I can. >>> >>> Cheers, >>> >>> Richard >>> >>> >>> >>> >>> On 28 May 2014 20:01, Holly Becker <hb...@ar...> wrote: >>>> >>>> Hello all, >>>> >>>> Can another release of the python client be made? There some features >>>> available on the Github version [1] that I'd like in a packaged version, and >>>> I think it's been long enough since the last release that another one is >>>> reasonable. I filed an issue for this as well [2]. >>>> >>>> Thanks! >>>> Holly Becker >>>> >>>> [1] https://github.com/swordapp/python-client-sword2 >>>> [2] https://github.com/swordapp/python-client-sword2/issues/8 >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Time is money. Stop wasting it! Get your web API in 5 minutes. >>>> www.restlet.com/download >>>> http://p.sf.net/sfu/restlet >>>> _______________________________________________ >>>> sword-app-tech mailing list >>>> swo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >>>> >>> >>> >>> >>> -- >>> >>> Richard Jones, >>> >>> Founder, Cottage Labs >>> t: @richard_d_jones, @cottagelabs >>> w: http://cottagelabs.com >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Open source business process management suite built on Java and Eclipse >>> Turn processes into business applications with Bonita BPM Community >>> Edition >>> Quickly connect people, data, and systems into organized workflows >>> Winner of BOSSIE, CODIE, OW2 and Gartner awards >>> http://p.sf.net/sfu/Bonitasoft >>> >>> _______________________________________________ >>> sword-app-tech mailing list >>> swo...@li... >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >>> >> >> >> >> -- >> Philip Durbin >> Software Developer for http://dataverse.org >> http://www.iq.harvard.edu/people/philip-durbin > > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com > -- Philip Durbin Software Developer for http://dataverse.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2014-07-10 08:54:53
|
Hi Folks, A couple of emails about doing releases and merging pull requests have prompted me to bring up the issue of code governance for our various sword libraries. Essentially, I'm concerned that I've become an unnecessary bottleneck for these libraries, and wonder what would be the best solution to that. My current position is that I use the sword code myself in a variety of contexts but I have very little spare time on top of client-work to manage the core code (accepting pull requests, responding on list, making releases, etc). Obviously people are free to fork and modify the code as needed, but maybe it would be better if more of us had admin rights on the repositories? I'm hoping, also, that there will be more funding at this end at some point for some more "core" development, but opportunities are currently vague, so I don't think there's any point in holding out for it. Instead it seems much better to try to increase community engagement with the code and try to sustain it that way. What do you all think? Does anyone have a burning desire to be a sword "committer"? Cheers, Richard -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2014-07-10 08:47:50
|
So that would cover both these pull requests, right: https://github.com/swordapp/python-client-sword2/pulls Will do! Cheers, Richard On 7 July 2014 18:53, Philip Durbin <phi...@ha...> wrote: > Richard, if you could also please look at contributions at > https://github.com/rliebz/python-client-sword2 I'd appreciate it. That > code is being used in an integration between Open Science Framework (OSF) > and Dataverse: > https://github.com/CenterForOpenScience/openscienceframework.org/issues/112 > > > On Mon, Jul 7, 2014 at 12:44 PM, Richard Jones <ri...@co...> > wrote: > >> Hi Holly, >> >> Thanks for bringing this up - good point. I have a tendency just to use >> the tip of the master whenever I need this. I'm pretty tied up at the >> moment, but I've put it on my todo list and I'll get to it as soon as I can. >> >> Cheers, >> >> Richard >> >> >> >> >> On 28 May 2014 20:01, Holly Becker <hb...@ar...> wrote: >> >>> Hello all, >>> >>> Can another release of the python client be made? There some features >>> available on the Github version [1] that I'd like in a packaged version, >>> and I think it's been long enough since the last release that another one >>> is reasonable. I filed an issue for this as well [2]. >>> >>> Thanks! >>> Holly Becker >>> >>> [1] https://github.com/swordapp/python-client-sword2 >>> [2] https://github.com/swordapp/python-client-sword2/issues/8 >>> >>> >>> ------------------------------------------------------------------------------ >>> Time is money. Stop wasting it! Get your web API in 5 minutes. >>> www.restlet.com/download >>> http://p.sf.net/sfu/restlet >>> _______________________________________________ >>> sword-app-tech mailing list >>> swo...@li... >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >>> >>> >> >> >> -- >> >> Richard Jones, >> >> Founder, Cottage Labs >> t: @richard_d_jones, @cottagelabs >> w: http://cottagelabs.com >> >> >> >> ------------------------------------------------------------------------------ >> Open source business process management suite built on Java and Eclipse >> Turn processes into business applications with Bonita BPM Community >> Edition >> Quickly connect people, data, and systems into organized workflows >> Winner of BOSSIE, CODIE, OW2 and Gartner awards >> http://p.sf.net/sfu/Bonitasoft >> >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> > > > -- > Philip Durbin > Software Developer for http://dataverse.org <http://thedata.org> > http://www.iq.harvard.edu/people/philip-durbin > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Philip D. <phi...@ha...> - 2014-07-09 13:01:46
|
Ah, thanks, I understand the problem space better now. It sounds like you'd be open to a pull request. I'd probably start by creating an issue though. Perhaps an issue with a title such as: SWORD Statement could expose information more per file under "content" than "type" and "src" For example, that line for me might say: <content type="text/tab-separated-values" src=" https://pdurbin.pagekite.me:8181/dvn/api/data-deposit/v1/swordv2/edit-media/file/23/50by1000.tab "/> But it could say: <content type="text/tab-separated-values" src=" https://pdurbin.pagekite.me:8181/dvn/api/data-deposit/v1/swordv2/edit-media/file/23/50by1000.tab" filename="50by1000.tab"/> I'd love to be able to expose a filename *not* within a URL/IRI because URLs have certain limits on characters. For example I replace a space with an underscore here to avoid an IRISyntaxException: https://github.com/IQSS/dataverse/blob/77b981d85014a622f8ce0de8e7a23c22e2b73ae1/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/StatementManagerImpl.java#L116 Phil On Wed, Jul 9, 2014 at 7:08 AM, Richard Jones <ri...@co...> wrote: > Hi Phil, > > I think this is probably the problem: > > > https://github.com/swordapp/JavaServer2.0/blob/master/src/main/java/org/swordapp/server/AtomStatement.java#L58 > > (and similarly in the ORE format) > > There's no way to actually serialise the properties on the resource part. > You'd probably want to either patch the AtomStatement, or extend it with > our own implementation which is returned by the StatementManagerImpl. > > Cheers, > > Richard > > > > On 19 June 2014 14:58, Philip Durbin <phi...@ha...> wrote: > >> Question about https://github.com/swordapp/JavaServer2.0 >> >> I was originally hoping to expose more metadata about files (besides >> MediaType) and took a couple runs at using properties on a >> ResourcePart but it doesn't seem show in the Atom statement. >> >> This is what I have commented out (with a todo) at >> >> https://github.com/IQSS/dataverse/blob/77b981d85014a622f8ce0de8e7a23c22e2b73ae1/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/StatementManagerImpl.java#L133 >> >> /** >> * @todo: Why are properties set on a ResourcePart not >> * exposed when you GET a Statement? >> */ >> Map<String, String> properties = new HashMap<String, String>(); >> properties.put("filename", studyFile.getFileName()); >> properties.put("category", studyFile.getLatestCategory()); >> properties.put("originalFileType", studyFile.getOriginalFileType()); >> properties.put("id", studyFile.getId().toString()); >> properties.put("UNF", studyFile.getUnf()); >> resourcePart.setProperties(properties); >> statement.addResource(resourcePart); >> >> I was hoping this would show up in each <entry> for a file. >> >> Any advice here? Is this just a dumb Java coding error on my part? >> What are properties for? Thanks! >> >> Phil >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin >> >> >> ------------------------------------------------------------------------------ >> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions >> Find What Matters Most in Your Big Data with HPCC Systems >> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. >> Leverages Graph Analysis for Fast Processing & Easy Data Exploration >> http://p.sf.net/sfu/hpccsystems >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com > > -- Philip Durbin Software Developer for http://dataverse.org <http://thedata.org> http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2014-07-09 11:32:09
|
Hi Phil, I think this is probably the problem: https://github.com/swordapp/JavaServer2.0/blob/master/src/main/java/org/swordapp/server/AtomStatement.java#L58 (and similarly in the ORE format) There's no way to actually serialise the properties on the resource part. You'd probably want to either patch the AtomStatement, or extend it with our own implementation which is returned by the StatementManagerImpl. Cheers, Richard On 19 June 2014 14:58, Philip Durbin <phi...@ha...> wrote: > Question about https://github.com/swordapp/JavaServer2.0 > > I was originally hoping to expose more metadata about files (besides > MediaType) and took a couple runs at using properties on a > ResourcePart but it doesn't seem show in the Atom statement. > > This is what I have commented out (with a todo) at > > https://github.com/IQSS/dataverse/blob/77b981d85014a622f8ce0de8e7a23c22e2b73ae1/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/StatementManagerImpl.java#L133 > > /** > * @todo: Why are properties set on a ResourcePart not > * exposed when you GET a Statement? > */ > Map<String, String> properties = new HashMap<String, String>(); > properties.put("filename", studyFile.getFileName()); > properties.put("category", studyFile.getLatestCategory()); > properties.put("originalFileType", studyFile.getOriginalFileType()); > properties.put("id", studyFile.getId().toString()); > properties.put("UNF", studyFile.getUnf()); > resourcePart.setProperties(properties); > statement.addResource(resourcePart); > > I was hoping this would show up in each <entry> for a file. > > Any advice here? Is this just a dumb Java coding error on my part? > What are properties for? Thanks! > > Phil > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > > ------------------------------------------------------------------------------ > HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions > Find What Matters Most in Your Big Data with HPCC Systems > Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. > Leverages Graph Analysis for Fast Processing & Easy Data Exploration > http://p.sf.net/sfu/hpccsystems > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Philip D. <phi...@ha...> - 2014-07-07 18:16:30
|
Richard, if you could also please look at contributions at https://github.com/rliebz/python-client-sword2 I'd appreciate it. That code is being used in an integration between Open Science Framework (OSF) and Dataverse: https://github.com/CenterForOpenScience/openscienceframework.org/issues/112 On Mon, Jul 7, 2014 at 12:44 PM, Richard Jones <ri...@co...> wrote: > Hi Holly, > > Thanks for bringing this up - good point. I have a tendency just to use > the tip of the master whenever I need this. I'm pretty tied up at the > moment, but I've put it on my todo list and I'll get to it as soon as I can. > > Cheers, > > Richard > > > > > On 28 May 2014 20:01, Holly Becker <hb...@ar...> wrote: > >> Hello all, >> >> Can another release of the python client be made? There some features >> available on the Github version [1] that I'd like in a packaged version, >> and I think it's been long enough since the last release that another one >> is reasonable. I filed an issue for this as well [2]. >> >> Thanks! >> Holly Becker >> >> [1] https://github.com/swordapp/python-client-sword2 >> [2] https://github.com/swordapp/python-client-sword2/issues/8 >> >> >> ------------------------------------------------------------------------------ >> Time is money. Stop wasting it! Get your web API in 5 minutes. >> www.restlet.com/download >> http://p.sf.net/sfu/restlet >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com > > > > ------------------------------------------------------------------------------ > Open source business process management suite built on Java and Eclipse > Turn processes into business applications with Bonita BPM Community Edition > Quickly connect people, data, and systems into organized workflows > Winner of BOSSIE, CODIE, OW2 and Gartner awards > http://p.sf.net/sfu/Bonitasoft > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > -- Philip Durbin Software Developer for http://dataverse.org <http://thedata.org> http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2014-07-07 17:13:10
|
Hi Holly, Thanks for bringing this up - good point. I have a tendency just to use the tip of the master whenever I need this. I'm pretty tied up at the moment, but I've put it on my todo list and I'll get to it as soon as I can. Cheers, Richard On 28 May 2014 20:01, Holly Becker <hb...@ar...> wrote: > Hello all, > > Can another release of the python client be made? There some features > available on the Github version [1] that I'd like in a packaged version, > and I think it's been long enough since the last release that another one > is reasonable. I filed an issue for this as well [2]. > > Thanks! > Holly Becker > > [1] https://github.com/swordapp/python-client-sword2 > [2] https://github.com/swordapp/python-client-sword2/issues/8 > > > ------------------------------------------------------------------------------ > Time is money. Stop wasting it! Get your web API in 5 minutes. > www.restlet.com/download > http://p.sf.net/sfu/restlet > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Philip D. <phi...@ha...> - 2014-06-19 14:05:43
|
Question about https://github.com/swordapp/JavaServer2.0 I was originally hoping to expose more metadata about files (besides MediaType) and took a couple runs at using properties on a ResourcePart but it doesn't seem show in the Atom statement. This is what I have commented out (with a todo) at https://github.com/IQSS/dataverse/blob/77b981d85014a622f8ce0de8e7a23c22e2b73ae1/src/main/java/edu/harvard/iq/dataverse/api/datadeposit/StatementManagerImpl.java#L133 /** * @todo: Why are properties set on a ResourcePart not * exposed when you GET a Statement? */ Map<String, String> properties = new HashMap<String, String>(); properties.put("filename", studyFile.getFileName()); properties.put("category", studyFile.getLatestCategory()); properties.put("originalFileType", studyFile.getOriginalFileType()); properties.put("id", studyFile.getId().toString()); properties.put("UNF", studyFile.getUnf()); resourcePart.setProperties(properties); statement.addResource(resourcePart); I was hoping this would show up in each <entry> for a file. Any advice here? Is this just a dumb Java coding error on my part? What are properties for? Thanks! Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Holly B. <hb...@ar...> - 2014-05-28 19:26:50
|
Hello all, Can another release of the python client be made? There some features available on the Github version [1] that I'd like in a packaged version, and I think it's been long enough since the last release that another one is reasonable. I filed an issue for this as well [2]. Thanks! Holly Becker [1] https://github.com/swordapp/python-client-sword2 [2] https://github.com/swordapp/python-client-sword2/issues/8 |
|
From: Richard J. <ri...@co...> - 2014-05-28 08:44:25
|
Hi Phil, The sword clients themselves are pretty raw, and designed only to take the data you want to send and serialise it to the wire and then parse the response and give you a decent programmatic way of interacting with it. You could certainly write a wrapper around a client which gave you a bit more control over what documents you could construct and send, but I would certainly keep those separate from the client itself. Going in a similar direction, although certainly not the same, I have been working with the University of Oxford to build a git-like client environment for sword, which allows you to select local files or specify metadata (in dc, or other formats), and to identify repositories to which you would like to deposit, and then give you a "push" from your local object to some remote repository. This doesn't do anything clever with metadata, but it does have a whole aspect to do with packaging, and perhaps the concept could be extended to cover the kind of validation that you describe below. Essentially, when you deposit you ask it to either deposit all the content in some package format, or you ask it to deposit a specific metadata format. In the package deposit case it loads a plugin which knows how to do that format, and it builds the zip for you before sending it - at this stage you could validate the content of the object to check that it can be packaged, and give the user feedback if this validation fails. In the metadata deposit case, it just sends the XML of the metadata that it has on record, but you could theoretically make the process more plugin-oriented, so that you could specify a validation route prior to deposit. At the moment the code is in a not-quite-finished state, but you are welcome to take a look. The core library is here: https://github.com/CottageLabs/dip and there is the beginnings of a command line environment here: https://github.com/CottageLabs/dip-ui Also, generally interested in your thoughts on this - is it useful, are we going in an interesting direction? Cheers, Richard On 20 May 2014 16:13, Philip Durbin <phi...@ha...> wrote: > I was just looking at "Typically, the subject will be represented > using keywords, key phrases, or classification codes. Recommended best > practice is to use a controlled vocabulary." from > http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#subject > > Let's pretend I take this advice and only want to support a controlled > vocabulary for dcterms:subject. Something like this: > > <dcterms:subject>Arts and Humanities > > <dcterms:subject>Astronomy and Astrophysics > > <dcterms:subject>Business and Management > > etc. etc. > > But can I communicate my list of controlled vocabulary for > dcterms:subject (and possibly other fields such as dcterms:language) > to the SWORD client in the Service Document? > > Of would I let the SWORD client *try* to create an Atom resource and > when it fails (because it can't possibly guess what my controlled > vocabulary is) that's when I state in the error message what the > controlled vocabulary is? "Sorry, you must use one of the following > values: Arts and Humanities, ..." Seems messy to parse. > > I expect the answer to all this will be... don't even try this... the > "S" in SWORD is for "simple". :) > > But I'm curious what people think. > > Phil > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > > ------------------------------------------------------------------------------ > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE > Instantly run your Selenium tests across 300+ browser/OS combos. > Get unparalleled scalability from the best Selenium testing platform > available > Simple to use. Nothing to install. Get started now for free." > http://p.sf.net/sfu/SauceLabs > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2014-05-21 11:42:45
|
Hi Joerg, I notice you never got a response to this, so here's a couple of thoughts: The main thing is that sword doesn't care about the metadata format that you are depositing - its job is to handle the transport bit of the deposit. Probably your issue is with the DSpace package ingesters not supporting your metadata format (which would be true whether you were using sword or not to get the content in). If you have custom metadata in your METS manifest, you'll need to modify the DSpaceMETSIntegester (I think it's called) to deal with it. Hope that helps! Cheers, Richard On 30 January 2014 20:30, Joerg Messer <joe...@ub...> wrote: > Greetings, > > I'm trying to get my head around how best to use Sword v2 to upload a > large lecture video collection to our circle.ubc.ca IR. My initial > attempt was to use the Python client implementation which seemed to work > well when I tested it against the demo.dspace.org DSpace 4 server. I was > hoping that our DSpace 3.2 servers would work just as well but that was not > the case. I couldn't even get access to the service document. I don't > know if it's an issue with the Python client or DSpace server Sword > implementation. I haven't been able to find any specific references to > this issue on the Net. > > I'm now trying to prepare the upload package directly but I'm having some > issues with the metadata. I've successfully loaded some sample data using > curl which is encouraging but I was wondering if there was a way to map the > metadata in the METS manifest file to support QDC? It does appear to > support DC, MODS and EPDCX but I have a few fields for which I could not > find a mapping. Here's a sample of the metadata that we're trying to > upload. > > { > "dc.contributor.author" : "Carlsson, Gunnar" > "dc.title" : "Opening remarks: State of TDA" > "dc.type" : "Moving Image" > "dc.date.issued" : "2012-10-15T09:03" > "dc.language.iso" : "eng" > "dc.description.abstract" : "" > "dc.description.affiliation" : "Stanford University" > "dc.description.scholarlevel" : "Faculty" > "dc.format.extent" : "43 minutes" > "dc.publisher" : "Banff International Research Station" > "dc.description.reviewstatus" : "Unreviewed" > "dc.relation" : "12w5081: Topological data analysis and machine learning theory" > "dc.subject" : "Data mining" > } > > Any thoughts would be most welcome. Sword seems to be a great tool when > it works but it does have a bit of a learning curve. > > -- > Joerg Messer > Programmer Analyst > University of British Columbia > Library Systems and Information Technology | Data Services > IKBLC, 1961 East Mall, Vancouver, BC, Canada, V6T 1Z1 > Office 1.604.822.5091 | Mobile 1.604.374.2739 > joe...@ub... > > > ------------------------------------------------------------------------------ > WatchGuard Dimension instantly turns raw network data into actionable > security intelligence. It gives you real-time visual feedback on key > security issues and trends. Skip the complicated setup - simply import > a virtual appliance and go from zero to informed in seconds. > > http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2014-05-21 11:19:26
|
Hi Phil, Looks like devious usage of dcterms xml :) I'm sure a metadata purist would not be so happy, but from sword's point of view this is all fine. I guess either way you need to modify the ingester on the repository end to pull out all your special attributes, and so if you wanted to be less-dc about it, you could just replace the existing ingester which understands the full OJS export format. Just thinking out loud: OJS has a native export format (which contains the base64 encoded files) and also supports the DOAJ format. Would be cool if they were available over sword, and if there was an ingester for DSpace that could handle them. Probably wouldn't be an enormous amount of work, either! Cheers, Richard On 1 May 2014 21:25, Philip Durbin <phi...@ha...> wrote: > Hi Richard et al., > > So sorry for the glacially slow reply. I'm not sure about the header > idea. I haven't given it much thought, to be honest. > > I did want say, "Yes! Exactly!" to you saying, "the schema didn't even > come close to fitting dcterms" because in integrating Dataverse (my > side) and Open Journal Systems (their side) we agreed upon what I've > been calling "the XML attribute" hack to pack more (desperately > needed) information into a dcterms XML element. > > This is best illustrated by example, I believe. > > Sure we support run-of-the-mill dcterms elements like this... > > <dcterms:title>Roasting at Home</dcterms:title> > <dcterms:creator>Peets, John</dcterms:creator> > <dcterms:creator>Stumptown, Jane</dcterms:creator> > <dcterms:publisher>Coffee Bean State University</dcterms:publisher> > > ... but in order to get the article citation from OJS into Dataverse, > we special case the "dcterms:isReferencedBy" element, allowing OJS to > put extra XML attributes (DDI stuff like holdingsURI, agency, and > IDNo) into the XML element like this: > > <dcterms:isReferencedBy > holdingsURI="http://dx.doi.org/10.1038/dvn333" agency="DOI" > IDNo="10.1038/dvn333">Peets, J., & Stumptown, J. (2013). > Roasting at Home. New England Journal of Coffee, 3(1), > 22-34.</dcterms:isReferencedBy> > > It gets the job done AND I just learned today at > http://irclog.iq.harvard.edu/dvn/2014-05-01#i_8982 that OJS got this > "hack" merged into the PHP bindings for SWORD in this pull request: > > Allow attributes on dcterms elements in Atom entries by jwhitney · > Pull Request #6 · swordapp/swordappv2-php-library - > https://github.com/swordapp/swordappv2-php-library/pull/6 > > So maybe we (Dataverse and OJS) are not so off the mark with this... > "interpretation" of SWORD... but I certainly feel like this isn't in > the official spec and I don't know if this sort of thing is supported > in any other client or server libraries. > > Phew! Glad to get that off my chest! :) > > Phil > > > On Mon, Feb 10, 2014 at 1:42 PM, Richard Jones <ri...@co...> > wrote: > > Hi Phil, > > > > We have been doing this for the Duo project in Oslo. We're transferring > > some custom student theses metadata from the national student system to > the > > repository, and the schema didn't even come close to fitting dcterms, so > we > > home-cooked our own (using dc where possible, but extending where > > necessary). Here's my SwordEntryIngester that deals with the > metadata-only > > portion of the deposit (which is only a few fields): > > > > > https://github.com/nye-duo/Duo-DSpace/blob/master/src/main/java/no/uio/duo/FSEntryIngester.java > > > > When doing a full package deposit we have a larger metadata file, which > is > > crosswalked just using an xslt to the native DSpace schema. > > > > So, not crazy - absolutely intended usage :) > > > > A thought that has been bashing around in my head for a while - we > currently > > have a way to describe what format the package being deposited is (via > the > > Packaging header), but no way to say what kind of metadata you are > > depositing. From a server point of view, this means that if you support > > multiple metadata formats, you need a bit of code that can look and try > and > > figure it out, rather than relying on an explicit cue from the deposit. > But > > would adding a header to give this information be viable/useful? > > > > Cheers, > > > > Richard > > > > > > On 10 February 2014 17:50, Philip Durbin <phi...@ha...> > wrote: > >> > >> Dear SWORDers, > >> > >> Sure, we all support dcterms, which is a simple key/value schema, in > >> the Atom entry when creating a Resource. > >> > >> But what about going beyond dcterms? > >> > >> We are interested in supporting richer formats, some of which are > >> hierarchical, such as DDI, as detailed below. > >> > >> Is anyone supporting anything other than dcterms when creating > >> Resources? Is this crazy talk? > >> > >> Thanks in advance for the sanity check! > >> > >> Phil > >> > >> p.s. Here are the details of our situation: > >> > >> So far, according to the OJS Dataverse plugin testers surveyed with > >> results recorded at > >> > >> > https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodDJyd0pZdnlDZ3I5eWxnOHBmV1Q4dHc&usp=sharing > >> the most commonly requested feature is the ability to customize which > >> metadata fields are available as part of the data deposit form, which > >> should be implemented in a future version. In order to support this, > >> we will need to expand the API's metadata support beyond Dublin Core > >> metadata. SWORD Protocol *should* be flexible enough for us to use > >> other standards like DDI. At > >> > >> > http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_creatingresource_entry > >> the SWORDv2 spec says (emphasis added): > >> > >> > * The client *SHOULD add Dublin Core* [DublinCore] terms to the Atom > >> > Entry as foreign markup (if appropriate); the terms MUST be embedded > as > >> > direct children of the atom:entry element, if present. > >> > * The client *MAY add any other metadata formats or foreign markup* to > >> > the atom:entry element > >> > >> We interpret this to mean that in addition to Dublin Core (dcterms, > >> specifically), the SWORD spec is flexible enough to support wildly > >> different metadata formats such as DataCite (https://www.datacite.org > >> ), DDI (Data Documentation Initiative: http://www.ddialliance.org ), > >> VO (Virtual Observatory: http://www.ivoa.net/documents/latest/RM.html > >> ) ISA-Tab (Investigation, Study, and Assay in Tabular format: > >> http://isatab.sourceforge.net/format.html ), etc. > >> > >> >From https://redmine.hmdc.harvard.edu/issues/3425 > >> > >> -- > >> Philip Durbin > >> Software Developer for http://thedata.org > >> http://www.iq.harvard.edu/people/philip-durbin > >> > >> > >> > ------------------------------------------------------------------------------ > >> Android™ apps run on BlackBerry®10 > >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > >> Now with support for Jelly Bean, Bluetooth, Mapview and more. > >> Get your Android app in front of a whole new audience. Start now. > >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> sword-app-tech mailing list > >> swo...@li... > >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > > > > > > > > -- > > > > Richard Jones, > > > > Founder, Cottage Labs > > t: @richard_d_jones, @cottagelabs > > w: http://cottagelabs.com > > > > > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: TAYLOR R. <Rob...@ed...> - 2014-05-21 08:46:30
|
Hi Philip, Please ignore my first hasty answer. I'm still thinking in SWORD v1 terms. SWORD v1 does what you ask in a manner, the Service Document can specify profiles of sorts. I'm forgetting my SWORD terminology but I know you can say, for example, that the server supports a Mets Profile of a particular type. In an ideal world that profile would be described elsewhere in sufficient detail, and made available programatically, to allow you build a compliant SWORD submission. Sadly the whole area surrounding Mets profiles and package description is a bit underdeveloped so in practice I doubt this would be a practical approach, and as for SWORD v2 I'm afraid I have no idea. Sorry, Robin. Robin Taylor Main Library University of Edinburgh ________________________________________ From: TAYLOR Robin <Rob...@ed...> Sent: 21 May 2014 09:21 To: Philip Durbin; <swo...@li...> Subject: Re: [sword-app-tech] communicating a controlled vocabulary to SWORD client Hi Philip, Lets assume for the sake of discussion that you are right and that the answer is... don't even try this :) , then what about considering another web service alongside SWORD? It might be useful for a repository, or similar, to provide a profile(s) of supported metadata for the use of a SWORD client, or any other ingest mechanism for that matter. Cheers, Robin. Robin Taylor Main Library University of Edinburgh ________________________________________ From: Philip Durbin <phi...@ha...> Sent: 20 May 2014 16:13 To: <swo...@li...> Subject: [sword-app-tech] communicating a controlled vocabulary to SWORD client I was just looking at "Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary." from http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#subject Let's pretend I take this advice and only want to support a controlled vocabulary for dcterms:subject. Something like this: <dcterms:subject>Arts and Humanities <dcterms:subject>Astronomy and Astrophysics <dcterms:subject>Business and Management etc. etc. But can I communicate my list of controlled vocabulary for dcterms:subject (and possibly other fields such as dcterms:language) to the SWORD client in the Service Document? Of would I let the SWORD client *try* to create an Atom resource and when it fails (because it can't possibly guess what my controlled vocabulary is) that's when I state in the error message what the controlled vocabulary is? "Sorry, you must use one of the following values: Arts and Humanities, ..." Seems messy to parse. I expect the answer to all this will be... don't even try this... the "S" in SWORD is for "simple". :) But I'm curious what people think. Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ sword-app-tech mailing list swo...@li... https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ sword-app-tech mailing list swo...@li... https://lists.sourceforge.net/lists/listinfo/sword-app-tech |
|
From: TAYLOR R. <Rob...@ed...> - 2014-05-21 08:21:52
|
Hi Philip, Lets assume for the sake of discussion that you are right and that the answer is... don't even try this :) , then what about considering another web service alongside SWORD? It might be useful for a repository, or similar, to provide a profile(s) of supported metadata for the use of a SWORD client, or any other ingest mechanism for that matter. Cheers, Robin. Robin Taylor Main Library University of Edinburgh ________________________________________ From: Philip Durbin <phi...@ha...> Sent: 20 May 2014 16:13 To: <swo...@li...> Subject: [sword-app-tech] communicating a controlled vocabulary to SWORD client I was just looking at "Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary." from http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#subject Let's pretend I take this advice and only want to support a controlled vocabulary for dcterms:subject. Something like this: <dcterms:subject>Arts and Humanities <dcterms:subject>Astronomy and Astrophysics <dcterms:subject>Business and Management etc. etc. But can I communicate my list of controlled vocabulary for dcterms:subject (and possibly other fields such as dcterms:language) to the SWORD client in the Service Document? Of would I let the SWORD client *try* to create an Atom resource and when it fails (because it can't possibly guess what my controlled vocabulary is) that's when I state in the error message what the controlled vocabulary is? "Sorry, you must use one of the following values: Arts and Humanities, ..." Seems messy to parse. I expect the answer to all this will be... don't even try this... the "S" in SWORD is for "simple". :) But I'm curious what people think. Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin ------------------------------------------------------------------------------ "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE Instantly run your Selenium tests across 300+ browser/OS combos. Get unparalleled scalability from the best Selenium testing platform available Simple to use. Nothing to install. Get started now for free." http://p.sf.net/sfu/SauceLabs _______________________________________________ sword-app-tech mailing list swo...@li... https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: Philip D. <phi...@ha...> - 2014-05-20 15:42:09
|
I was just looking at "Typically, the subject will be represented using keywords, key phrases, or classification codes. Recommended best practice is to use a controlled vocabulary." from http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#subject Let's pretend I take this advice and only want to support a controlled vocabulary for dcterms:subject. Something like this: <dcterms:subject>Arts and Humanities <dcterms:subject>Astronomy and Astrophysics <dcterms:subject>Business and Management etc. etc. But can I communicate my list of controlled vocabulary for dcterms:subject (and possibly other fields such as dcterms:language) to the SWORD client in the Service Document? Of would I let the SWORD client *try* to create an Atom resource and when it fails (because it can't possibly guess what my controlled vocabulary is) that's when I state in the error message what the controlled vocabulary is? "Sorry, you must use one of the following values: Arts and Humanities, ..." Seems messy to parse. I expect the answer to all this will be... don't even try this... the "S" in SWORD is for "simple". :) But I'm curious what people think. Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2014-05-01 20:47:55
|
Hi Richard et al., So sorry for the glacially slow reply. I'm not sure about the header idea. I haven't given it much thought, to be honest. I did want say, "Yes! Exactly!" to you saying, "the schema didn't even come close to fitting dcterms" because in integrating Dataverse (my side) and Open Journal Systems (their side) we agreed upon what I've been calling "the XML attribute" hack to pack more (desperately needed) information into a dcterms XML element. This is best illustrated by example, I believe. Sure we support run-of-the-mill dcterms elements like this... <dcterms:title>Roasting at Home</dcterms:title> <dcterms:creator>Peets, John</dcterms:creator> <dcterms:creator>Stumptown, Jane</dcterms:creator> <dcterms:publisher>Coffee Bean State University</dcterms:publisher> ... but in order to get the article citation from OJS into Dataverse, we special case the "dcterms:isReferencedBy" element, allowing OJS to put extra XML attributes (DDI stuff like holdingsURI, agency, and IDNo) into the XML element like this: <dcterms:isReferencedBy holdingsURI="http://dx.doi.org/10.1038/dvn333" agency="DOI" IDNo="10.1038/dvn333">Peets, J., & Stumptown, J. (2013). Roasting at Home. New England Journal of Coffee, 3(1), 22-34.</dcterms:isReferencedBy> It gets the job done AND I just learned today at http://irclog.iq.harvard.edu/dvn/2014-05-01#i_8982 that OJS got this "hack" merged into the PHP bindings for SWORD in this pull request: Allow attributes on dcterms elements in Atom entries by jwhitney · Pull Request #6 · swordapp/swordappv2-php-library - https://github.com/swordapp/swordappv2-php-library/pull/6 So maybe we (Dataverse and OJS) are not so off the mark with this... "interpretation" of SWORD... but I certainly feel like this isn't in the official spec and I don't know if this sort of thing is supported in any other client or server libraries. Phew! Glad to get that off my chest! :) Phil On Mon, Feb 10, 2014 at 1:42 PM, Richard Jones <ri...@co...> wrote: > Hi Phil, > > We have been doing this for the Duo project in Oslo. We're transferring > some custom student theses metadata from the national student system to the > repository, and the schema didn't even come close to fitting dcterms, so we > home-cooked our own (using dc where possible, but extending where > necessary). Here's my SwordEntryIngester that deals with the metadata-only > portion of the deposit (which is only a few fields): > > https://github.com/nye-duo/Duo-DSpace/blob/master/src/main/java/no/uio/duo/FSEntryIngester.java > > When doing a full package deposit we have a larger metadata file, which is > crosswalked just using an xslt to the native DSpace schema. > > So, not crazy - absolutely intended usage :) > > A thought that has been bashing around in my head for a while - we currently > have a way to describe what format the package being deposited is (via the > Packaging header), but no way to say what kind of metadata you are > depositing. From a server point of view, this means that if you support > multiple metadata formats, you need a bit of code that can look and try and > figure it out, rather than relying on an explicit cue from the deposit. But > would adding a header to give this information be viable/useful? > > Cheers, > > Richard > > > On 10 February 2014 17:50, Philip Durbin <phi...@ha...> wrote: >> >> Dear SWORDers, >> >> Sure, we all support dcterms, which is a simple key/value schema, in >> the Atom entry when creating a Resource. >> >> But what about going beyond dcterms? >> >> We are interested in supporting richer formats, some of which are >> hierarchical, such as DDI, as detailed below. >> >> Is anyone supporting anything other than dcterms when creating >> Resources? Is this crazy talk? >> >> Thanks in advance for the sanity check! >> >> Phil >> >> p.s. Here are the details of our situation: >> >> So far, according to the OJS Dataverse plugin testers surveyed with >> results recorded at >> >> https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodDJyd0pZdnlDZ3I5eWxnOHBmV1Q4dHc&usp=sharing >> the most commonly requested feature is the ability to customize which >> metadata fields are available as part of the data deposit form, which >> should be implemented in a future version. In order to support this, >> we will need to expand the API's metadata support beyond Dublin Core >> metadata. SWORD Protocol *should* be flexible enough for us to use >> other standards like DDI. At >> >> http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_creatingresource_entry >> the SWORDv2 spec says (emphasis added): >> >> > * The client *SHOULD add Dublin Core* [DublinCore] terms to the Atom >> > Entry as foreign markup (if appropriate); the terms MUST be embedded as >> > direct children of the atom:entry element, if present. >> > * The client *MAY add any other metadata formats or foreign markup* to >> > the atom:entry element >> >> We interpret this to mean that in addition to Dublin Core (dcterms, >> specifically), the SWORD spec is flexible enough to support wildly >> different metadata formats such as DataCite (https://www.datacite.org >> ), DDI (Data Documentation Initiative: http://www.ddialliance.org ), >> VO (Virtual Observatory: http://www.ivoa.net/documents/latest/RM.html >> ) ISA-Tab (Investigation, Study, and Assay in Tabular format: >> http://isatab.sourceforge.net/format.html ), etc. >> >> >From https://redmine.hmdc.harvard.edu/issues/3425 >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin >> >> >> ------------------------------------------------------------------------------ >> Android™ apps run on BlackBerry®10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com > -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2014-02-10 19:04:21
|
Hi Phil, We have been doing this for the Duo project in Oslo. We're transferring some custom student theses metadata from the national student system to the repository, and the schema didn't even come close to fitting dcterms, so we home-cooked our own (using dc where possible, but extending where necessary). Here's my SwordEntryIngester that deals with the metadata-only portion of the deposit (which is only a few fields): https://github.com/nye-duo/Duo-DSpace/blob/master/src/main/java/no/uio/duo/FSEntryIngester.java When doing a full package deposit we have a larger metadata file, which is crosswalked just using an xslt to the native DSpace schema. So, not crazy - absolutely intended usage :) A thought that has been bashing around in my head for a while - we currently have a way to describe what format the package being deposited is (via the Packaging header), but no way to say what kind of metadata you are depositing. From a server point of view, this means that if you support multiple metadata formats, you need a bit of code that can look and try and figure it out, rather than relying on an explicit cue from the deposit. But would adding a header to give this information be viable/useful? Cheers, Richard On 10 February 2014 17:50, Philip Durbin <phi...@ha...> wrote: > Dear SWORDers, > > Sure, we all support dcterms, which is a simple key/value schema, in > the Atom entry when creating a Resource. > > But what about going beyond dcterms? > > We are interested in supporting richer formats, some of which are > hierarchical, such as DDI, as detailed below. > > Is anyone supporting anything other than dcterms when creating > Resources? Is this crazy talk? > > Thanks in advance for the sanity check! > > Phil > > p.s. Here are the details of our situation: > > So far, according to the OJS Dataverse plugin testers surveyed with > results recorded at > > https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodDJyd0pZdnlDZ3I5eWxnOHBmV1Q4dHc&usp=sharing > the most commonly requested feature is the ability to customize which > metadata fields are available as part of the data deposit form, which > should be implemented in a future version. In order to support this, > we will need to expand the API's metadata support beyond Dublin Core > metadata. SWORD Protocol *should* be flexible enough for us to use > other standards like DDI. At > > http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_creatingresource_entry > the SWORDv2 spec says (emphasis added): > > > * The client *SHOULD add Dublin Core* [DublinCore] terms to the Atom > Entry as foreign markup (if appropriate); the terms MUST be embedded as > direct children of the atom:entry element, if present. > > * The client *MAY add any other metadata formats or foreign markup* to > the atom:entry element > > We interpret this to mean that in addition to Dublin Core (dcterms, > specifically), the SWORD spec is flexible enough to support wildly > different metadata formats such as DataCite (https://www.datacite.org > ), DDI (Data Documentation Initiative: http://www.ddialliance.org ), > VO (Virtual Observatory: http://www.ivoa.net/documents/latest/RM.html > ) ISA-Tab (Investigation, Study, and Assay in Tabular format: > http://isatab.sourceforge.net/format.html ), etc. > > >From https://redmine.hmdc.harvard.edu/issues/3425 > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > > ------------------------------------------------------------------------------ > Android™ apps run on BlackBerry®10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Philip D. <phi...@ha...> - 2014-02-10 18:16:23
|
Dear SWORDers, Sure, we all support dcterms, which is a simple key/value schema, in the Atom entry when creating a Resource. But what about going beyond dcterms? We are interested in supporting richer formats, some of which are hierarchical, such as DDI, as detailed below. Is anyone supporting anything other than dcterms when creating Resources? Is this crazy talk? Thanks in advance for the sanity check! Phil p.s. Here are the details of our situation: So far, according to the OJS Dataverse plugin testers surveyed with results recorded at https://docs.google.com/spreadsheet/ccc?key=0AjeLxEN77UZodDJyd0pZdnlDZ3I5eWxnOHBmV1Q4dHc&usp=sharing the most commonly requested feature is the ability to customize which metadata fields are available as part of the data deposit form, which should be implemented in a future version. In order to support this, we will need to expand the API's metadata support beyond Dublin Core metadata. SWORD Protocol *should* be flexible enough for us to use other standards like DDI. At http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_creatingresource_entry the SWORDv2 spec says (emphasis added): > * The client *SHOULD add Dublin Core* [DublinCore] terms to the Atom Entry as foreign markup (if appropriate); the terms MUST be embedded as direct children of the atom:entry element, if present. > * The client *MAY add any other metadata formats or foreign markup* to the atom:entry element We interpret this to mean that in addition to Dublin Core (dcterms, specifically), the SWORD spec is flexible enough to support wildly different metadata formats such as DataCite (https://www.datacite.org ), DDI (Data Documentation Initiative: http://www.ddialliance.org ), VO (Virtual Observatory: http://www.ivoa.net/documents/latest/RM.html ) ISA-Tab (Investigation, Study, and Assay in Tabular format: http://isatab.sourceforge.net/format.html ), etc. >From https://redmine.hmdc.harvard.edu/issues/3425 -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: George H. <geo...@ed...> - 2014-01-31 13:17:16
|
Hi Phil You'll find your key in your profile, you'll need an EASE login though. I've got some notes here: https://www.wiki.ed.ac.uk/display/datashare/sword2 Regards George On 31/01/14 13:10, Philip Durbin wrote: > Thanks, Stuart, this is very interesting. I spent a couple minutes at > http://datashare.is.ed.ac.uk but I couldn't quickly determine how a > users could discover their randomly generated API key in case they > want to deposit via something somewhat low level like curl or a script > that makes use of language bindings provided at > https://github.com/swordapp > > Or am I thinking about this the wrong way? If the API key is a proxy > for my password, I need it (and my username) to make a deposit via > SWORD. Maybe my user account page would simply expose the API key to > me and I could reset it if my key were ever compromised. > > Phil > > On Mon, Jan 27, 2014 at 3:18 PM, LEWIS Stuart <Stu...@ed...> wrote: >> Thanks for sharing this Phil. >> >> Interestingly today I found someone else that has done exactly the same >> thing! http://datashare.is.ed.ac.uk/ is the University of Edinburgh¹s >> DSpace data repository. It uses a single-sign-on system, which obviously >> doesn¹t work well with things like SWORD. >> >> To get around this, the developer George Hamilton (cc¹d) has added a >> randomly generated API key to each users¹ profile, and this is used as a >> proxy for a password for SWORD deposits. >> >> A neat solution, and sits well alongside similar API key configurations >> for web-based systems. >> >> Thanks, >> >> >> Stuart Lewis >> Head of Research and Learning Services >> Deputy Director Library & University Collections, Information Services >> University of Edinburgh >> Stu...@ed... >> >> >> >> >> >> On 27/01/2014 19:25, "Philip Durbin" <phi...@ha...> wrote: >> >> Um. Sorry for the late reply. :) >> >> On the topic of API keys and SWORD, this just came across my radar: >> >> 'Enter the dashboard's IP address into the "Remote name" field and the >> user and API key noted earlier into the "Api username" and "Api key" >> fields' -- https://www.archivematica.org/wiki/Sword_API#Configuration >> >> Very interesting. >> >> Phil >> >> On Thu, Aug 1, 2013 at 2:49 PM, Richard Jones <ri...@co...> >> wrote: >>> Nope, no attempt to use OAuth with SWORD that I'm aware of. We toyed >>> with trying to do this as part of the protocol, and then decided that >>> it was Too Hard, might put people off implementing, and also ought to >>> be orthogonal to the task that sword is trying to carry out, so we >>> decided to leave it up to implementers to decide. >>> >>> Do you think that any modifications to sword are required in order to >>> permit OAuth? Our principal during development was to make sure we >>> didn't do anything which prevented such things, but I would like some >>> confirmation that we succeeded! >>> >>> Cheers, >>> >>> Richard >>> >>> On 1 August 2013 17:08, Philip Durbin <phi...@ha...> wrote: >>>> On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin >>>> <phi...@ha...> wrote: >>>>> Does anyone's SWORDv2 implementation use API keys (negotiated via >>>>> OAuth, maybe?) rather than HTTP Basic Authentication >>>>> (username/password) in conjunction with a TLS connection? >>>> It looks like Richard asked a similar question here: >>>> >>>> Re: [Sword-TAP] on-behalf-of vs. OAuth - >>>> >>>> http://www.mail-archive.com/swo...@li...urceforge >>>> .net/msg00141.html >>>> >>>> >From what I can tell, no one has implemented OAuth or similar with >>>> SWORD yet. >>>> >>>> Sorry for not noticing this post earlier. >>>> >>>> Phil >>>> >>>> -- >>>> Philip Durbin >>>> Software Developer for http://thedata.org >>>> http://www.iq.harvard.edu/people/philip-durbin >>>> >>>> >>>> ------------------------------------------------------------------------- >>>> ----- >>>> Get your SQL database under version control now! >>>> Version control is standard for application code, but databases havent >>>> caught up. So what steps can you take to put your SQL databases under >>>> version control? Why should you start doing it? Read more to find out. >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clkt >>>> rk >>>> _______________________________________________ >>>> sword-app-tech mailing list >>>> swo...@li... >>>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >>> >>> >>> -- >>> >>> Richard Jones, >>> >>> Founder, Cottage Labs >>> t: @richard_d_jones, @cottagelabs >>> w: http://cottagelabs.com >> >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin >> >> --------------------------------------------------------------------------- >> --- >> CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> Learn Why More Businesses Are Choosing CenturyLink Cloud For >> Critical Workloads, Development Environments & Everything In Between. >> Get a Quote or Start a Free Trial Today. >> http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktr >> k >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: Philip D. <phi...@ha...> - 2014-01-31 13:10:46
|
Thanks, Stuart, this is very interesting. I spent a couple minutes at http://datashare.is.ed.ac.uk but I couldn't quickly determine how a users could discover their randomly generated API key in case they want to deposit via something somewhat low level like curl or a script that makes use of language bindings provided at https://github.com/swordapp Or am I thinking about this the wrong way? If the API key is a proxy for my password, I need it (and my username) to make a deposit via SWORD. Maybe my user account page would simply expose the API key to me and I could reset it if my key were ever compromised. Phil On Mon, Jan 27, 2014 at 3:18 PM, LEWIS Stuart <Stu...@ed...> wrote: > Thanks for sharing this Phil. > > Interestingly today I found someone else that has done exactly the same > thing! http://datashare.is.ed.ac.uk/ is the University of Edinburgh¹s > DSpace data repository. It uses a single-sign-on system, which obviously > doesn¹t work well with things like SWORD. > > To get around this, the developer George Hamilton (cc¹d) has added a > randomly generated API key to each users¹ profile, and this is used as a > proxy for a password for SWORD deposits. > > A neat solution, and sits well alongside similar API key configurations > for web-based systems. > > Thanks, > > > Stuart Lewis > Head of Research and Learning Services > Deputy Director Library & University Collections, Information Services > University of Edinburgh > Stu...@ed... > > > > > > On 27/01/2014 19:25, "Philip Durbin" <phi...@ha...> wrote: > > Um. Sorry for the late reply. :) > > On the topic of API keys and SWORD, this just came across my radar: > > 'Enter the dashboard's IP address into the "Remote name" field and the > user and API key noted earlier into the "Api username" and "Api key" > fields' -- https://www.archivematica.org/wiki/Sword_API#Configuration > > Very interesting. > > Phil > > On Thu, Aug 1, 2013 at 2:49 PM, Richard Jones <ri...@co...> > wrote: >> Nope, no attempt to use OAuth with SWORD that I'm aware of. We toyed >> with trying to do this as part of the protocol, and then decided that >> it was Too Hard, might put people off implementing, and also ought to >> be orthogonal to the task that sword is trying to carry out, so we >> decided to leave it up to implementers to decide. >> >> Do you think that any modifications to sword are required in order to >> permit OAuth? Our principal during development was to make sure we >> didn't do anything which prevented such things, but I would like some >> confirmation that we succeeded! >> >> Cheers, >> >> Richard >> >> On 1 August 2013 17:08, Philip Durbin <phi...@ha...> wrote: >>> On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin >>> <phi...@ha...> wrote: >>>> Does anyone's SWORDv2 implementation use API keys (negotiated via >>>> OAuth, maybe?) rather than HTTP Basic Authentication >>>> (username/password) in conjunction with a TLS connection? >>> >>> It looks like Richard asked a similar question here: >>> >>> Re: [Sword-TAP] on-behalf-of vs. OAuth - >>> >>>http://www.mail-archive.com/swo...@li...urceforge >>>.net/msg00141.html >>> >>> >From what I can tell, no one has implemented OAuth or similar with >>>SWORD yet. >>> >>> Sorry for not noticing this post earlier. >>> >>> Phil >>> >>> -- >>> Philip Durbin >>> Software Developer for http://thedata.org >>> http://www.iq.harvard.edu/people/philip-durbin >>> >>> >>>------------------------------------------------------------------------- >>>----- >>> Get your SQL database under version control now! >>> Version control is standard for application code, but databases havent >>> caught up. So what steps can you take to put your SQL databases under >>> version control? Why should you start doing it? Read more to find out. >>> >>>http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clkt >>>rk >>> _______________________________________________ >>> sword-app-tech mailing list >>> swo...@li... >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> >> >> -- >> >> Richard Jones, >> >> Founder, Cottage Labs >> t: @richard_d_jones, @cottagelabs >> w: http://cottagelabs.com > > > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > --------------------------------------------------------------------------- > --- > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktr > k > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Joerg M. <joe...@ub...> - 2014-01-30 20:30:41
|
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Greetings,<br>
<br>
I'm trying to get my head around how best to use Sword v2 to upload
a large lecture video collection to our circle.ubc.ca IR. My
initial attempt was to use the Python client implementation which
seemed to work well when I tested it against the demo.dspace.org
DSpace 4 server. I was hoping that our DSpace 3.2 servers would
work just as well but that was not the case. I couldn't even get
access to the service document. I don't know if it's an issue with
the Python client or DSpace server Sword implementation. I haven't
been able to find any specific references to this issue on the Net.<br>
<br>
I'm now trying to prepare the upload package directly but I'm having
some issues with the metadata. I've successfully loaded some sample
data using curl which is encouraging but I was wondering if there
was a way to map the metadata in the METS manifest file to support
QDC? It does appear to support DC, MODS and EPDCX but I have a few
fields for which I could not find a mapping. Here's a sample of the
metadata that we're trying to upload. <br>
<br>
<pre wrap="">{
"dc.contributor.author" : "Carlsson, Gunnar"
"dc.title" : "Opening remarks: State of TDA"
"dc.type" : "Moving Image"
"dc.date.issued" : "2012-10-15T09:03"
"dc.language.iso" : "eng"
"dc.description.abstract" : ""
"dc.description.affiliation" : "Stanford University"
"dc.description.scholarlevel" : "Faculty"
"dc.format.extent" : "43 minutes"
"dc.publisher" : "Banff International Research Station"
"dc.description.reviewstatus" : "Unreviewed"
"dc.relation" : "12w5081: Topological data analysis and machine learning theory"
"dc.subject" : "Data mining"
}
</pre>
Any thoughts would be most welcome. Sword seems to be a great tool
when it works but it does have a bit of a learning curve.<br>
<br>
<div class="moz-signature">-- <br>
<div style="font-weight:normal;font-size:9pt">
<span style="font-weight:900;font-size:10pt;letter-spacing:4px">Joerg
Messer</span><br>
Programmer Analyst<br>
University of British Columbia<br>
Library Systems and Information Technology | Data Services<br>
IKBLC, 1961 East Mall, Vancouver, BC, Canada, V6T 1Z1<br>
Office 1.604.822.5091 | Mobile 1.604.374.2739<br>
<a class="moz-txt-link-abbreviated" href="mailto:joe...@ub...">joe...@ub...</a><br>
</div>
</div>
</body>
</html>
|
|
From: LEWIS S. <Stu...@ed...> - 2014-01-27 20:18:20
|
Thanks for sharing this Phil. Interestingly today I found someone else that has done exactly the same thing! http://datashare.is.ed.ac.uk/ is the University of Edinburgh¹s DSpace data repository. It uses a single-sign-on system, which obviously doesn¹t work well with things like SWORD. To get around this, the developer George Hamilton (cc¹d) has added a randomly generated API key to each users¹ profile, and this is used as a proxy for a password for SWORD deposits. A neat solution, and sits well alongside similar API key configurations for web-based systems. Thanks, Stuart Lewis Head of Research and Learning Services Deputy Director Library & University Collections, Information Services University of Edinburgh Stu...@ed... On 27/01/2014 19:25, "Philip Durbin" <phi...@ha...> wrote: Um. Sorry for the late reply. :) On the topic of API keys and SWORD, this just came across my radar: 'Enter the dashboard's IP address into the "Remote name" field and the user and API key noted earlier into the "Api username" and "Api key" fields' -- https://www.archivematica.org/wiki/Sword_API#Configuration Very interesting. Phil On Thu, Aug 1, 2013 at 2:49 PM, Richard Jones <ri...@co...> wrote: > Nope, no attempt to use OAuth with SWORD that I'm aware of. We toyed > with trying to do this as part of the protocol, and then decided that > it was Too Hard, might put people off implementing, and also ought to > be orthogonal to the task that sword is trying to carry out, so we > decided to leave it up to implementers to decide. > > Do you think that any modifications to sword are required in order to > permit OAuth? Our principal during development was to make sure we > didn't do anything which prevented such things, but I would like some > confirmation that we succeeded! > > Cheers, > > Richard > > On 1 August 2013 17:08, Philip Durbin <phi...@ha...> wrote: >> On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin >> <phi...@ha...> wrote: >>> Does anyone's SWORDv2 implementation use API keys (negotiated via >>> OAuth, maybe?) rather than HTTP Basic Authentication >>> (username/password) in conjunction with a TLS connection? >> >> It looks like Richard asked a similar question here: >> >> Re: [Sword-TAP] on-behalf-of vs. OAuth - >> >>http://www.mail-archive.com/swo...@li...urceforge >>.net/msg00141.html >> >> >From what I can tell, no one has implemented OAuth or similar with >>SWORD yet. >> >> Sorry for not noticing this post earlier. >> >> Phil >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin >> >> >>------------------------------------------------------------------------- >>----- >> Get your SQL database under version control now! >> Version control is standard for application code, but databases havent >> caught up. So what steps can you take to put your SQL databases under >> version control? Why should you start doing it? Read more to find out. >> >>http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clkt >>rk >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin --------------------------------------------------------------------------- --- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktr k _______________________________________________ sword-app-tech mailing list swo...@li... https://lists.sourceforge.net/lists/listinfo/sword-app-tech The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: Philip D. <phi...@ha...> - 2014-01-27 19:52:06
|
Um. Sorry for the late reply. :) On the topic of API keys and SWORD, this just came across my radar: 'Enter the dashboard's IP address into the "Remote name" field and the user and API key noted earlier into the "Api username" and "Api key" fields' -- https://www.archivematica.org/wiki/Sword_API#Configuration Very interesting. Phil On Thu, Aug 1, 2013 at 2:49 PM, Richard Jones <ri...@co...> wrote: > Nope, no attempt to use OAuth with SWORD that I'm aware of. We toyed > with trying to do this as part of the protocol, and then decided that > it was Too Hard, might put people off implementing, and also ought to > be orthogonal to the task that sword is trying to carry out, so we > decided to leave it up to implementers to decide. > > Do you think that any modifications to sword are required in order to > permit OAuth? Our principal during development was to make sure we > didn't do anything which prevented such things, but I would like some > confirmation that we succeeded! > > Cheers, > > Richard > > On 1 August 2013 17:08, Philip Durbin <phi...@ha...> wrote: >> On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin >> <phi...@ha...> wrote: >>> Does anyone's SWORDv2 implementation use API keys (negotiated via >>> OAuth, maybe?) rather than HTTP Basic Authentication >>> (username/password) in conjunction with a TLS connection? >> >> It looks like Richard asked a similar question here: >> >> Re: [Sword-TAP] on-behalf-of vs. OAuth - >> http://www.mail-archive.com/swo...@li.../msg00141.html >> >> >From what I can tell, no one has implemented OAuth or similar with SWORD yet. >> >> Sorry for not noticing this post earlier. >> >> Phil >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin >> >> ------------------------------------------------------------------------------ >> Get your SQL database under version control now! >> Version control is standard for application code, but databases havent >> caught up. So what steps can you take to put your SQL databases under >> version control? Why should you start doing it? Read more to find out. >> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Pottinger, H. J. <Pot...@mi...> - 2014-01-03 16:19:32
|
Hi, I flagged this thread before the holidays, intending to come back to it. Thanks for the reminder, Tim, I'm afraid I flag a lot of email with good intentions, and don't always follow up on those intentions. But, enough of regrets, it's a new year! Nate, your AbstractBagitIngester.java (see [2] down-thread a bit) modifications look great, I plan to borrow them for my repository, and I thought I'd ask if you'd consider submitting a Pull Request for this work? I've taken the liberty of creating a JIRA issue for the Bagit package ingester (see https://jira.duraspace.org/browse/DS-1852), but we would need to create another issue for the community and collection deposit work you did at the same time -- I want to see that added, too. :-) Can you describe that work for me, so I can write a JIRA issue for it, or, even better, you can create a JIRA issue for the work yourself. https://jira.duraspace.org/secure/CreateIssue!default.jspa The topic of being able to load batches of items via SWORD to DSpace is one that comes up from time to time, and was a very hot topic of discussion at OR11 in Austin, if memory serves. See also DS-1641 (https://jira.duraspace.org/browse/DS-1641). As soon as a JIRA issue for the collection and community deposit features exists, you can submit a Pull Request for this work. Check out the Code Contribution Guidelines for more details: https://wiki.duraspace.org/display/DSPACE/Code+Contribution+Guidelines#Code ContributionGuidelines-1.Makeyourcodeavailable(preferrablyinGitHub)andcreat eaticketinourIssueTracker If you need help with any of this, let me (or any other DSpace committer) know. We are happy to help. -- HARDY POTTINGER <pot...@um...> University of Missouri Library Systems http://lso.umsystem.edu/~pottingerhj/ https://MOspace.umsystem.edu/ "What day is it?" "It's today," squeaked Piglet. "My favorite day," said Pooh. On 1/3/14 9:08 AM, "Tim Donohue" <tdo...@du...> wrote: >Just a few minor clarifications.. > >The BagIt code for Replication Task Suite was actually written by >Richard Rodgers at MIT. I know it a little, but Richard R knows it even >better. :) > >Also, the Replication Task Suite code supports both METS and BagIt based >AIPs. The METS format is more built out/widely used, and actually >*should* work fine with SWORD (though it needs a SWORD ingest plugin >still). The ReplicationTaskSuite BagIt code is still more of "beta" >code, so it may need some updates/modifications to work better with >SWORD deposits. However, as mentioned, Richard R & I would welcome >anyone to send us Pull Requests or patches to improve the Replication >Task Suite: >https://github.com/DSpace/dspace-replicate > >In any case, I hope that everyone continues to post their work publicly >so that we can all take advantage and potentially work together on a >shared solution. > >Thanks, >- Tim > >On 12/26/2013 2:45 PM, Nathaniel Ryckman wrote: >> Hi Bill, >> >> During last summer, I also was led to believe the only significant work >> done on a Sword Baggit Ingester was performed by Tim Donohue and Richard >> Jones. Dryad appeared to have had plans to work on the system, but I did >> not find any code produced by them. >> >> I ended up testing the possibility of using Tim Donohue's >> ReplicationTaskSuite and discovered that it was specifically designed to >> restore data that had already been backed up and would not easily be >> modified to accept deposits from SWORD [1]. >> >> I, however, used Tim's code as a model to build an >> AbstractBagitIngester. The system allowed me to conform to SWORD v1.0 >> standards. I was able to deposit items using the system modifications >> that I wrote. If you are interested, see my AbstractBagitIngester for a >> good starting point [2]. >> >> If I remember correctly, I was asked to build an ingester that would >> depend solely on file hierarchy and Baggit standards rather than a >> manifest file, so I do not think I have support for OAI-ORE manifests. >> >> Anyway, good luck! If you have any questions, feel free to ask. >> >> Sincerely, >> Nate R. >> >> [1] >> >>http://dspace.2283337.n4.nabble.com/batch-import-of-Bagit-formated-collec >>tions-and-or-conversion-script-for-Bagit-to-SAF-td3697073.html#a4665944 >> [2] >> >>https://github.com/nryckman/DSpace-3.1/blob/master/dspace-api/src/main/ja >>va/org/dspace/content/packager/AbstractBagitIngester.java >> >> >> On Thu, Dec 19, 2013 at 11:53 AM, Ingram, William A >> <win...@il... <mailto:win...@il...>> wrote: >> >> Thanks for the feedback, guys. I'll report back once I've got >> something to look at. >> >> Thanks! >> Bill >> >> Bill Ingram >> University of Illinois >> Urbana, IL 61801 >> 217-333-4648 <tel:217-333-4648> >> win...@il... <mailto:win...@il...> >> >> -----Original Message----- >> From: Tim Donohue [mailto:tdo...@du... >> <mailto:tdo...@du...>] >> Sent: Thursday, December 19, 2013 9:21 AM >> To: Ingram, William A >> Cc: Richard Jones; swo...@li... >> <mailto:swo...@li...> >> Subject: Re: [sword-app-tech] DSpace SWORD deposit using BagIt and >> OAI:ORE >> >> Hi Bill, >> >> There's also some DSpace + BagIt work in the Replication Task Suite >> addon which can generate Archival Information Packages (AIPs), >> either METS-based AIPs or BagIt-based AIPs, for all content within >> DSpace. >> These AIPs can be used to backup/restore DSpace or copy/move content >> between DSpace instances. However, it does not involve OAI-ORE at >> this time. >> >> It also is not currently connected up to DSpace's SWORD server. So, >> you cannot currently deposit these AIPs via SWORD. Though, since >> DSpace's SWORD server is configurable, I suspect it should not be >> difficult to configure it to accept these AIPs (likely just need a >> simple SWORD plugin to call the Replication Task Suite code). This >> likely would be something that others would find useful as well. >> >> Overview: >>https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite >> Code at: https://github.com/DSpace/dspace-replicate >> >> In any case, whatever work you do, I do suggest trying to post your >> code up in a public area (GitHub or similar) and making others aware >> of it. I keep hearing of folks that are interested in DSpace + >> BagIt, but the projects are not easy to locate. So, if you are >> looking for more collaborators, you also may wish to post about this >> on the dspace-devel mailing list, and/or create a wiki page in the >> DSpace Wiki. >> >> Good luck, >> >> Tim >> >> >> On 12/19/2013 8:18 AM, Richard Jones wrote: >> > I'm not aware of any work on DSpace with BagIt/OAI-ORE, although >>I >> > have done some BagIt package handling for a project I did with >>the >> > University of Oslo using a custom metadata format. >> > >> > Here's an example of the bagit format we were working with: >> > https://github.com/nye-duo/Duo-DSpace/tree/master/bagit >> > >> > Here's the SWORD plugin which deals with the bagit: >> > >> >>https://github.com/nye-duo/Duo-DSpace/blob/master/src/main/java/no/uio >> > /duo/FSBagItIngester.java >> > >> > And we also wrote a BagItLibrary to wrap our particular profile >> of the >> > format, so that it could be packed and unpacked at either end in >>a >> > consistent way: https://github.com/nye-duo/BagItLibrary >> > >> > Don't know if any of that is trivially re-usable, but hopefully >>it >> > will be helpful. >> > >> > Cheers, >> > >> > Richard >> > >> > >> > >> > >> > On 18 December 2013 19:30, Ingram, William A >> <win...@il... <mailto:win...@il...> >> > <mailto:win...@il... <mailto:win...@il...>>> >>wrote: >> > >> > I'm currently working on a research project investigating >> expanding >> > the capabilities of the IDEALS (DSpace) repository, allowing >>for >> > deposit of objects wrapped in BagIt packages and containing >> OAI-ORE >> > manifests.____ >> > >> > __ __ >> > >> > I saw online that there was a Summer of Code request for >> this, but >> > no one picked up on it. ____ >> > >> > __ __ >> > >> > Has anyone tried this? I don't want to repeat work that >> someone has >> > already done. ____ >> > >> > __ __ >> > >> > I wrote to the Dryad list, and based on the responses I got, >> this is >> > still something that people want. ____ >> > >> > __ __ >> > >> > Thanks!____ >> > >> > Bill____ >> >> >>------------------------------------------------------------------------- >>----- >> Rapidly troubleshoot problems before they affect your business. >>Most IT >> organizations don't have a clear picture of how application >>performance >> affects their revenue. With AppDynamics, you get 100% visibility >> into your >> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of >> AppDynamics Pro! >> >>http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clkt >>rk >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> <mailto:swo...@li...> >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> >> >> >> >> >>------------------------------------------------------------------------- >>----- >> Rapidly troubleshoot problems before they affect your business. Most IT >> organizations don't have a clear picture of how application performance >> affects their revenue. With AppDynamics, you get 100% visibility into >>your >> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of >>AppDynamics Pro! >> >>http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clkt >>rk >> >> >> >> _______________________________________________ >> sword-app-tech mailing list >> swo...@li... >> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> > >-------------------------------------------------------------------------- >---- >Rapidly troubleshoot problems before they affect your business. Most IT >organizations don't have a clear picture of how application performance >affects their revenue. With AppDynamics, you get 100% visibility into >your >Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics >Pro! >http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktr >k >_______________________________________________ >sword-app-tech mailing list >swo...@li... >https://lists.sourceforge.net/lists/listinfo/sword-app-tech |
|
From: Tim D. <tdo...@du...> - 2014-01-03 15:08:48
|
Just a few minor clarifications.. The BagIt code for Replication Task Suite was actually written by Richard Rodgers at MIT. I know it a little, but Richard R knows it even better. :) Also, the Replication Task Suite code supports both METS and BagIt based AIPs. The METS format is more built out/widely used, and actually *should* work fine with SWORD (though it needs a SWORD ingest plugin still). The ReplicationTaskSuite BagIt code is still more of "beta" code, so it may need some updates/modifications to work better with SWORD deposits. However, as mentioned, Richard R & I would welcome anyone to send us Pull Requests or patches to improve the Replication Task Suite: https://github.com/DSpace/dspace-replicate In any case, I hope that everyone continues to post their work publicly so that we can all take advantage and potentially work together on a shared solution. Thanks, - Tim On 12/26/2013 2:45 PM, Nathaniel Ryckman wrote: > Hi Bill, > > During last summer, I also was led to believe the only significant work > done on a Sword Baggit Ingester was performed by Tim Donohue and Richard > Jones. Dryad appeared to have had plans to work on the system, but I did > not find any code produced by them. > > I ended up testing the possibility of using Tim Donohue's > ReplicationTaskSuite and discovered that it was specifically designed to > restore data that had already been backed up and would not easily be > modified to accept deposits from SWORD [1]. > > I, however, used Tim's code as a model to build an > AbstractBagitIngester. The system allowed me to conform to SWORD v1.0 > standards. I was able to deposit items using the system modifications > that I wrote. If you are interested, see my AbstractBagitIngester for a > good starting point [2]. > > If I remember correctly, I was asked to build an ingester that would > depend solely on file hierarchy and Baggit standards rather than a > manifest file, so I do not think I have support for OAI-ORE manifests. > > Anyway, good luck! If you have any questions, feel free to ask. > > Sincerely, > Nate R. > > [1] > http://dspace.2283337.n4.nabble.com/batch-import-of-Bagit-formated-collections-and-or-conversion-script-for-Bagit-to-SAF-td3697073.html#a4665944 > [2] > https://github.com/nryckman/DSpace-3.1/blob/master/dspace-api/src/main/java/org/dspace/content/packager/AbstractBagitIngester.java > > > On Thu, Dec 19, 2013 at 11:53 AM, Ingram, William A > <win...@il... <mailto:win...@il...>> wrote: > > Thanks for the feedback, guys. I'll report back once I've got > something to look at. > > Thanks! > Bill > > Bill Ingram > University of Illinois > Urbana, IL 61801 > 217-333-4648 <tel:217-333-4648> > win...@il... <mailto:win...@il...> > > -----Original Message----- > From: Tim Donohue [mailto:tdo...@du... > <mailto:tdo...@du...>] > Sent: Thursday, December 19, 2013 9:21 AM > To: Ingram, William A > Cc: Richard Jones; swo...@li... > <mailto:swo...@li...> > Subject: Re: [sword-app-tech] DSpace SWORD deposit using BagIt and > OAI:ORE > > Hi Bill, > > There's also some DSpace + BagIt work in the Replication Task Suite > addon which can generate Archival Information Packages (AIPs), > either METS-based AIPs or BagIt-based AIPs, for all content within > DSpace. > These AIPs can be used to backup/restore DSpace or copy/move content > between DSpace instances. However, it does not involve OAI-ORE at > this time. > > It also is not currently connected up to DSpace's SWORD server. So, > you cannot currently deposit these AIPs via SWORD. Though, since > DSpace's SWORD server is configurable, I suspect it should not be > difficult to configure it to accept these AIPs (likely just need a > simple SWORD plugin to call the Replication Task Suite code). This > likely would be something that others would find useful as well. > > Overview: https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite > Code at: https://github.com/DSpace/dspace-replicate > > In any case, whatever work you do, I do suggest trying to post your > code up in a public area (GitHub or similar) and making others aware > of it. I keep hearing of folks that are interested in DSpace + > BagIt, but the projects are not easy to locate. So, if you are > looking for more collaborators, you also may wish to post about this > on the dspace-devel mailing list, and/or create a wiki page in the > DSpace Wiki. > > Good luck, > > Tim > > > On 12/19/2013 8:18 AM, Richard Jones wrote: > > I'm not aware of any work on DSpace with BagIt/OAI-ORE, although I > > have done some BagIt package handling for a project I did with the > > University of Oslo using a custom metadata format. > > > > Here's an example of the bagit format we were working with: > > https://github.com/nye-duo/Duo-DSpace/tree/master/bagit > > > > Here's the SWORD plugin which deals with the bagit: > > > https://github.com/nye-duo/Duo-DSpace/blob/master/src/main/java/no/uio > > /duo/FSBagItIngester.java > > > > And we also wrote a BagItLibrary to wrap our particular profile > of the > > format, so that it could be packed and unpacked at either end in a > > consistent way: https://github.com/nye-duo/BagItLibrary > > > > Don't know if any of that is trivially re-usable, but hopefully it > > will be helpful. > > > > Cheers, > > > > Richard > > > > > > > > > > On 18 December 2013 19:30, Ingram, William A > <win...@il... <mailto:win...@il...> > > <mailto:win...@il... <mailto:win...@il...>>> wrote: > > > > I'm currently working on a research project investigating > expanding > > the capabilities of the IDEALS (DSpace) repository, allowing for > > deposit of objects wrapped in BagIt packages and containing > OAI-ORE > > manifests.____ > > > > __ __ > > > > I saw online that there was a Summer of Code request for > this, but > > no one picked up on it. ____ > > > > __ __ > > > > Has anyone tried this? I don't want to repeat work that > someone has > > already done. ____ > > > > __ __ > > > > I wrote to the Dryad list, and based on the responses I got, > this is > > still something that people want. ____ > > > > __ __ > > > > Thanks!____ > > > > Bill____ > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility > into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of > AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > <mailto:swo...@li...> > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > > > > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > |
13 messages has been excluded from this view by a project administrator.
