Menu
▾
▴
sword-app-tech — Mailing list for discussion of SWORD implemenation and development.
You can subscribe to this list here.
| 2008 |
Jan
(22) |
Feb
(8) |
Mar
(9) |
Apr
(4) |
May
(17) |
Jun
(29) |
Jul
(11) |
Aug
(13) |
Sep
(17) |
Oct
(14) |
Nov
(41) |
Dec
(8) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2009 |
Jan
(17) |
Feb
(26) |
Mar
(18) |
Apr
(1) |
May
(11) |
Jun
(20) |
Jul
(5) |
Aug
|
Sep
|
Oct
|
Nov
(3) |
Dec
|
| 2010 |
Jan
(23) |
Feb
(7) |
Mar
(9) |
Apr
(13) |
May
(5) |
Jun
|
Jul
(3) |
Aug
(5) |
Sep
|
Oct
(1) |
Nov
(3) |
Dec
|
| 2011 |
Jan
(3) |
Feb
|
Mar
(2) |
Apr
(1) |
May
|
Jun
(14) |
Jul
(22) |
Aug
(1) |
Sep
(2) |
Oct
(11) |
Nov
(11) |
Dec
(35) |
| 2012 |
Jan
(17) |
Feb
(12) |
Mar
(41) |
Apr
(40) |
May
(41) |
Jun
(27) |
Jul
(9) |
Aug
(1) |
Sep
|
Oct
(6) |
Nov
|
Dec
(11) |
| 2013 |
Jan
|
Feb
(4) |
Mar
(2) |
Apr
(8) |
May
(1) |
Jun
(18) |
Jul
(10) |
Aug
(16) |
Sep
(2) |
Oct
(1) |
Nov
(14) |
Dec
(11) |
| 2014 |
Jan
(7) |
Feb
(2) |
Mar
|
Apr
|
May
(8) |
Jun
(1) |
Jul
(7) |
Aug
(10) |
Sep
(8) |
Oct
(8) |
Nov
|
Dec
|
| 2015 |
Jan
|
Feb
(2) |
Mar
(2) |
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2016 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(4) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2017 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(6) |
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
|
Dec
(4) |
| 2018 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Philip D. <phi...@ha...> - 2013-10-10 12:47:27
|
I'm not sure if there is an R package for SWORD but I wanted to point out that https://github.com/leeper/dvn was recently updated to support SWORDv2 operations. As you might guess from the name, this "dvn" R package works specifically with Dataverse Network, but someone might be interested in abstracting out the SWORDv2 functionality to make a generic R package for SWORDv2 and put it on CRAN. Phil ---------- Forwarded message ---------- From: Philip Durbin <phi...@ha...> Date: Thu, Oct 10, 2013 at 8:39 AM Subject: Re: Re: [sword-app-tech] Dataverse Network (DVN) 3.6 in production as a SWORDv2 server To: "dat...@go..." <dat...@go...> And for the R programmers... https://github.com/leeper/dvn/issues/1 was recently closed so if you pull down the latest version with devtools like this: install_github(repo = 'dvn', username = 'leeper') You can start playing around with DVN's new Data Deposit API from within R! Phil On Sat, Sep 21, 2013 at 11:46 AM, Philip Durbin <phi...@ha...> wrote: > > Woo-hoo! > > ---------- Forwarded message ---------- > From: "LEWIS Stuart" <Stu...@ed...> > Date: Sep 21, 2013 8:46 AM > Subject: Re: [sword-app-tech] Dataverse Network (DVN) 3.6 in production as a SWORDv2 server > To: "Philip Durbin" <phi...@ha...>, "<swo...@li...>" <swo...@li...> > > Thanks Phil - I've now updated the web page as suggested: > > - http://swordapp.org/sword-v2/sword-v2-implementations/ > > Thanks, > > > Stuart Lewis > Head of Research and Learning Services > Deputy Director Library & University Collections, Information Services > University of Edinburgh > Stu...@ed... > > > > > > On 20/09/2013 20:40, "Philip Durbin" <phi...@ha...> wrote: > > I'm writing to announce that http://thedata.harvard.edu/dvn/ has been > upgraded to DVN 3.6, making it a SWORDv2 server. > > Our docs are at > http://thedata.harvard.edu/guides/dataverse-api-main.html#data-deposit-api > > If you'd like to list it at > http://swordapp.org/sword-v2/sword-v2-implementations/ please be my > guest. I think we are fairly compliant with the SWORD v2 spec but I > welcome feedback either way. Our code is at > https://github.com/IQSS/dvn > > Thank you to everyone on this list for the help over the last few > months. I really appreciate it! > > Phil > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > --------------------------------------------------------------------------- > --- > LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! > 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, > SharePoint > 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack > includes > Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. > http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: LEWIS S. <Stu...@ed...> - 2013-09-21 13:46:16
|
Thanks Phil - I've now updated the web page as suggested: - http://swordapp.org/sword-v2/sword-v2-implementations/ Thanks, Stuart Lewis Head of Research and Learning Services Deputy Director Library & University Collections, Information Services University of Edinburgh Stu...@ed... On 20/09/2013 20:40, "Philip Durbin" <phi...@ha...> wrote: I'm writing to announce that http://thedata.harvard.edu/dvn/ has been upgraded to DVN 3.6, making it a SWORDv2 server. Our docs are at http://thedata.harvard.edu/guides/dataverse-api-main.html#data-deposit-api If you'd like to list it at http://swordapp.org/sword-v2/sword-v2-implementations/ please be my guest. I think we are fairly compliant with the SWORD v2 spec but I welcome feedback either way. Our code is at https://github.com/IQSS/dvn Thank you to everyone on this list for the help over the last few months. I really appreciate it! Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin --------------------------------------------------------------------------- --- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk _______________________________________________ sword-app-tech mailing list swo...@li... https://lists.sourceforge.net/lists/listinfo/sword-app-tech The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: Philip D. <phi...@ha...> - 2013-09-20 19:40:20
|
I'm writing to announce that http://thedata.harvard.edu/dvn/ has been upgraded to DVN 3.6, making it a SWORDv2 server. Our docs are at http://thedata.harvard.edu/guides/dataverse-api-main.html#data-deposit-api If you'd like to list it at http://swordapp.org/sword-v2/sword-v2-implementations/ please be my guest. I think we are fairly compliant with the SWORD v2 spec but I welcome feedback either way. Our code is at https://github.com/IQSS/dvn Thank you to everyone on this list for the help over the last few months. I really appreciate it! Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-22 16:21:35
|
On Thu, Jun 20, 2013 at 3:22 AM, LEWIS Stuart <Stu...@ed...> wrote: > Thanks for sharing this. The JavaServer.20 was originally written as part of the DSpace SWORDv2 implementation, but it was thought at the time it would be worth abstracting out the SWORD part from DSpace part, in the hope that this might be reusable. As far as I know, you're the first person to try using outside of DSpace (hence the lack of any other implementation or past experience). I could certainly be wrong but this SWORDv2 implementation also seems to be using the common SWORDv2 server library for Java: https://github.com/UNC-Libraries/Carolina-Digital-Repository/tree/master/sword-server/src/main/java/edu/unc/lib/dl/cdr/sword/server/managers So maybe I'm not alone. :) Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-20 15:59:12
|
Hi Richard! I just sent you separate pull request for the same issue: generalise getName/getFilename into getContentDispositionValue by pdurbin · Pull Request #3 · swordapp/JavaServer2.0 - https://github.com/swordapp/JavaServer2.0/pull/3 Here's the text I put in the pull request: --- This pull request aims to resolve "this is the same code as above, but with a different token; generalise" FIXME as well as the "first character truncated from filename in Content-Disposition" issue reported in http://www.mail-archive.com/swo...@li.../msg00337.html It uses http://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/ParameterParser.html Please note that a semicolon is used as a separator but http://svn.apache.org/viewvc/commons/proper/fileupload/trunk/src/test/java/org/apache/commons/fileupload/ParameterParserTest.java?r1=1455469&r2=1455470&pathrev=1515823& suggests that a comma might be a legitimate separator per https://issues.apache.org/jira/browse/FILEUPLOAD-139 --- Thanks, Phil On Thu, Aug 1, 2013 at 12:51 PM, Richard Jones <ri...@co...> wrote: > Hi Phil, > > On 28 June 2013 22:05, Philip Durbin <phi...@ha...> wrote: >> On Thu, Jun 27, 2013 at 10:53 AM, Philip Durbin >> <phi...@ha...> wrote: >>> Before the jar gets published, I believe I may have found a bug here: >>> >>> https://github.com/swordapp/JavaServer2.0/blob/31e625ac97be13f794e1b98512f8d1c48d751652/src/main/java/org/swordapp/server/SwordAPIEndpoint.java#L435 >>> >>> When I use the following command... >>> >>> curl --insecure -s -H "X-On-Behalf-Of: obo" --http1.0 --data-binary >>> "@example.zip" -H "Content-Disposition: filename=example.zip" -H >>> "Content-Type: application/zip" >>> https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 >>> >>> ... my "example.zip" file gets uploaded as "xample.zip" (i.e. the >>> first character is removed). I can easily reproduce this in Vagrant >>> with https://github.com/dvn/swordpoc if you'd like more detail. >> >> FYI, this bug has been confirmed by Bill McKinney who has submitted a >> pull request called "enhanced content disposition handling": >> https://github.com/swordapp/JavaServer2.0/pull/2 > > Great, thanks; I've not incorporated it yet purely due to lack of time > to look at sword for a month or so. I did bump into Bill at OR13 in > Canada, which was great, and he did prod me about it on your behalf :) > I will get this onto my todo list, though, thanks for submitting it! > > Cheers, > > Richard > >> For my part, I'm moved my proof of concept code into an experimental >> branch in our app itself: https://github.com/IQSS/dvn/commit/4838565 >> >> I can keep https://github.com/dvn/swordpoc around for anyone who wants >> to see the library exercised a bit with some working code. >> >> Phil >> >> -- >> Philip Durbin >> Software Developer for http://thedata.org >> http://www.iq.harvard.edu/people/philip-durbin > > > > -- > > Richard Jones, > > Founder, Cottage Labs > t: @richard_d_jones, @cottagelabs > w: http://cottagelabs.com -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-16 19:15:06
|
I recently added swordCollection.setCollectionPolicy(dataverse.getDepositTermsOfUse()) to getServiceDocument in ServiceDocumentManagerImpl and it works great, showing something like this in the Service Document: <collectionPolicy xmlns="http://purl.org/net/sword/terms/">Must pledge to do no evil.</collectionPolicy> Great. But now what? How would a SWORDv2 client accept this policy? >From http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html is seems like having a SWORDv2 client indicate acceptance of a collection policy is out of scope since it's getting into "policy management": "The scope of SWORD v2 will be limited to the deposit process between any two scholarly systems or between a user facing system and a service provider. This deposit process is only a portion of the full content lifecycle and does not attempt to provide support for collaborative or distributed authoring environments or policy management; it is focused entirely on the process of moving content from one location to another." Is this right? Out of scope? Do SWORDv2 server implementers have users accept the collection policy via some other mechanism such as a separate web interface or some other protocol? Thanks, Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-08 02:22:58
|
On Thu, Aug 1, 2013 at 12:51 PM, Richard Jones <ri...@co...> wrote: >>> ... my "example.zip" file gets uploaded as "xample.zip" (i.e. the >>> first character is removed). I can easily reproduce this in Vagrant >>> with https://github.com/dvn/swordpoc if you'd like more detail. >> >> FYI, this bug has been confirmed by Bill McKinney who has submitted a >> pull request called "enhanced content disposition handling": >> https://github.com/swordapp/JavaServer2.0/pull/2 > > Great, thanks; I've not incorporated it yet purely due to lack of time > to look at sword for a month or so. I did bump into Bill at OR13 in > Canada, which was great, and he did prod me about it on your behalf :) > I will get this onto my todo list, though, thanks for submitting it! Yes, Bill mentioned. Thanks so much for looking into this bug. Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-03 14:21:28
|
Hi SWORD folks, I tried a couple times to post the message below to http://support.rstudio.org/help/discussions/suggestions in late June but it never seemed to appear. Rather than losing the time I spent typing it up, I thought I'd just post it here instead so I can link to it. I hope you find it interesting. Phil p.s. The message I was trying to post: Is there any interest in having RStudio support SWORD (Simple Web-service Offering Repository Deposit)? (You can read more about SWORD at http://swordapp.org and http://en.wikipedia.org/wiki/SWORD_%28protocol%29 ) Christopher Gandrud and I have been discussing this on Twitter, but I thought I'd try to summarize. RStudio already has integrated support for Git, which is fantastic and has been highlighted in Christopher's recent paper: GitHub: A Tool for Social Data Set Development and Verification in the Cloud: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2199367 But what about permament archival of the data you're working with in RStudio? This line from http://blog.ouseful.info/2012/07/15/an-r-chitecture-for-reproducible-researchreportingdata-journalism/ struck me: > A recent addition, RStudio now integrates with RPubs.com, which means 1-click publishing of RMarkdown/knitr'd HTML to a hosted website is possible. Presumably, it wouldn't be too hard to extend RStudio so that publication to other online environments could be supported. (Hmm, thinks... could RStudio support publication using Github pages maybe, or something more general, such as SWORD/Atom Publishing?!) Other publication routes have also been demonstrated – for example, here's a recipe for publishing to WordPress from R). What if publishing to an archive was only a few clicks away? I'll just include the last few tweets which end with this one https://twitter.com/ChrisGandrud/status/347133428208132096 - @philipdurbin Sword is nice. Probably big startup costs for many researchers. Git daily version control + sword deposit? - @ChrisGandrud yeah, I too was thinking combining both. What start up costs with SWORD though? Is it hard? What if Rstudio supported it? - @philipdurbin If @rstudioapp supported it (e.g. you could send an RStudio Project to a @swordapp repo) the startup costs would be low. Full disclosure: I'm working on implementing SWORD v2 on the server side for an open source application (Dataverse Network) in https://redmine.hmdc.harvard.edu/issues/3108 so I'm interested in a growing list of client side implementations: http://swordapp.org/sword-v2/sword-v2-implementations/ -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: LEWIS S. <Stu...@ed...> - 2013-08-02 08:37:28
|
Hi all, >>>>>p.s. I'm growing concerned that this mailing list is so quiet (and >>>>>only admins can see the number of people subscribed). Have people >>>>>moved on from SWORD to some other standard? If so, which one? >>>> >>>>I just checked - there are 175 subscribers to this list. >> >>Thanks, Stuart. I'm glad to hear this number is as large as it is. >>Many mailing lists allow this number to be discoverable by the >>subscribers and if it's easy to do so, I would encourage making this >>change so subscribers don't have to ask. > >I don't think that I have admin access on this list, so may have to >wait for Stuart. > >Stuart - can I have (or do I already have) admin access to the list? >If not, could you elevate my account? We use the standard SourceForge mailman interface: - https://lists.sourceforge.net/lists/listinfo/sword-app-tech I'm not sure if it is possible to surface this information, but if anyone knows how - let me know! Thanks, Stuart Lewis Head of Research and Learning Services Deputy Director Library & University Collections, Information Services University of Edinburgh Stu...@ed... On 01/08/2013 19:47, "Richard Jones" <ri...@co...> wrote: Hi Phil, Having one of my period re-sync's with the mailing list :) >>>> p.s. I'm growing concerned that this mailing list is so quiet (and >>>> only admins can see the number of people subscribed). Have people >>>> moved on from SWORD to some other standard? If so, which one? >>> >>> I just checked - there are 175 subscribers to this list. > > Thanks, Stuart. I'm glad to hear this number is as large as it is. > Many mailing lists allow this number to be discoverable by the > subscribers and if it's easy to do so, I would encourage making this > change so subscribers don't have to ask. I don't think that I have admin access on this list, so may have to wait for Stuart. Stuart - can I have (or do I already have) admin access to the list? If not, could you elevate my account? >>> As far as I know, SWORD is the main contender in town when it comes to >>>a standardized deposit interface to this type of repository. I've also >>>wondered about the quietness of this list. I think there may be a few >>>reasons: one, is that a lot of repository users are still grappling >>>with their repositories, without yet getting as far as accepting remote >>>deposits. Second, SWORD doesn't yet really have an active community >>>sharing deposit tools. Partly this is because many uses of SWORD will >>>be very specific point-to-point integrations, which might not be of >>>interest to too many others. >>> >>> It would be good to hear a wider discussion about this, and how we >>>share more about our individual uses of SWORD. >> >> I think one of the main issues is exactly where to ask about what. >> Because SWORD is a standard, but the technical questions are really >> about implementations, where is the best place to post about problems? >> For example, if the problems are specifically with the DSpace >> implementation of SWORD, it is /probably/ better to ask on dspace-dev. > > I'm sure as implementers begin their work many will have questions > about the SWORD specification itself (I know I do), so I'm absolutely > supportive of and thankful for this mailing list. Glad that it is useful! I'm hoping as well that we'll be able to do another iteration on the spec to iron out issues that people are having with things like multipart, and I hope that this list can be active in the discussions around that iteration. >> Also, because we're in the early stages of community development with >> SWORD, Stuart and I are a bit of a bottleneck on this list - usually >> one of us is required to respond, and if we're unavailable for any >> length of time (e.g. I've been travelling for nearly 2 weeks now, and >> am emailing from the fourth row of a session at OAI8 right now :) ), >> then the list looks dead. >> >> We are hoping to have some discussions around sword sustainability >> with Jisc quite soon which community development and support like this >> is going to be a key part of. >> >> Very interested in people's thoughts as to how to make things better. > > I've mentioned this in passing but I'll repeat my offer to log an IRC > channel on Freenode to discuss SWORD. I'm discussing implementation > details that would not be of general interest at > http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel > dedicated to the SWORD spec itself would be fantastic. I find chat to > be a great way to get a quick pulse on an issue. I fear the walls of > text I've been sending to this mailing list are simply too much at > once. :) I'd be very glad to include an IRC channel that people can use on the website, or in the READMEs, but I have to confess to not being very good at using chat channels, so I'd most likely not hang around there all the time! > Anyway, I've very interested in community development in general and > happy to help in any way I can. (I don't mean to beat a dead horse > about IRC.) I'm learning a lot about SWORD and AtomPub and working on > our implementation is actually a lot of fun. :) Ok, well I'll definitely keep the list up-to-date with developments at this end. > p.s. In other news, we (royal we, Peter Bull, actually) are starting > to use Richard's https://github.com/swordapp/python-client-sword2 to > cook up a test client > https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very > thankful for all the libraries that have been published! :D Cheers, Richard > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com> The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: LEWIS S. <Stu...@ed...> - 2013-08-02 08:28:31
|
Hi Nick, EasyDeposit was designed for one-off deposits rather than any on-going asset management, so was never upgraded to SWORDv2. It can be configured to use the SWORDv2 libraries, but only deposits using the same methods. If your colleague prefers PHP, point him towards https://github.com/swordapp/swordappv2-php-library/ Thanks, Stuart Lewis Head of Research and Learning Services Deputy Director Library & University Collections, Information Services University of Edinburgh Stu...@ed...<mailto:Stu...@ed...> From: <Sheppard>, "n.e...@le...<mailto:n.e...@le...>" <n.e...@le...<mailto:n.e...@le...>> Date: Friday, 2 August 2013 08:36 To: "swo...@li...<mailto:swo...@li...>" <swo...@li...<mailto:swo...@li...>> Subject: Re: [sword-app-tech] SWORD community development and support (Kathi Fletcher) I nearly posted yesterday...but then didn't and went to EPrints-tech instead. We are in the early stages of scoping EPrints (3.3.11)…tried to hook up EasyDeposit, got a bit confused along the way but did finally get it working. Sort of. I've a colleague in our school of art and architecture who has developed a system to manage non-textual files (video/images mainly) and we discussed pushing them to a repository via sword. As a non-programmer I find sword implementation tricky without a lot of technical support and/or easy to use tools...my colleague is a php programmer but doesn't live in repository land so at this stage I just wanted to point him in the right direction. Having used EasyDeposit before and as it's written in php(?) I thought it a good place to start...though it's just v1.3 I think and probably makes sense for us to look at v2 for full CRUD and as we're using EPrints 3.3.11. What clients/tools/implementation guides are available - had a look at Richard Jones' Java client on github but beyond my technical ken I think! Any advice much appreciated both pitched at my limited technical nouse and how best to get my colleague started (I'll point him at the list of course!) Thanks Nick Message: 1 Date: Thu, 1 Aug 2013 10:36:27 -0500 From: Kathi Fletcher <Kat...@gm...<mailto:Kat...@gm...>> Subject: Re: [sword-app-tech] SWORD community development and support To: Philip Durbin <phi...@ha...<mailto:phi...@ha...>> Cc: "swo...@li...<mailto:swo...@li...>" <swo...@li...<mailto:swo...@li...>>, LEWIS Stuart <Stu...@ed...<mailto:Stu...@ed...>> Message-ID: <CAFbMqTY1to5VteLEnCs2Y0bqak5pdaARbQVAj+JGGkPMm=Q+...@ma...<mailto:CAFbMqTY1to5VteLEnCs2Y0bqak5pdaARbQVAj+JGGkPMm=Q+...@ma...>> Content-Type: text/plain; charset="iso-8859-1" Just a quick update, since the last post was about the quietness of the list. We are still actively using SWORD to deposit content to Connexionx ( cnx.org). It just works, so we haven't had much traffic on the list! Kathi On Thu, Aug 1, 2013 at 8:02 AM, Philip Durbin <phi...@ha...<mailto:phi...@ha...>>wrote: > Thanks, Stuart and Richard. My reply is inline below. > > On Thu, Jun 20, 2013 at 4:41 AM, Richard Jones > <ri...@co...<mailto:ri...@co...>> > wrote: > >>> p.s. I'm growing concerned that this mailing list is so quiet (and > >>> only admins can see the number of people subscribed). Have people > >>> moved on from SWORD to some other standard? If so, which one? > >> > >> I just checked - there are 175 subscribers to this list. > > Thanks, Stuart. I'm glad to hear this number is as large as it is. > Many mailing lists allow this number to be discoverable by the > subscribers and if it's easy to do so, I would encourage making this > change so subscribers don't have to ask. > > >> As far as I know, SWORD is the main contender in town when it comes > >> to > a standardized deposit interface to this type of repository. I've > also wondered about the quietness of this list. I think there may be > a few > reasons: one, is that a lot of repository users are still grappling > with their repositories, without yet getting as far as accepting > remote deposits. Second, SWORD doesn't yet really have an active > community sharing deposit tools. Partly this is because many uses of > SWORD will be very specific point-to-point integrations, which might > not be of interest to too many others. > >> > >> It would be good to hear a wider discussion about this, and how we > share more about our individual uses of SWORD. > > > > I think one of the main issues is exactly where to ask about what. > > Because SWORD is a standard, but the technical questions are really > > about implementations, where is the best place to post about problems? > > For example, if the problems are specifically with the DSpace > > implementation of SWORD, it is /probably/ better to ask on dspace-dev. > > I'm sure as implementers begin their work many will have questions > about the SWORD specification itself (I know I do), so I'm absolutely > supportive of and thankful for this mailing list. > > > Also, because we're in the early stages of community development > > with SWORD, Stuart and I are a bit of a bottleneck on this list - > > usually one of us is required to respond, and if we're unavailable > > for any length of time (e.g. I've been travelling for nearly 2 weeks > > now, and am emailing from the fourth row of a session at OAI8 right > > now :) ), then the list looks dead. > > > > We are hoping to have some discussions around sword sustainability > > with Jisc quite soon which community development and support like > > this is going to be a key part of. > > > > Very interested in people's thoughts as to how to make things better. > > I've mentioned this in passing but I'll repeat my offer to log an IRC > channel on Freenode to discuss SWORD. I'm discussing implementation > details that would not be of general interest at > http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel > dedicated to the SWORD spec itself would be fantastic. I find chat to > be a great way to get a quick pulse on an issue. I fear the walls of > text I've been sending to this mailing list are simply too much at > once. :) > > Anyway, I've very interested in community development in general and > happy to help in any way I can. (I don't mean to beat a dead horse > about IRC.) I'm learning a lot about SWORD and AtomPub and working on > our implementation is actually a lot of fun. :) > > Phil > > p.s. In other news, we (royal we, Peter Bull, actually) are starting > to use Richard's https://github.com/swordapp/python-client-sword2 to > cook up a test client > https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very > thankful for all the libraries that have been published! > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > > ---------------------------------------------------------------------- > -------- Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.c > lktrk _______________________________________________ > sword-app-tech mailing list > swo...@li...<mailto:swo...@li...> > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > -- Katherine Fletcher, kat...@gm...<mailto:kat...@gm...> <kat...@gm...<mailto:kat...@gm...>> Twitter: kefletcher <http://www.twitter.com/kefletcher> Blog: kefletcher.blogspot.com <kat...@gm...<mailto:kat...@gm...>> -------------- next part -------------- An HTML attachment was scrubbed... ------------------------------ To view the terms under which this email is distributed, please go to http://disclaimer.leedsmet.ac.uk/email.htm -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
|
From: Sheppard, N. <N.E...@le...> - 2013-08-02 07:36:34
|
I nearly posted yesterday...but then didn't and went to EPrints-tech instead. We are in the early stages of scoping EPrints (3.3.11)...tried to hook up EasyDeposit, got a bit confused along the way but did finally get it working. Sort of.
I've a colleague in our school of art and architecture who has developed a system to manage non-textual files (video/images mainly) and we discussed pushing them to a repository via sword. As a non-programmer I find sword implementation tricky without a lot of technical support and/or easy to use tools...my colleague is a php programmer but doesn't live in repository land so at this stage I just wanted to point him in the right direction. Having used EasyDeposit before and as it's written in php(?) I thought it a good place to start...though it's just v1.3 I think and probably makes sense for us to look at v2 for full CRUD and as we're using EPrints 3.3.11. What clients/tools/implementation guides are available - had a look at Richard Jones' Java client on github but beyond my technical ken I think!
Any advice much appreciated both pitched at my limited technical nouse and how best to get my colleague started (I'll point him at the list of course!)
Thanks
Nick
Message: 1
Date: Thu, 1 Aug 2013 10:36:27 -0500
From: Kathi Fletcher <Kat...@gm...<mailto:Kat...@gm...>>
Subject: Re: [sword-app-tech] SWORD community development and support
To: Philip Durbin <phi...@ha...<mailto:phi...@ha...>>
Cc: "swo...@li...<mailto:swo...@li...>"
<swo...@li...<mailto:swo...@li...>>, LEWIS Stuart
<Stu...@ed...<mailto:Stu...@ed...>>
Message-ID:
<CAFbMqTY1to5VteLEnCs2Y0bqak5pdaARbQVAj+JGGkPMm=Q+...@ma...<mailto:CAFbMqTY1to5VteLEnCs2Y0bqak5pdaARbQVAj+JGGkPMm=Q+...@ma...>>
Content-Type: text/plain; charset="iso-8859-1"
Just a quick update, since the last post was about the quietness of the list. We are still actively using SWORD to deposit content to Connexionx ( cnx.org). It just works, so we haven't had much traffic on the list!
Kathi
On Thu, Aug 1, 2013 at 8:02 AM, Philip Durbin <phi...@ha...<mailto:phi...@ha...>>wrote:
> Thanks, Stuart and Richard. My reply is inline below.
>
> On Thu, Jun 20, 2013 at 4:41 AM, Richard Jones
> <ri...@co...<mailto:ri...@co...>>
> wrote:
> >>> p.s. I'm growing concerned that this mailing list is so quiet (and
> >>> only admins can see the number of people subscribed). Have people
> >>> moved on from SWORD to some other standard? If so, which one?
> >>
> >> I just checked - there are 175 subscribers to this list.
>
> Thanks, Stuart. I'm glad to hear this number is as large as it is.
> Many mailing lists allow this number to be discoverable by the
> subscribers and if it's easy to do so, I would encourage making this
> change so subscribers don't have to ask.
>
> >> As far as I know, SWORD is the main contender in town when it comes
> >> to
> a standardized deposit interface to this type of repository. I've
> also wondered about the quietness of this list. I think there may be
> a few
> reasons: one, is that a lot of repository users are still grappling
> with their repositories, without yet getting as far as accepting
> remote deposits. Second, SWORD doesn't yet really have an active
> community sharing deposit tools. Partly this is because many uses of
> SWORD will be very specific point-to-point integrations, which might
> not be of interest to too many others.
> >>
> >> It would be good to hear a wider discussion about this, and how we
> share more about our individual uses of SWORD.
> >
> > I think one of the main issues is exactly where to ask about what.
> > Because SWORD is a standard, but the technical questions are really
> > about implementations, where is the best place to post about problems?
> > For example, if the problems are specifically with the DSpace
> > implementation of SWORD, it is /probably/ better to ask on dspace-dev.
>
> I'm sure as implementers begin their work many will have questions
> about the SWORD specification itself (I know I do), so I'm absolutely
> supportive of and thankful for this mailing list.
>
> > Also, because we're in the early stages of community development
> > with SWORD, Stuart and I are a bit of a bottleneck on this list -
> > usually one of us is required to respond, and if we're unavailable
> > for any length of time (e.g. I've been travelling for nearly 2 weeks
> > now, and am emailing from the fourth row of a session at OAI8 right
> > now :) ), then the list looks dead.
> >
> > We are hoping to have some discussions around sword sustainability
> > with Jisc quite soon which community development and support like
> > this is going to be a key part of.
> >
> > Very interested in people's thoughts as to how to make things better.
>
> I've mentioned this in passing but I'll repeat my offer to log an IRC
> channel on Freenode to discuss SWORD. I'm discussing implementation
> details that would not be of general interest at
> http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel
> dedicated to the SWORD spec itself would be fantastic. I find chat to
> be a great way to get a quick pulse on an issue. I fear the walls of
> text I've been sending to this mailing list are simply too much at
> once. :)
>
> Anyway, I've very interested in community development in general and
> happy to help in any way I can. (I don't mean to beat a dead horse
> about IRC.) I'm learning a lot about SWORD and AtomPub and working on
> our implementation is actually a lot of fun. :)
>
> Phil
>
> p.s. In other news, we (royal we, Peter Bull, actually) are starting
> to use Richard's https://github.com/swordapp/python-client-sword2 to
> cook up a test client
> https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very
> thankful for all the libraries that have been published!
>
> --
> Philip Durbin
> Software Developer for http://thedata.org
> http://www.iq.harvard.edu/people/philip-durbin
>
>
> ----------------------------------------------------------------------
> -------- Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version control? Why should you start doing it? Read more to find out.
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.c
> lktrk _______________________________________________
> sword-app-tech mailing list
> swo...@li...<mailto:swo...@li...>
> https://lists.sourceforge.net/lists/listinfo/sword-app-tech
>
--
Katherine Fletcher, kat...@gm...<mailto:kat...@gm...> <kat...@gm...<mailto:kat...@gm...>>
Twitter: kefletcher <http://www.twitter.com/kefletcher> Blog:
kefletcher.blogspot.com
<kat...@gm...<mailto:kat...@gm...>>
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
To view the terms under which this email is distributed, please go to http://disclaimer.leedsmet.ac.uk/email.htm
|
|
From: Richard J. <ri...@co...> - 2013-08-01 18:50:07
|
Nope, no attempt to use OAuth with SWORD that I'm aware of. We toyed with trying to do this as part of the protocol, and then decided that it was Too Hard, might put people off implementing, and also ought to be orthogonal to the task that sword is trying to carry out, so we decided to leave it up to implementers to decide. Do you think that any modifications to sword are required in order to permit OAuth? Our principal during development was to make sure we didn't do anything which prevented such things, but I would like some confirmation that we succeeded! Cheers, Richard On 1 August 2013 17:08, Philip Durbin <phi...@ha...> wrote: > On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin > <phi...@ha...> wrote: >> Does anyone's SWORDv2 implementation use API keys (negotiated via >> OAuth, maybe?) rather than HTTP Basic Authentication >> (username/password) in conjunction with a TLS connection? > > It looks like Richard asked a similar question here: > > Re: [Sword-TAP] on-behalf-of vs. OAuth - > http://www.mail-archive.com/swo...@li.../msg00141.html > > >From what I can tell, no one has implemented OAuth or similar with SWORD yet. > > Sorry for not noticing this post earlier. > > Phil > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2013-08-01 18:47:47
|
Hi Phil, Having one of my period re-sync's with the mailing list :) >>>> p.s. I'm growing concerned that this mailing list is so quiet (and >>>> only admins can see the number of people subscribed). Have people >>>> moved on from SWORD to some other standard? If so, which one? >>> >>> I just checked - there are 175 subscribers to this list. > > Thanks, Stuart. I'm glad to hear this number is as large as it is. > Many mailing lists allow this number to be discoverable by the > subscribers and if it's easy to do so, I would encourage making this > change so subscribers don't have to ask. I don't think that I have admin access on this list, so may have to wait for Stuart. Stuart - can I have (or do I already have) admin access to the list? If not, could you elevate my account? >>> As far as I know, SWORD is the main contender in town when it comes to a standardized deposit interface to this type of repository. I've also wondered about the quietness of this list. I think there may be a few reasons: one, is that a lot of repository users are still grappling with their repositories, without yet getting as far as accepting remote deposits. Second, SWORD doesn't yet really have an active community sharing deposit tools. Partly this is because many uses of SWORD will be very specific point-to-point integrations, which might not be of interest to too many others. >>> >>> It would be good to hear a wider discussion about this, and how we share more about our individual uses of SWORD. >> >> I think one of the main issues is exactly where to ask about what. >> Because SWORD is a standard, but the technical questions are really >> about implementations, where is the best place to post about problems? >> For example, if the problems are specifically with the DSpace >> implementation of SWORD, it is /probably/ better to ask on dspace-dev. > > I'm sure as implementers begin their work many will have questions > about the SWORD specification itself (I know I do), so I'm absolutely > supportive of and thankful for this mailing list. Glad that it is useful! I'm hoping as well that we'll be able to do another iteration on the spec to iron out issues that people are having with things like multipart, and I hope that this list can be active in the discussions around that iteration. >> Also, because we're in the early stages of community development with >> SWORD, Stuart and I are a bit of a bottleneck on this list - usually >> one of us is required to respond, and if we're unavailable for any >> length of time (e.g. I've been travelling for nearly 2 weeks now, and >> am emailing from the fourth row of a session at OAI8 right now :) ), >> then the list looks dead. >> >> We are hoping to have some discussions around sword sustainability >> with Jisc quite soon which community development and support like this >> is going to be a key part of. >> >> Very interested in people's thoughts as to how to make things better. > > I've mentioned this in passing but I'll repeat my offer to log an IRC > channel on Freenode to discuss SWORD. I'm discussing implementation > details that would not be of general interest at > http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel > dedicated to the SWORD spec itself would be fantastic. I find chat to > be a great way to get a quick pulse on an issue. I fear the walls of > text I've been sending to this mailing list are simply too much at > once. :) I'd be very glad to include an IRC channel that people can use on the website, or in the READMEs, but I have to confess to not being very good at using chat channels, so I'd most likely not hang around there all the time! > Anyway, I've very interested in community development in general and > happy to help in any way I can. (I don't mean to beat a dead horse > about IRC.) I'm learning a lot about SWORD and AtomPub and working on > our implementation is actually a lot of fun. :) Ok, well I'll definitely keep the list up-to-date with developments at this end. > p.s. In other news, we (royal we, Peter Bull, actually) are starting > to use Richard's https://github.com/swordapp/python-client-sword2 to > cook up a test client > https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very > thankful for all the libraries that have been published! :D Cheers, Richard > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2013-08-01 17:43:51
|
Hi Phil, On 23 July 2013 18:32, Philip Durbin <phi...@ha...> wrote: > On Wed, Jul 17, 2013 at 11:14 AM, Philip Durbin > <phi...@ha...> wrote: >> 1. Am I doing it wrong by using the col-uri to both create the study >> and upload example.zip? > > After discussing this at http://irclog.iq.harvard.edu/dvn/2013-07-22 > I'm pretty sure I was indeed doing this wrong. > > I'm still doing a POST of an Atom entry to a col-uri to create a study. > > But now*, to upload example.zip, I'm doing a PUT to the edit-media-uri. Yup, that's definitely the right way round. The POST gives you an object that exists on the server, and then the PUT allows you to modify the content of that object. In your previous set-up you were probably overwriting the original item with the new item rather than adding to it. The Simple Sword Server is - as the name suggests - pretty Simple, and it might not have made that obvious :) > In the discussion above, the video at > http://cottagelabs.com/news/intro-to-sword-2 was pointed out to me and > I've found it very helpful. In fact, I started a public Google Doc > about it with screenshots, notes, and a table that shows the relevant > HTTP verbs (GET, POST, PUT, DELETE) mapped to the five URIs that are > in play (service-document-uri, collection-uri, edit-uri, > edit-media-uri, statement-uri): > > https://docs.google.com/document/d/1oyz3ZTfZA_7FFNpZNaxR6cDJzLXgm3X1ybwUkMQwL1Q/edit?usp=sharing > > Comments are certainly welcome. I tried to keep the document generic to SWORD. That's really great, I'm glad you found it useful. We had quite a lot of fun making the video too! Cheers, Richard > > I still have many questions about SWORD, but I'll keep wrestling. > Thanks for reading. > > Phil > > * As of https://github.com/IQSS/dvn/commit/1ba5287 > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Richard J. <ri...@co...> - 2013-08-01 17:16:57
|
Hi Phil, On 28 June 2013 22:05, Philip Durbin <phi...@ha...> wrote: > On Thu, Jun 27, 2013 at 10:53 AM, Philip Durbin > <phi...@ha...> wrote: >> Before the jar gets published, I believe I may have found a bug here: >> >> https://github.com/swordapp/JavaServer2.0/blob/31e625ac97be13f794e1b98512f8d1c48d751652/src/main/java/org/swordapp/server/SwordAPIEndpoint.java#L435 >> >> When I use the following command... >> >> curl --insecure -s -H "X-On-Behalf-Of: obo" --http1.0 --data-binary >> "@example.zip" -H "Content-Disposition: filename=example.zip" -H >> "Content-Type: application/zip" >> https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 >> >> ... my "example.zip" file gets uploaded as "xample.zip" (i.e. the >> first character is removed). I can easily reproduce this in Vagrant >> with https://github.com/dvn/swordpoc if you'd like more detail. > > FYI, this bug has been confirmed by Bill McKinney who has submitted a > pull request called "enhanced content disposition handling": > https://github.com/swordapp/JavaServer2.0/pull/2 Great, thanks; I've not incorporated it yet purely due to lack of time to look at sword for a month or so. I did bump into Bill at OR13 in Canada, which was great, and he did prod me about it on your behalf :) I will get this onto my todo list, though, thanks for submitting it! Cheers, Richard > For my part, I'm moved my proof of concept code into an experimental > branch in our app itself: https://github.com/IQSS/dvn/commit/4838565 > > I can keep https://github.com/dvn/swordpoc around for anyone who wants > to see the library exercised a bit with some working code. > > Phil > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Philip D. <phi...@ha...> - 2013-08-01 16:08:33
|
On Thu, Aug 1, 2013 at 10:07 AM, Philip Durbin <phi...@ha...> wrote: > Does anyone's SWORDv2 implementation use API keys (negotiated via > OAuth, maybe?) rather than HTTP Basic Authentication > (username/password) in conjunction with a TLS connection? It looks like Richard asked a similar question here: Re: [Sword-TAP] on-behalf-of vs. OAuth - http://www.mail-archive.com/swo...@li.../msg00141.html >From what I can tell, no one has implemented OAuth or similar with SWORD yet. Sorry for not noticing this post earlier. Phil -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Kathi F. <Kat...@gm...> - 2013-08-01 15:36:35
|
Just a quick update, since the last post was about the quietness of the list. We are still actively using SWORD to deposit content to Connexionx ( cnx.org). It just works, so we haven't had much traffic on the list! Kathi On Thu, Aug 1, 2013 at 8:02 AM, Philip Durbin <phi...@ha...>wrote: > Thanks, Stuart and Richard. My reply is inline below. > > On Thu, Jun 20, 2013 at 4:41 AM, Richard Jones <ri...@co...> > wrote: > >>> p.s. I'm growing concerned that this mailing list is so quiet (and > >>> only admins can see the number of people subscribed). Have people > >>> moved on from SWORD to some other standard? If so, which one? > >> > >> I just checked - there are 175 subscribers to this list. > > Thanks, Stuart. I'm glad to hear this number is as large as it is. > Many mailing lists allow this number to be discoverable by the > subscribers and if it's easy to do so, I would encourage making this > change so subscribers don't have to ask. > > >> As far as I know, SWORD is the main contender in town when it comes to > a standardized deposit interface to this type of repository. I've also > wondered about the quietness of this list. I think there may be a few > reasons: one, is that a lot of repository users are still grappling with > their repositories, without yet getting as far as accepting remote > deposits. Second, SWORD doesn't yet really have an active community > sharing deposit tools. Partly this is because many uses of SWORD will be > very specific point-to-point integrations, which might not be of interest > to too many others. > >> > >> It would be good to hear a wider discussion about this, and how we > share more about our individual uses of SWORD. > > > > I think one of the main issues is exactly where to ask about what. > > Because SWORD is a standard, but the technical questions are really > > about implementations, where is the best place to post about problems? > > For example, if the problems are specifically with the DSpace > > implementation of SWORD, it is /probably/ better to ask on dspace-dev. > > I'm sure as implementers begin their work many will have questions > about the SWORD specification itself (I know I do), so I'm absolutely > supportive of and thankful for this mailing list. > > > Also, because we're in the early stages of community development with > > SWORD, Stuart and I are a bit of a bottleneck on this list - usually > > one of us is required to respond, and if we're unavailable for any > > length of time (e.g. I've been travelling for nearly 2 weeks now, and > > am emailing from the fourth row of a session at OAI8 right now :) ), > > then the list looks dead. > > > > We are hoping to have some discussions around sword sustainability > > with Jisc quite soon which community development and support like this > > is going to be a key part of. > > > > Very interested in people's thoughts as to how to make things better. > > I've mentioned this in passing but I'll repeat my offer to log an IRC > channel on Freenode to discuss SWORD. I'm discussing implementation > details that would not be of general interest at > http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel > dedicated to the SWORD spec itself would be fantastic. I find chat to > be a great way to get a quick pulse on an issue. I fear the walls of > text I've been sending to this mailing list are simply too much at > once. :) > > Anyway, I've very interested in community development in general and > happy to help in any way I can. (I don't mean to beat a dead horse > about IRC.) I'm learning a lot about SWORD and AtomPub and working on > our implementation is actually a lot of fun. :) > > Phil > > p.s. In other news, we (royal we, Peter Bull, actually) are starting > to use Richard's https://github.com/swordapp/python-client-sword2 to > cook up a test client > https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very > thankful for all the libraries that have been published! > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech > -- Katherine Fletcher, kat...@gm... <kat...@gm...> Twitter: kefletcher <http://www.twitter.com/kefletcher> Blog: kefletcher.blogspot.com <kat...@gm...> |
|
From: Philip D. <phi...@ha...> - 2013-08-01 14:08:12
|
Does anyone's SWORDv2 implementation use API keys (negotiated via OAuth, maybe?) rather than HTTP Basic Authentication (username/password) in conjunction with a TLS connection? In all of my testing I'm using HTTPS to send passwords securely, but supporting API keys would be nice. Thanks, Phil p.s. This came up in a meeting yesterday and I just entered the suggestion at https://redmine.hmdc.harvard.edu/issues/3208 after some discussion at http://irclog.iq.harvard.edu/dvn/2013-08-01 -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-08-01 13:02:10
|
Thanks, Stuart and Richard. My reply is inline below. On Thu, Jun 20, 2013 at 4:41 AM, Richard Jones <ri...@co...> wrote: >>> p.s. I'm growing concerned that this mailing list is so quiet (and >>> only admins can see the number of people subscribed). Have people >>> moved on from SWORD to some other standard? If so, which one? >> >> I just checked - there are 175 subscribers to this list. Thanks, Stuart. I'm glad to hear this number is as large as it is. Many mailing lists allow this number to be discoverable by the subscribers and if it's easy to do so, I would encourage making this change so subscribers don't have to ask. >> As far as I know, SWORD is the main contender in town when it comes to a standardized deposit interface to this type of repository. I've also wondered about the quietness of this list. I think there may be a few reasons: one, is that a lot of repository users are still grappling with their repositories, without yet getting as far as accepting remote deposits. Second, SWORD doesn't yet really have an active community sharing deposit tools. Partly this is because many uses of SWORD will be very specific point-to-point integrations, which might not be of interest to too many others. >> >> It would be good to hear a wider discussion about this, and how we share more about our individual uses of SWORD. > > I think one of the main issues is exactly where to ask about what. > Because SWORD is a standard, but the technical questions are really > about implementations, where is the best place to post about problems? > For example, if the problems are specifically with the DSpace > implementation of SWORD, it is /probably/ better to ask on dspace-dev. I'm sure as implementers begin their work many will have questions about the SWORD specification itself (I know I do), so I'm absolutely supportive of and thankful for this mailing list. > Also, because we're in the early stages of community development with > SWORD, Stuart and I are a bit of a bottleneck on this list - usually > one of us is required to respond, and if we're unavailable for any > length of time (e.g. I've been travelling for nearly 2 weeks now, and > am emailing from the fourth row of a session at OAI8 right now :) ), > then the list looks dead. > > We are hoping to have some discussions around sword sustainability > with Jisc quite soon which community development and support like this > is going to be a key part of. > > Very interested in people's thoughts as to how to make things better. I've mentioned this in passing but I'll repeat my offer to log an IRC channel on Freenode to discuss SWORD. I'm discussing implementation details that would not be of general interest at http://irclog.iq.harvard.edu/dvn/2013-07-30 for example, but a channel dedicated to the SWORD spec itself would be fantastic. I find chat to be a great way to get a quick pulse on an issue. I fear the walls of text I've been sending to this mailing list are simply too much at once. :) Anyway, I've very interested in community development in general and happy to help in any way I can. (I don't mean to beat a dead horse about IRC.) I'm learning a lot about SWORD and AtomPub and working on our implementation is actually a lot of fun. :) Phil p.s. In other news, we (royal we, Peter Bull, actually) are starting to use Richard's https://github.com/swordapp/python-client-sword2 to cook up a test client https://github.com/dvn/swordpoc/tree/master/dvn_client . We're very thankful for all the libraries that have been published! -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-07-23 17:32:32
|
On Wed, Jul 17, 2013 at 11:14 AM, Philip Durbin <phi...@ha...> wrote: > 1. Am I doing it wrong by using the col-uri to both create the study > and upload example.zip? After discussing this at http://irclog.iq.harvard.edu/dvn/2013-07-22 I'm pretty sure I was indeed doing this wrong. I'm still doing a POST of an Atom entry to a col-uri to create a study. But now*, to upload example.zip, I'm doing a PUT to the edit-media-uri. In the discussion above, the video at http://cottagelabs.com/news/intro-to-sword-2 was pointed out to me and I've found it very helpful. In fact, I started a public Google Doc about it with screenshots, notes, and a table that shows the relevant HTTP verbs (GET, POST, PUT, DELETE) mapped to the five URIs that are in play (service-document-uri, collection-uri, edit-uri, edit-media-uri, statement-uri): https://docs.google.com/document/d/1oyz3ZTfZA_7FFNpZNaxR6cDJzLXgm3X1ybwUkMQwL1Q/edit?usp=sharing Comments are certainly welcome. I tried to keep the document generic to SWORD. I still have many questions about SWORD, but I'll keep wrestling. Thanks for reading. Phil * As of https://github.com/IQSS/dvn/commit/1ba5287 -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Philip D. <phi...@ha...> - 2013-07-17 15:15:00
|
On Thu, Jul 11, 2013 at 11:09 AM, Richard Jones <ri...@co...> wrote: > It is my plan and hope to back all of the multipart OUT of the sword > spec for a future version (like a 2.1), so I strongly recommend not > using multipart deposit. Instead do a POST of an Atom Entry and a PUT > of the Media Resource in two distinct HTTP requests. Thank you very much for this explanation, Richard. I have many follow up questions for you or anyone on the list. However, before I dive into POST vs. PUT I'd like to back up and explain a little more at a high level what I'm trying to achieve. Our first use case for SWORDv2 is to allow Open Journal Systems (OJS) to deposit data into Dataverse Network (DVN). http://projects.iq.harvard.edu/ojs-dvn has the details but just imagine that an academic journal called the New England Journal of Coffee is hosted in an installation of OJS. An author publishes paper called "Roasting at Home" and has some data in a file called example.zip. On the DVN side, we have already created a dataverse called "NEJC Dataverse" dedicated to that journal. It's empty at first, but the academic paper will eventually have a study within that dataverse. That study is a container (with metadata such as author and publication data) into which files will be deposited. Continuing the example, the OJS plugin would use SWORDv2 to do the following: 1. Retrieve a service document that lists SWORDv2 collections for the NEJC Dataverse. 2. Create a study within the NEJC Dataverse called "Roasting at Home" (to match the academic paper published in OJS). 3. Upload example.zip to the "Roasting at Home" study in the NEJC Dataverse. (I hope this makes sense. If you look at the "Textbooks and Test Scores" study at http://hdl.handle.net/1902.1/11252 and click "Data & Analysis" you can see a number of .zip files that have been uploaded to the study.) Now, back to SWORDv2... I want to make sure I'm understanding correctly how to achieve the three steps above. As I understand it, each step could be expressed as a curl command: 1. Retrieve a service document (GET): curl -s http://sword:sword@localhost:8080/sd-uri 2. Create a resource from an atom entry (POST) using the "collection" IRI: curl --http1.0 --data-binary "@atom-entry-study.xml" -H "Content-Type: application/atom+xml" http://sword:sword@localhost:8080/col-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9 3. Upload example.zip (PUT) using the "edit media" IRI: curl --upload-file example.zip -H "Content-Disposition: filename=example.zip" -H "Content-Type: application/zip" http://sword:sword@localhost:8080/em-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2 With those curl commands I've switched to the URLs (sd-uri, col-uri, em-uri) supported by the SWORDv2 reference implementation: https://github.com/swordapp/Simple-Sword-Server The reason I'm asking about this is that in the SWORDv2 implementation I'm working on for DVN, I'm using col-uri for both the creation of the study (creating a resource with an atom entry) and the upload of example.zip. The latest code is here: https://github.com/IQSS/dvn/commit/1b7e5e0 It works, and I'm pleased but... after your comment about POST vs. PUT I wonder if I'm doing it wrong since the CollectionServletDefault (used for the equivalent of col-uri) at https://github.com/swordapp/JavaServer2.0 doesn't support PUT. MediaResourceServletDefault on the other hand (the equivalent of em-iri) does support both POST and PUT. This morning I experimented with the curl commands above against the reference implementation of SWORDv2 and captured the output in the readme at https://github.com/dvn/swordpoc (I'll include the relevant portion below.) So, to sum up my questions... 1. Am I doing it wrong by using the col-uri to both create the study and upload example.zip? 2. Should I be using col-ui to create the study and em-iri to upload example.zip? Thanks, Phil p.s. From the https://github.com/dvn/swordpoc readme: ### A more complicated SSS example: two step deposit In the example above, "example.zip" is deposited into a SWORD collection, but let's imagine a more complicated example. Let's say we want to perform the following steps: 1. Create a resource with an atom entry (i.e. an XML file): http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_creatingresource_entry 2. Upload example.zip to the newly created resource: http://swordapp.github.io/SWORDv2-Profile/SWORDProfile.html#protocoloperations_editingcontent_binary These steps translate into two distinct HTTP request, as explained by the SWORDv2 spec lead, Richard Jones... > It is my plan and hope to back all of the multipart OUT of the sword > spec for a future version (like a 2.1), so I strongly recommend not > using multipart deposit. Instead do a POST of an Atom Entry and a PUT > of the Media Resource in two distinct HTTP requests. ... at http://www.mail-archive.com/swo...@li.../msg00327.html #### Retrieve a service document (GET) (For brevity, we only include 2 of the 10 collections in the Service Document) [root@logus sss-client]# curl -s http://sword:sword@localhost:8080/sd-uri <service xmlns:dcterms="http://purl.org/dc/terms/" xmlns:sword="http://purl.org/net/sword/terms/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns="http://www.w3.org/2007/app"> <sword:version>2.0</sword:version> <sword:maxUploadSize>16777216</sword:maxUploadSize> <workspace> <atom:title>Main Site</atom:title> <collection href="http://localhost:8080/col-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9"> <atom:title>Collection 4e1f7a9a-f8d4-4795-b230-195acb6680c9</atom:title> <accept>*/*</accept> <accept alternate="multipart-related">*/*</accept> <sword:collectionPolicy>Collection Policy</sword:collectionPolicy> <dcterms:abstract>Collection Description</dcterms:abstract> <sword:mediation>true</sword:mediation> <sword:treatment>Treatment description</sword:treatment> <sword:acceptPackaging>http://purl.org/net/sword/package/SimpleZip</sword:acceptPackaging> <sword:acceptPackaging>http://purl.org/net/sword/package/Binary</sword:acceptPackaging> <sword:acceptPackaging>http://purl.org/net/sword/package/METSDSpaceSIP</sword:acceptPackaging> <sword:service>http://localhost:8080/sd-uri/95234b85-c987-45f2-bbc6-05f79298db35</sword:service> </collection> <collection href="http://localhost:8080/col-uri/cf9ac85a-2f36-4843-af8f-e00d7b2a1ac5"> <atom:title>Collection cf9ac85a-2f36-4843-af8f-e00d7b2a1ac5</atom:title> <accept>*/*</accept> <accept alternate="multipart-related">*/*</accept> <sword:collectionPolicy>Collection Policy</sword:collectionPolicy> <dcterms:abstract>Collection Description</dcterms:abstract> <sword:mediation>true</sword:mediation> <sword:treatment>Treatment description</sword:treatment> <sword:acceptPackaging>http://purl.org/net/sword/package/SimpleZip</sword:acceptPackaging> <sword:acceptPackaging>http://purl.org/net/sword/package/Binary</sword:acceptPackaging> <sword:acceptPackaging>http://purl.org/net/sword/package/METSDSpaceSIP</sword:acceptPackaging> <sword:service>http://localhost:8080/sd-uri/bd46dd22-420c-4606-abe4-8c6f5eccbb63</sword:service> </collection> </workspace> </service> #### Create a resource from an atom entry (POST) We POST (using --data-binary) our atom-entry-study.xml content... [root@logus sss-client]# cat atom-entry-study.xml <?xml version="1.0"?> <entry xmlns="http://www.w3.org/2005/Atom" xmlns:dcterms="http://purl.org/dc/terms/"> <title>The first study for the New England Journal of Coffee dataverse</title> <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id> <updated>2005-10-07T17:17:08Z</updated> <author><name>Creator</name></author> <summary type="text">The abstract</summary> <dcterms:title>Roasting at Home</dcterms:title> <dcterms:creator>Peets, John</dcterms:creator> <dcterms:creator>Stumptown, Jane</dcterms:creator> </entry> [root@logus sss-client]# ... to the first collection we saw in the service document ("4e1f7a9a-f8d4-4795-b230-195acb6680c9"): [root@logus sss-client]# curl --http1.0 --data-binary "@atom-entry-study.xml" -H "Content-Type: application/atom+xml" http://sword:sword@localhost:8080/col-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9 <entry xmlns:dcterms="http://purl.org/dc/terms/" xmlns:sword="http://purl.org/net/sword/terms/" xmlns="http://www.w3.org/2005/Atom"> <title>The first study for the New England Journal of Coffee dataverse</title> <id>tag:container@sss/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2</id> <updated>2013-07-17T15:40:32Z</updated> <author> <name>Creator</name> </author> <summary type="text">The abstract</summary> <generator uri="http://www.swordapp.org/sss" version="1.0"/> <dcterms:date>2005-10-07T17:17:08Z</dcterms:date> <dcterms:abstract>The abstract</dcterms:abstract> <dcterms:title>The first study for the New England Journal of Coffee dataverse</dcterms:title> <dcterms:title>Roasting at Home</dcterms:title> <dcterms:creator>Creator</dcterms:creator> <dcterms:creator>Peets, John</dcterms:creator> <dcterms:creator>Stumptown, Jane</dcterms:creator> <sword:verboseDescription>SSS has done this, that and the other to process the deposit</sword:verboseDescription> <sword:treatment>Treatment description</sword:treatment> <link rel="alternate" href="http://localhost:8080/html/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2"/> <content type="application/zip" src="http://localhost:8080/cont-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2"/> <link rel="edit" href="http://localhost:8080/edit-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2"/> <link rel="edit-media" href="http://localhost:8080/em-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2"/> <link rel="edit-media" type="application/atom+xml;type=feed" href="http://localhost:8080/em-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2.atom"/> <link rel="http://purl.org/net/sword/terms/add" href="http://localhost:8080/edit-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2"/> <sword:packaging>http://purl.org/net/sword/package/SimpleZip</sword:packaging> <link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed" href="http://localhost:8080/state-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2.atom"/> <link rel="http://purl.org/net/sword/terms/statement" type="application/rdf+xml" href="http://localhost:8080/state-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2.rdf"/> </entry> This creates a number of files under the "4e1f7a9a-f8d4-4795-b230-195acb6680c9" collection: [root@logus sss-client]# cd /tmp/sss-server/store/4e1f7a9a-f8d4-4795-b230-195acb6680c9 [root@logus 4e1f7a9a-f8d4-4795-b230-195acb6680c9]# ls -1d */* 70566b02-76a9-496d-b1bb-356ba9acc7f2/atom.xml 70566b02-76a9-496d-b1bb-356ba9acc7f2/sss_deposit-receipt.xml 70566b02-76a9-496d-b1bb-356ba9acc7f2/sss_metadata.xml 70566b02-76a9-496d-b1bb-356ba9acc7f2/sss_statement.atom.xml 70566b02-76a9-496d-b1bb-356ba9acc7f2/sss_statement.xml We can think of "70566b02-76a9-496d-b1bb-356ba9acc7f2" as the unique identifier for the resource we just created. It contains a number of files: - `atom.xml` matches exactly the `atom-entry-study.xml` file we used to create the resource. - `sss_deposit-receipt.xml` is the output we saw from the curl command when we created the resource. - `sss_metadata.xml` is a simpler version of `atom.xml` (i.e. `<dcterms:creator>` becomes just `<creator>`). - `sss_statement.atom.xml` shows the state of the resource, indicating the following: "The work has passed through review and is now in the archive". - `sss_statement.xml` is similar to `sss_statement.atom.xml` in that it shows the state, but is longer and in RDF format. #### Upload example.zip (PUT) Next we upload example.zip to the resource we created ("70566b02-76a9-496d-b1bb-356ba9acc7f2"). We use --upload-file for this, which does a PUT. [root@logus 4e1f7a9a-f8d4-4795-b230-195acb6680c9]# cd /tmp/sss-client [root@logus sss-client]# curl --upload-file example.zip -H "Content-Disposition: filename=example.zip" -H "Content-Type: application/zip" http://sword:sword@localhost:8080/em-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2 [root@logus sss-client]# We don't see any output above but the console output indicates some activity... 2013-07-17 16:14:23,311 - sss - INFO - Authentication required 2013-07-17 16:14:23,311 - sss - INFO - Authentication details: sword:sword; On Behalf Of: None /tmp/sss-server/store 2013-07-17 16:14:23,312 - sss - INFO - Received Binary deposit request 2013-07-17 16:14:23,313 - sss - INFO - Replace request has file content - updating 2013-07-17 16:14:23,314 - sss - INFO - Content replaced 127.0.0.1:39706 - - [17/Jul/2013 16:14:23] "HTTP/1.1 PUT /em-uri/4e1f7a9a-f8d4-4795-b230-195acb6680c9/70566b02-76a9-496d-b1bb-356ba9acc7f2" - 204 No Content ... and the files in the directory for the resource have changed: [root@logus 70566b02-76a9-496d-b1bb-356ba9acc7f2]# git status # On branch master # Changed but not updated: # (use "git add/rm <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # deleted: atom.xml # modified: sss_deposit-receipt.xml # modified: sss_statement.atom.xml # modified: sss_statement.xml # # Untracked files: # (use "git add <file>..." to include in what will be committed) # # 2013-07-17T16:14:23Z_example.zip no changes added to commit (use "git add" and/or "git commit -a") [root@logus 70566b02-76a9-496d-b1bb-356ba9acc7f2]# The `atom.xml` file has been deleted, which was an exact copy of the `atom-entry-study.xml` file we used to create the resource. Some of the content from that file such as our `<dcterms:creator>` entries are still preserved in `sss_deposit-receipt.xml` and `sss_metadata.xml`. -- Philip Durbin Software Developer for http://thedata.org http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2013-07-11 17:10:39
|
On 11 July 2013 17:15, Edwin Shin <edw...@yo...> wrote: > > On Jul 11, 2013, at 12:09 PM, Richard Jones <ri...@co...> wrote: > >> Hi Phil, >> >> Yeah, multipart is critically broken across all implementations. The >> Apache Commons/Java approach doesn't work at all (notwithstanding >> Eddie's workaround), and the python implementation is highly variable >> (depending on framework used), but none of them truly work. > > > The commons-fileupload approach doesn't work because that library doesn't claim to do multipart/related and therefore, unsurprisingly, it doesn't. > > It's largely trivial to implement multipart/related support. I reported this bug and provided a fix over a year ago and mentioned it again on the list last October. It's a bit misleading to say it's broken across all platforms when it feels more like a studious avoidance of just applying the bug fix. You reported this bug to Commons File Upload or here? I don't see any pull requests on the JavaServer2.0 library which pertains to this - apologies if I've missed a patch file. I presume your fix only pertains to the java stuff, and not to the python stuff as well? Cheers, Richard > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
|
From: Edwin S. <edw...@yo...> - 2013-07-11 16:15:10
|
On Jul 11, 2013, at 12:09 PM, Richard Jones <ri...@co...> wrote: > Hi Phil, > > Yeah, multipart is critically broken across all implementations. The > Apache Commons/Java approach doesn't work at all (notwithstanding > Eddie's workaround), and the python implementation is highly variable > (depending on framework used), but none of them truly work. The commons-fileupload approach doesn't work because that library doesn't claim to do multipart/related and therefore, unsurprisingly, it doesn't. It's largely trivial to implement multipart/related support. I reported this bug and provided a fix over a year ago and mentioned it again on the list last October. It's a bit misleading to say it's broken across all platforms when it feels more like a studious avoidance of just applying the bug fix. |
|
From: Edwin S. <edw...@yo...> - 2013-07-11 15:52:13
|
You should double-check if it's indeed multipart/related that Servlet 3.0 supports and not just multipart/form-data (my internet connection is flaking out right now so can't seem to get to any of the spec pages). Also, the subsequent commit I linked to stops the nonsense of reading everything into memory. On Jul 10, 2013, at 6:42 PM, Philip Durbin <phi...@ha...> wrote: > Very helpful, thanks, Eddie. I see one of your commit comments is this... > > "Best as I can tell, commons-fileupload doesn't actually handle > multipart/related correctly. > > This is currently a naive hack that reads everything into memory, but > it at least parses multipart/related correctly (if you have the heap > space)." > > I wonder what you think about this comment from > http://irclog.greptilian.com/sourcefu/2013-07-10#i_10152 ... > > "semiosis: side note, servlet 3 supports multipart, no need for apache > commons multipart anymore" > > Phil > > On Wed, Jul 10, 2013 at 5:28 PM, Edwin Shin > <edw...@yo...> wrote: >> Philip, >> >> If this is related to, ahem, multipart-related support, that simply didn't work in swordapp (at least last year). It was one of the issues I addressed in my fork, see: >> >> https://github.com/mediashelf/sword2-server/commit/5458968f7d4e73c76bd35c18f6d4102d99663db3 >> >> https://github.com/mediashelf/sword2-server/commit/1f6c677c36b409cb929370df31d28e162cf56632 >> >> -Eddie >> >> On Jul 10, 2013, at 5:05 PM, Philip Durbin <phi...@ha...> wrote: >> >>> I'm still using sword2-server-1.0-classes.jar and got a strange >>> exception today when trying to make deposit.isMultipart() true with >>> the proper arguments to curl. >>> >>> I'm able to reproduce the exception in my Vagrant environment so I >>> just added this issue: >>> >>> MalformedStreamException with Content-Type: multipart/related · Issue >>> #2 · dvn/swordpoc - https://github.com/dvn/swordpoc/issues/2 >>> >>> Anyway, I'm still investigating this but I thought I'd bring it up >>> while it's on my mind. >>> >>> Before working on this too hard I'll probably return to cases where >>> deposit.isBinaryOnly() or deposit.isEntryOnly() is true... seems like >>> I'll want to get multipart working eventually though... >>> >>> Phil >>> >>> p.s. Here's what I wrote in that issue #2: >>> >>> While looking at isMultipart() in >>> https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/Deposit.java >>> I attempted to use curl to upload multipart.dat from >>> https://github.com/swordapp/Simple-Sword-Server/blob/master/tests/resources/multipart.dat >>> with these commands... >>> >>> wget https://raw.github.com/swordapp/Simple-Sword-Server/master/tests/resources/multipart.dat >>> curl --insecure --data-binary "@multipart.dat" -H 'Content-Type: >>> multipart/related; boundary="===============0670350989=="' -H >>> "MIME-Version: 1.0" >>> https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 >>> >>> ... and got this exception: >>> >>> [#|2013-07-10T21:39:34.677+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=19;_ThreadName=Thread-2;|StandardWrapperValve[collection]: >>> PWC1406: Servlet.service() for servlet collection threw exception >>> org.apache.commons.fileupload.MultipartStream$MalformedStreamException: >>> Header section has more than 10240 bytes (maybe it is not properly >>> terminated) >>> at org.apache.commons.fileupload.MultipartStream.readHeaders(MultipartStream.java:542) >>> at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:976) >>> at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.<init>(FileUploadBase.java:942) >>> at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331) >>> at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:349) >>> at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126) >>> at org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) >>> at org.swordapp.server.SwordAPIEndpoint.addDepositPropertiesFromMultipart(SwordAPIEndpoint.java:235) >>> at org.swordapp.server.CollectionAPI.post(CollectionAPI.java:152) >>> at org.swordapp.server.servlets.CollectionServletDefault.doPost(CollectionServletDefault.java:48) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) >>> at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) >>> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281) >>> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) >>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) >>> at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) >>> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161) >>> at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331) >>> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231) >>> at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317) >>> at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195) >>> at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860) >>> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757) >>> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056) >>> at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229) >>> at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) >>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) >>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) >>> at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) >>> at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) >>> at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) >>> at com.sun.grizzly.ContextTask.run(ContextTask.java:71) >>> at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) >>> at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) >>> is this line... >>> >>> List<DiskFileItem> items = upload.parseRequest(request); >>> >>> ... which can be seen at >>> https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/SwordAPIEndpoint.java#L496 >>> >>> >>> -- >>> Philip Durbin >>> Software Developer for http://thedata.org >>> http://www.iq.harvard.edu/people/philip-durbin >>> >>> ------------------------------------------------------------------------------ >>> See everything from the browser to the database with AppDynamics >>> Get end-to-end visibility with application monitoring from AppDynamics >>> Isolate bottlenecks and diagnose root cause in seconds. >>> Start your free trial of AppDynamics Pro today! >>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> sword-app-tech mailing list >>> swo...@li... >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> > > > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin |
|
From: Richard J. <ri...@co...> - 2013-07-11 15:09:53
|
Hi Phil, Yeah, multipart is critically broken across all implementations. The Apache Commons/Java approach doesn't work at all (notwithstanding Eddie's workaround), and the python implementation is highly variable (depending on framework used), but none of them truly work. It is my plan and hope to back all of the multipart OUT of the sword spec for a future version (like a 2.1), so I strongly recommend not using multipart deposit. Instead do a POST of an Atom Entry and a PUT of the Media Resource in two distinct HTTP requests. Cheers, Richard On 10 July 2013 22:42, Philip Durbin <phi...@ha...> wrote: > Very helpful, thanks, Eddie. I see one of your commit comments is this... > > "Best as I can tell, commons-fileupload doesn't actually handle > multipart/related correctly. > > This is currently a naive hack that reads everything into memory, but > it at least parses multipart/related correctly (if you have the heap > space)." > > I wonder what you think about this comment from > http://irclog.greptilian.com/sourcefu/2013-07-10#i_10152 ... > > "semiosis: side note, servlet 3 supports multipart, no need for apache > commons multipart anymore" > > Phil > > On Wed, Jul 10, 2013 at 5:28 PM, Edwin Shin > <edw...@yo...> wrote: >> Philip, >> >> If this is related to, ahem, multipart-related support, that simply didn't work in swordapp (at least last year). It was one of the issues I addressed in my fork, see: >> >> https://github.com/mediashelf/sword2-server/commit/5458968f7d4e73c76bd35c18f6d4102d99663db3 >> >> https://github.com/mediashelf/sword2-server/commit/1f6c677c36b409cb929370df31d28e162cf56632 >> >> -Eddie >> >> On Jul 10, 2013, at 5:05 PM, Philip Durbin <phi...@ha...> wrote: >> >>> I'm still using sword2-server-1.0-classes.jar and got a strange >>> exception today when trying to make deposit.isMultipart() true with >>> the proper arguments to curl. >>> >>> I'm able to reproduce the exception in my Vagrant environment so I >>> just added this issue: >>> >>> MalformedStreamException with Content-Type: multipart/related · Issue >>> #2 · dvn/swordpoc - https://github.com/dvn/swordpoc/issues/2 >>> >>> Anyway, I'm still investigating this but I thought I'd bring it up >>> while it's on my mind. >>> >>> Before working on this too hard I'll probably return to cases where >>> deposit.isBinaryOnly() or deposit.isEntryOnly() is true... seems like >>> I'll want to get multipart working eventually though... >>> >>> Phil >>> >>> p.s. Here's what I wrote in that issue #2: >>> >>> While looking at isMultipart() in >>> https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/Deposit.java >>> I attempted to use curl to upload multipart.dat from >>> https://github.com/swordapp/Simple-Sword-Server/blob/master/tests/resources/multipart.dat >>> with these commands... >>> >>> wget https://raw.github.com/swordapp/Simple-Sword-Server/master/tests/resources/multipart.dat >>> curl --insecure --data-binary "@multipart.dat" -H 'Content-Type: >>> multipart/related; boundary="===============0670350989=="' -H >>> "MIME-Version: 1.0" >>> https://sword:sword@localhost:8181/swordpoc/collection/a4f21cdc-f20c-4c82-b63e-5df81f809417 >>> >>> ... and got this exception: >>> >>> [#|2013-07-10T21:39:34.677+0200|WARNING|glassfish3.1.2|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=19;_ThreadName=Thread-2;|StandardWrapperValve[collection]: >>> PWC1406: Servlet.service() for servlet collection threw exception >>> org.apache.commons.fileupload.MultipartStream$MalformedStreamException: >>> Header section has more than 10240 bytes (maybe it is not properly >>> terminated) >>> at org.apache.commons.fileupload.MultipartStream.readHeaders(MultipartStream.java:542) >>> at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.findNextItem(FileUploadBase.java:976) >>> at org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.<init>(FileUploadBase.java:942) >>> at org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331) >>> at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:349) >>> at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest(ServletFileUpload.java:126) >>> at org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) >>> at org.swordapp.server.SwordAPIEndpoint.addDepositPropertiesFromMultipart(SwordAPIEndpoint.java:235) >>> at org.swordapp.server.CollectionAPI.post(CollectionAPI.java:152) >>> at org.swordapp.server.servlets.CollectionServletDefault.doPost(CollectionServletDefault.java:48) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:688) >>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) >>> at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) >>> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281) >>> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) >>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) >>> at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) >>> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161) >>> at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331) >>> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231) >>> at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317) >>> at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195) >>> at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860) >>> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757) >>> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056) >>> at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229) >>> at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) >>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) >>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) >>> at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) >>> at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) >>> at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) >>> at com.sun.grizzly.ContextTask.run(ContextTask.java:71) >>> at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) >>> at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> org.swordapp.server.SwordAPIEndpoint.getPartsFromRequest(SwordAPIEndpoint.java:496) >>> is this line... >>> >>> List<DiskFileItem> items = upload.parseRequest(request); >>> >>> ... which can be seen at >>> https://github.com/swordapp/JavaServer2.0/blob/314f4dab35d801397be3e0c5a9c7e53c4b736bef/src/main/java/org/swordapp/server/SwordAPIEndpoint.java#L496 >>> >>> >>> -- >>> Philip Durbin >>> Software Developer for http://thedata.org >>> http://www.iq.harvard.edu/people/philip-durbin >>> >>> ------------------------------------------------------------------------------ >>> See everything from the browser to the database with AppDynamics >>> Get end-to-end visibility with application monitoring from AppDynamics >>> Isolate bottlenecks and diagnose root cause in seconds. >>> Start your free trial of AppDynamics Pro today! >>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> sword-app-tech mailing list >>> swo...@li... >>> https://lists.sourceforge.net/lists/listinfo/sword-app-tech >> > > > > -- > Philip Durbin > Software Developer for http://thedata.org > http://www.iq.harvard.edu/people/philip-durbin > > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > _______________________________________________ > sword-app-tech mailing list > swo...@li... > https://lists.sourceforge.net/lists/listinfo/sword-app-tech -- Richard Jones, Founder, Cottage Labs t: @richard_d_jones, @cottagelabs w: http://cottagelabs.com |
13 messages has been excluded from this view by a project administrator.
