connotea-code-devel

[Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

Hi there,

  Is there anything wrong with this ?
http://www.connotea.org/data/add?uri=3Dhttp%3A%2F%=20
2Fbioinfo.icapture.ubc.ca%2Fekawas&tags=3Dtest%2C+%22test+post%22=20
+another+test

I think authentication is working fine, but I get a 501 response

=E2=88=92
	<rdf:RDF>
=E2=88=92
	<Response rdf:about=3D"">
<code>501</code>
<message>Not Implemented</message>
<isFailure>1</isFailure>
<user>MrED</user>
<apiVersion>0.1.0</apiVersion>
<bibliotechVersion>1.7.1</bibliotechVersion>
</Response>
</rdf:RDF>

whats up ?

thanks
-Ben

Re: [Connotea-code-devel] post not implemented ?

[Martin F. <ma...@ne...>]

Ben,

Has to be HTTP POST not GET.

I'm attaching a script that I developed for testing.

Martin

Benjamin Good wrote:
> Hi there,
> 
>   Is there anything wrong with this ?
> http://www.connotea.org/data/add?uri=http%3A%2F% 
> 2Fbioinfo.icapture.ubc.ca%2Fekawas&tags=test%2C+%22test+post%22 
> +another+test
> 
> I think authentication is working fine, but I get a 501 response
> 
> −
> 	<rdf:RDF>
> −
> 	<Response rdf:about="">
> <code>501</code>
> <message>Not Implemented</message>
> <isFailure>1</isFailure>
> <user>MrED</user>
> <apiVersion>0.1.0</apiVersion>
> <bibliotechVersion>1.7.1</bibliotechVersion>
> </Response>
> </rdf:RDF>
> 
> whats up ?
> 
> thanks
> -Ben

Re: [Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

Hi Martin,

Switching to POST changes the error to a 400 .

I have it working from a simple html form posting through firefox, =20
but am trying to get it going in java.  The java program succeeds in =20
GETting data from connotea - just can't post.
Is there a particular user-agent or other request property that i =20
should be using?

thanks
-Ben


On Aug 2, 2007, at 12:34 PM, Martin Flack wrote:

> Ben,
>
> Has to be HTTP POST not GET.
>
> I'm attaching a script that I developed for testing.
>
> Martin
>
> Benjamin Good wrote:
>> Hi there,
>>   Is there anything wrong with this ?
>> http://www.connotea.org/data/add?uri=3Dhttp%3A%2F% =20
>> 2Fbioinfo.icapture.ubc.ca%2Fekawas&tags=3Dtest%2C+%22test+post%22 =20
>> +another+test
>> I think authentication is working fine, but I get a 501 response
>> =E2=88=92
>> 	<rdf:RDF>
>> =E2=88=92
>> 	<Response rdf:about=3D"">
>> <code>501</code>
>> <message>Not Implemented</message>
>> <isFailure>1</isFailure>
>> <user>MrED</user>
>> <apiVersion>0.1.0</apiVersion>
>> <bibliotechVersion>1.7.1</bibliotechVersion>
>> </Response>
>> </rdf:RDF>
>> whats up ?
>> thanks
>> -Ben
>
> #!/bin/bash
> # post something to Connotea from the command line, requires lwp-=20
> request
> # lwp-request is part of libwww-perl which is a package on some =20
> Linux platforms and available on CPAN
> # edit the variables below for input
> # output is a dump of the HTTP transaction to stdout including =20
> Connotea's XML response
>
> user=3DEDIT
> pass=3DEDIT
> host=3Dwww.connotea.org
> # note all following fields should be url escaped
> uri=3Dhttp%3A//www.cnn.com
> # separate tags with %20 (spacebar)
> tags=3Dabc%20def%20ghi
> # could also pass in usertitle, description, mywork, private, etc.
>
> echo "uri=3D$uri&tags=3D$tags" | lwp-request -C "$user:$pass" -u -U -s =
-=20
> S -e -m POST "http://$host/data/add"
>

Re: [Connotea-code-devel] post not implemented ?

[Martin F. <ma...@ne...>]

Try adding WWW::Connotea in the User-Agent string.

Other than that just try to see what the Java program is doing 
differently...?

Martin

Benjamin Good wrote:
> Hi Martin,
> 
> Switching to POST changes the error to a 400 .
> 
> I have it working from a simple html form posting through firefox,  
> but am trying to get it going in java.  The java program succeeds in  
> GETting data from connotea - just can't post.
> Is there a particular user-agent or other request property that i  
> should be using?
> 
> thanks
> -Ben

Re: [Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

Had already tried that one (as well as pretending to be a browser).
Doing my best to figure out what the difference is.. but no luck so  
far.  Here is the method I'm using (authentication not included but  
working fine).

-ben

public String postToConnotea(String url){
		String response = "";
		BufferedReader in = null;
		try {
			URL connoU = new URL(url);

			HttpURLConnection conno = (HttpURLConnection) connoU.openConnection 
();
			conno.setRequestProperty("User-Agent", "WWW::Connotea");
			conno.setRequestMethod("POST");
			conno.setDoOutput(true);
			int code = conno.getResponseCode();
			System.out.println("code "+code+" for request url "+url);
			switch(code){
			case 400:
				conno.disconnect();
				System.err.println("400 - Server did not like form of the request");
				return null;
			case 401:
				conno.disconnect();
				System.err.println("401 - Server did not accept user credentials");
				return null;
			case 404:
				conno.disconnect();
				System.err.println("404 - Query returned no results");
				return null;
			case 503:
				conno.disconnect();
				System.err.println("503 - Connotea server failed to respond");
				return null;
			default: //200 all good
				System.out.println("server accepted request, retrieving data");
				in = new BufferedReader(
						new InputStreamReader(
								conno.getInputStream()));
				if(in!=null){
					response = in.readLine();
					in.close();
				}
				conno.disconnect();
			}

		}catch (MalformedURLException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (Exception e) { //catch other exceptions here (like  
SaxParsing exceptions reading RDF)
			e.printStackTrace();
		}
		return response;
	}




On Aug 2, 2007, at 2:28 PM, Martin Flack wrote:

> Try adding WWW::Connotea in the User-Agent string.
>
> Other than that just try to see what the Java program is doing  
> differently...?
>
> Martin
>
> Benjamin Good wrote:
>> Hi Martin,
>> Switching to POST changes the error to a 400 .
>> I have it working from a simple html form posting through  
>> firefox,  but am trying to get it going in java.  The java program  
>> succeeds in  GETting data from connotea - just can't post.
>> Is there a particular user-agent or other request property that i   
>> should be using?
>> thanks
>> -Ben
>

Re: [Connotea-code-devel] post not implemented ?

[Martin F. <ma...@ne...>]

Your example does not seem to set a payload though. The URL should be 
free of parameters and their should be content that contains the 
parameters, encoded with application/x-www-form-urlencoded.

Benjamin Good wrote:
> Had already tried that one (as well as pretending to be a browser).
> Doing my best to figure out what the difference is.. but no luck so 
> far.  Here is the method I'm using (authentication not included but 
> working fine).
> 
> -ben

Re: [Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

Usually adding the parameters into the URL works .. why would it not  
in this case?

I get the same error (400) if I set the params in the code

	URL connoU = new URL("http://www.connotea.org/data/add");
			
	HttpURLConnection conno = (HttpURLConnection) connoU.openConnection();
	conno.setRequestProperty("User-Agent", "WWW::Connotea");
	conno.setRequestProperty("uri", URLEncoder.encode(uri,"UTF-8"));
	conno.setRequestProperty("tag", URLEncoder.encode(tag,"UTF-8"));
	conno.setRequestMethod("POST");

I've been digging for some else that has this running in Java, but  
the only one I could find (from Pierre Lindenbaum) didn't use the web  
api - it emulated a user session by storing and sending cookies for  
authentication and sending requests including button clicks.  (His  
ConnoteaClient program - but it doesn't seem to be work ing anymore).

-Ben


On Aug 2, 2007, at 4:57 PM, Martin Flack wrote:

> Your example does not seem to set a payload though. The URL should  
> be free of parameters and their should be content that contains the  
> parameters, encoded with application/x-www-form-urlencoded.
>
> Benjamin Good wrote:
>> Had already tried that one (as well as pretending to be a browser).
>> Doing my best to figure out what the difference is.. but no luck  
>> so far.  Here is the method I'm using (authentication not included  
>> but working fine).
>> -ben
>

Re: [Connotea-code-devel] post not implemented ?

[Martin F. <ma...@ne...>]

I don't know Java libraries, but it seems unlikely that you should be 
using setRequestProperty() for both User-Agent (an HTTP header) and uri 
(a form parameter).

POST's are supposed to put their parameters in the payload (like your 
web browser does) by spec.

Take a look at how they encode and attach the form payload on these 
examples:
http://www.javaworld.com/javaworld/javatips/jw-javatip34.html
http://developers.sun.com/mobility/midp/ttips/HTTPPost/

Cheers,
Martin

Benjamin Good wrote:
> Usually adding the parameters into the URL works .. why would it not in 
> this case?
> 
> I get the same error (400) if I set the params in the code
> 
> URL connoU = new URL("http://www.connotea.org/data/add");
> 
> 
> HttpURLConnection conno = (HttpURLConnection) connoU.openConnection();
> conno.setRequestProperty("User-Agent", "WWW::Connotea");
> conno.setRequestProperty("uri", URLEncoder.encode(uri,"UTF-8"));
> conno.setRequestProperty("tag", URLEncoder.encode(tag,"UTF-8"));
> conno.setRequestMethod("POST");
> 
> I've been digging for some else that has this running in Java, but the 
> only one I could find (from Pierre Lindenbaum) didn't use the web api - 
> it emulated a user session by storing and sending cookies for 
> authentication and sending requests including button clicks.  (His 
> ConnoteaClient program - but it doesn't seem to be work ing anymore).
> 
> -Ben

Re: [Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

OK,

Apologies all, for using this list as my own personal "intro to java  
web programming" course..  Just to finish this off, the following  
code frag does what I was trying to do

String encodedData = "uri="+URLEncoder.encode(uri,"UTF-8") 
+"&tags="+URLEncoder.encode(tags,"UTF-8");		
URL connoU = new URL("http://www.connotea.org/data/add");			
HttpURLConnection conno = (HttpURLConnection) connoU.openConnection();	
conno.setDoOutput(true);
OutputStream os = conno.getOutputStream();
os.write( encodedData.getBytes() );
int code = conno.getResponseCode();
System.out.println("response code from connotea="+code);

If anyone else is having trouble reading and writing to the api from  
java, please do let me know - think I've made it through most of the  
cases.  The only one I haven't finished yet is the job of writing to  
the connotea wiki from code.  Seems that it is possible to do that  
via browser emulation (cookies etc.) but it would be cool if it was  
added to the API.

cheers all
-Ben



On Aug 2, 2007, at 5:45 PM, Martin Flack wrote:

> I don't know Java libraries, but it seems unlikely that you should  
> be using setRequestProperty() for both User-Agent (an HTTP header)  
> and uri (a form parameter).
>
> POST's are supposed to put their parameters in the payload (like  
> your web browser does) by spec.
>
> Take a look at how they encode and attach the form payload on these  
> examples:
> http://www.javaworld.com/javaworld/javatips/jw-javatip34.html
> http://developers.sun.com/mobility/midp/ttips/HTTPPost/
>
> Cheers,
> Martin
>
> Benjamin Good wrote:
>> Usually adding the parameters into the URL works .. why would it  
>> not in this case?
>> I get the same error (400) if I set the params in the code
>> URL connoU = new URL("http://www.connotea.org/data/add");
>> HttpURLConnection conno = (HttpURLConnection) connoU.openConnection 
>> ();
>> conno.setRequestProperty("User-Agent", "WWW::Connotea");
>> conno.setRequestProperty("uri", URLEncoder.encode(uri,"UTF-8"));
>> conno.setRequestProperty("tag", URLEncoder.encode(tag,"UTF-8"));
>> conno.setRequestMethod("POST");
>> I've been digging for some else that has this running in Java, but  
>> the only one I could find (from Pierre Lindenbaum) didn't use the  
>> web api - it emulated a user session by storing and sending  
>> cookies for authentication and sending requests including button  
>> clicks.  (His ConnoteaClient program - but it doesn't seem to be  
>> work ing anymore).
>> -Ben
>

Re: [Connotea-code-devel] post not implemented ?

[Christopher T. <chr...@ch...>]

Benjamin Good wrote:
> If anyone else is having trouble reading and writing to the api from  
> java, please do let me know - think I've made it through most of the  
> cases.  The only one I haven't finished yet is the job of writing to  
> the connotea wiki from code.  Seems that it is possible to do that  
> via browser emulation (cookies etc.) but it would be cool if it was  
> added to the API.
> 

Hi Ben,

(To introduce myself: I'm a developer in the Emerging Technologies
Department at Nature.)

Just so you know, I'm just in the process of wrapping up a Java Connotea
API client. I haven't encountered any major problems implementing this
yet (fingers crossed ;D) ... there are a few small things that I was
going to summarize on this list as soon as I had wrapped the thing up,
but they were mainly questions to do with things like  "Why not HTTP
delete for remove?" etc.

Posting does work - my client is built on Restlet
(http://www.restlet.org/), which I can highly recommend and it will make
your task a million times simpler than using HttpURLConnection and
friends directly.

Here is a sample piece of code I wrote during development which works
for me and posts a link to Connotea via the API:


package org.connotea;

import org.restlet.Client;
import org.restlet.data.ChallengeResponse;
import org.restlet.data.ChallengeScheme;
import org.restlet.data.Form;
import org.restlet.data.Method;
import org.restlet.data.Parameter;
import org.restlet.data.Protocol;
import org.restlet.data.Reference;
import org.restlet.data.Request;
import org.restlet.data.Response;

public class Driver
{
    public static void main(String[] args) throws Throwable
    {
        Client client = new Client(Protocol.HTTP);

        Reference reference = new
Reference("http://www.connotea.org/data/add");

        Request request = new Request(Method.POST, reference);

        ChallengeScheme challengeScheme = ChallengeScheme.HTTP_BASIC;
        ChallengeResponse challengeResponse = new ChallengeResponse(
                challengeScheme, "your_username", "your_password");
        request.setChallengeResponse(challengeResponse);

        Form form = new Form();
        form.add(new Parameter("uri", "http://www.nature.com/"));
        form.add(new Parameter("tags", "science journal"));
        form.encode();

        request.setEntity(form.getWebRepresentation());

        Response response = client.handle(request);
        System.out.println(response.getStatus());
    }
}

I hope to complete a beta of the Java client I am working on today, as
it happens, so you could just wait for that if you like. The above code
demonstrates how Java client is really working "under the hood", but it
provides a front-end and Connotea "domain objects" which encapsulate all
the currently implemented API methods for convenient use in your
applications.

All the best,

Chris

p.s. has anyone else noticed that if you pass an invalid (i.e.
unencoded) url to /data/remove you get a default server 404, whereas
doing the same with any of the gets returns an RDF response? Seems like
one to fix?

Re: [Connotea-code-devel] post not implemented ?

[Martin F. <ma...@ne...>]

Thanks Chris, looking forward to the Java API.

Christopher Townson wrote:
> p.s. has anyone else noticed that if you pass an invalid (i.e.
> unencoded) url to /data/remove you get a default server 404, whereas
> doing the same with any of the gets returns an RDF response? Seems like
> one to fix?

Interesting. The code doesn't specify that to happen so it must be a 
mod_perl issue. I'll put it on the to-do list. Thanks.

Martin

Re: [Connotea-code-devel] post not implemented ?

[Benjamin G. <go...@in...>]

Yes..  that did seem strange and is, of course, wrong.  Apologies for  
my idiocy.
However, I can't get it to work with the methods you directed me to  
either.  Always get the 400 response.

any other ideas?
-Ben

String uri = "http://www.cnn.com";
String tags = "test123";

String encodedData = URLEncoder.encode("uri="+uri 
+"&tags="+tags,"UTF-8");
		
			URL connoU = new URL("http://www.connotea.org/data/add");			
			HttpURLConnection conno = (HttpURLConnection) connoU.openConnection 
();
			
			conno.setRequestProperty("User-Agent", "WWW::Connotea");
			conno.setRequestProperty("Content-Type", "application/x-www-form- 
urlencoded");
			conno.setRequestMethod("POST");
			conno.setRequestProperty( "Content-Length", ""+encodedData.length 
() );
			conno.setDoOutput(true);
			conno.setDoInput(true);
			conno.setUseCaches(false);
			OutputStream os = conno.getOutputStream();
			os.write( encodedData.getBytes() );
			os.flush();
			os.close();
			int code = conno.getResponseCode();
			System.out.println("response code from connotea="+code);



On Aug 2, 2007, at 5:45 PM, Martin Flack wrote:

> I don't know Java libraries, but it seems unlikely that you should  
> be using setRequestProperty() for both User-Agent (an HTTP header)  
> and uri (a form parameter).
>
> POST's are supposed to put their parameters in the payload (like  
> your web browser does) by spec.
>
> Take a look at how they encode and attach the form payload on these  
> examples:
> http://www.javaworld.com/javaworld/javatips/jw-javatip34.html
> http://developers.sun.com/mobility/midp/ttips/HTTPPost/
>
> Cheers,
> Martin
>
> Benjamin Good wrote:
>> Usually adding the parameters into the URL works .. why would it  
>> not in this case?
>> I get the same error (400) if I set the params in the code
>> URL connoU = new URL("http://www.connotea.org/data/add");
>> HttpURLConnection conno = (HttpURLConnection) connoU.openConnection 
>> ();
>> conno.setRequestProperty("User-Agent", "WWW::Connotea");
>> conno.setRequestProperty("uri", URLEncoder.encode(uri,"UTF-8"));
>> conno.setRequestProperty("tag", URLEncoder.encode(tag,"UTF-8"));
>> conno.setRequestMethod("POST");
>> I've been digging for some else that has this running in Java, but  
>> the only one I could find (from Pierre Lindenbaum) didn't use the  
>> web api - it emulated a user session by storing and sending  
>> cookies for authentication and sending requests including button  
>> clicks.  (His ConnoteaClient program - but it doesn't seem to be  
>> work ing anymore).
>> -Ben
>