Menu
▾
▴
connotea-code-devel
|
From: Benjamin G. <go...@in...> - 2006-08-11 06:31:06
|
Hi all, Got another error of the same type as before: Character reference "" is an invalid XML character. Suppose I'll have to implement some error handling on this end. Aside from that, I can't seem to go more than a few requests without hitting a 503. As far as I can tell I should not be hitting the throttling rules, but perhaps I am?? Out of curiosity, is anyone else making frequent hits to the web api? I'm getting a little concerned that I won't be able to produce the little application I had hoped for do to the current instability of the server. If some one else is making frequent calls without issue, then I must be screwing something up - otherwise.. I don't know - perhaps I should wait until an official release comes out? I totally understand that this is still a work in progress - its just so cool that I'm impatient to start playing with it (and this app would fit into a paper I'm working on quite nicely). thanks -Ben(G) |
|
From: Martin F. <ma...@ne...> - 2006-08-11 14:31:20
|
Benjamin Good wrote: > Got another error of the same type as before: Character reference > "" is an invalid XML character. Suppose I'll have to implement some > error handling on this end. Do you happen to know the query that got you that? > Aside from that, I can't seem to go more than a few requests without > hitting a 503. As far as I can tell I should not be hitting the > throttling rules, but perhaps I am?? Do you have any control over your user agent string? Could you append " WWW::Connotea" to it to indicate a Web API compliant application? It's the string we use in the WWW::Connotea Perl library but it's a fast cheap hack to giving you "human" level access. Martin |
|
From: Benjamin G. <go...@in...> - 2006-08-11 15:32:42
|
On Aug 11, 2006, at 7:32 AM, Martin Flack wrote: > Benjamin Good wrote: >> Got another error of the same type as before: Character reference >> "" is an invalid XML character. Suppose I'll have to >> implement some error handling on this end. > > Do you happen to know the query that got you that? q=astronomy > >> Aside from that, I can't seem to go more than a few requests >> without hitting a 503. As far as I can tell I should not be >> hitting the throttling rules, but perhaps I am?? > > Do you have any control over your user agent string? I'm looking into it. I think the answer will eventually be yes, but its not obvious to me how to accomplish this yet. > > Could you append " WWW::Connotea" to it to indicate a Web API > compliant application? It's the string we use in the WWW::Connotea > Perl library but it's a fast cheap hack to giving you "human" level > access. As soon as I can figure out how I will. It seems dangerous though, what is the point of implementing API-level throttling if a hacker/ bot could hide so easily? Why not implement separate or privileged account status for web API users? -Ben(G) > > Martin > |
|
From: Martin F. <ma...@ne...> - 2006-08-11 19:06:22
|
Benjamin Good wrote: >> Could you append " WWW::Connotea" to it to indicate a Web API >> compliant application? It's the string we use in the WWW::Connotea >> Perl library but it's a fast cheap hack to giving you "human" level >> access. > > As soon as I can figure out how I will. It seems dangerous though, what > is the point of implementing API-level throttling if a hacker/bot could > hide so easily? Why not implement separate or privileged account status > for web API users? We probably will; I'll talk to Ben (L) about using logged in status as part of the throttling, but he's apparently out today. It's all a bit of a game. You mention hackers and bots in one phrase but they are separate. Hackers can always apply for and receive whatever authorization they need. (I mean, put it this way, I don't know *you're* not a nefarious mortgage offer poster. ;-) Just kidding...) But what we're trying to stem the tide of is bots that are running all over the web, or even just social bookmarking sites, whose authors have not necessarily done research on Connotea in particular. A few simple obstacles such as these do help a lot with that crowd. Martin |
|
From: Robert F. <xro...@go...> - 2006-08-22 09:26:40
|
just one more question regarding the throttling: why not send a retry-after header with the http 503 response? i just tried to figure out a smart default retry-after period, but couldn't come up with one. so if the throttling is already based on data like the server load, it should be possible to compute some estimates about when the load may be better. what do you think? robert On 8/11/06, Martin Flack <ma...@ne...> wrote: > Benjamin Good wrote: > >> Could you append " WWW::Connotea" to it to indicate a Web API > >> compliant application? It's the string we use in the WWW::Connotea > >> Perl library but it's a fast cheap hack to giving you "human" level > >> access. > > > > As soon as I can figure out how I will. It seems dangerous though, what > > is the point of implementing API-level throttling if a hacker/bot could > > hide so easily? Why not implement separate or privileged account status > > for web API users? > > We probably will; I'll talk to Ben (L) about using logged in status as > part of the throttling, but he's apparently out today. > > It's all a bit of a game. You mention hackers and bots in one phrase but > they are separate. Hackers can always apply for and receive whatever > authorization they need. (I mean, put it this way, I don't know *you're* > not a nefarious mortgage offer poster. ;-) Just kidding...) But what > we're trying to stem the tide of is bots that are running all over the > web, or even just social bookmarking sites, whose authors have not > necessarily done research on Connotea in particular. A few simple > obstacles such as these do help a lot with that crowd. > > Martin > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Connotea-code-devel mailing list > Con...@li... > https://lists.sourceforge.net/lists/listinfo/connotea-code-devel > |
|
From: Martin F. <ma...@ne...> - 2006-08-22 14:29:41
|
There should be one there actually. We're using the rule that says you can say the number of seconds to wait. I'll double check the code. Martin Robert Forkel wrote: > just one more question regarding the throttling: why not send a > retry-after header with the http 503 response? i just tried to figure > out a smart default retry-after period, but couldn't come up with one. > so if the throttling is already based on data like the server load, it > should be possible to compute some estimates about when the load may > be better. > what do you think? > robert > > On 8/11/06, Martin Flack <ma...@ne...> wrote: >> Benjamin Good wrote: >>>> Could you append " WWW::Connotea" to it to indicate a Web API >>>> compliant application? It's the string we use in the WWW::Connotea >>>> Perl library but it's a fast cheap hack to giving you "human" level >>>> access. >>> As soon as I can figure out how I will. It seems dangerous though, what >>> is the point of implementing API-level throttling if a hacker/bot could >>> hide so easily? Why not implement separate or privileged account status >>> for web API users? >> We probably will; I'll talk to Ben (L) about using logged in status as >> part of the throttling, but he's apparently out today. >> >> It's all a bit of a game. You mention hackers and bots in one phrase but >> they are separate. Hackers can always apply for and receive whatever >> authorization they need. (I mean, put it this way, I don't know *you're* >> not a nefarious mortgage offer poster. ;-) Just kidding...) But what >> we're trying to stem the tide of is bots that are running all over the >> web, or even just social bookmarking sites, whose authors have not >> necessarily done research on Connotea in particular. A few simple >> obstacles such as these do help a lot with that crowd. >> >> Martin >> >> ------------------------------------------------------------------------- >> Using Tomcat but need to do more? Need to support web services, security? >> Get stuff done quickly with pre-integrated technology to make your job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Connotea-code-devel mailing list >> Con...@li... >> https://lists.sourceforge.net/lists/listinfo/connotea-code-devel >> > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Connotea-code-devel mailing list > Con...@li... > https://lists.sourceforge.net/lists/listinfo/connotea-code-devel |
|
From: Robert F. <xro...@go...> - 2006-08-22 14:43:28
|
that's what i got: reply: 'HTTP/1.1 503 Service Temporarily Unavailable\r\n' header: Date: Tue, 22 Aug 2006 09:03:14 GMT header: Server: Apache/2.0.46 (Red Hat) header: Last-Modified: Wed, 28 Jun 2006 02:37:18 GMT header: ETag: "33fb1-1c8-b1ab1380" header: Accept-Ranges: bytes header: Content-Length: 456 header: Connection: close header: Content-Type: text/html; charset=UTF-8 Martin Flack wrote: > There should be one there actually. We're using the rule that says you > can say the number of seconds to wait. > > I'll double check the code. > > Martin > > Robert Forkel wrote: >> just one more question regarding the throttling: why not send a >> retry-after header with the http 503 response? i just tried to figure >> out a smart default retry-after period, but couldn't come up with one. >> so if the throttling is already based on data like the server load, it >> should be possible to compute some estimates about when the load may >> be better. >> what do you think? >> robert >> >> On 8/11/06, Martin Flack <ma...@ne...> wrote: >>> Benjamin Good wrote: >>>>> Could you append " WWW::Connotea" to it to indicate a Web API >>>>> compliant application? It's the string we use in the WWW::Connotea >>>>> Perl library but it's a fast cheap hack to giving you "human" level >>>>> access. >>>> As soon as I can figure out how I will. It seems dangerous though, >>>> what >>>> is the point of implementing API-level throttling if a hacker/bot >>>> could >>>> hide so easily? Why not implement separate or privileged account >>>> status >>>> for web API users? >>> We probably will; I'll talk to Ben (L) about using logged in status as >>> part of the throttling, but he's apparently out today. >>> >>> It's all a bit of a game. You mention hackers and bots in one phrase >>> but >>> they are separate. Hackers can always apply for and receive whatever >>> authorization they need. (I mean, put it this way, I don't know >>> *you're* >>> not a nefarious mortgage offer poster. ;-) Just kidding...) But what >>> we're trying to stem the tide of is bots that are running all over the >>> web, or even just social bookmarking sites, whose authors have not >>> necessarily done research on Connotea in particular. A few simple >>> obstacles such as these do help a lot with that crowd. >>> >>> Martin >>> >>> ------------------------------------------------------------------------- >>> >>> Using Tomcat but need to do more? Need to support web services, >>> security? >>> Get stuff done quickly with pre-integrated technology to make your >>> job easier >>> Download IBM WebSphere Application Server v.1.0.1 based on Apache >>> Geronimo >>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>> >>> _______________________________________________ >>> Connotea-code-devel mailing list >>> Con...@li... >>> https://lists.sourceforge.net/lists/listinfo/connotea-code-devel >>> >> >> ------------------------------------------------------------------------- >> >> Using Tomcat but need to do more? Need to support web services, >> security? >> Get stuff done quickly with pre-integrated technology to make your >> job easier >> Download IBM WebSphere Application Server v.1.0.1 based on Apache >> Geronimo >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >> _______________________________________________ >> Connotea-code-devel mailing list >> Con...@li... >> https://lists.sourceforge.net/lists/listinfo/connotea-code-devel > > |
|
From: Martin F. <ma...@ne...> - 2006-08-11 19:57:41
|
Benjamin Good wrote: >> Do you happen to know the query that got you that? > > q=astronomy I've manually taken care of this. I will have to design a systemic fix so these can't be output. Martin |
|
From: Benjamin G. <go...@in...> - 2006-08-11 17:02:11
|
>
> Do you have any control over your user agent string?
>
> Could you append " WWW::Connotea" to it to indicate a Web API
> compliant application? It's the string we use in the WWW::Connotea
> Perl library but it's a fast cheap hack to giving you "human" level
> access.
Missed this in my first Doc scan... Here it is for Java:
HttpURLConnection conno = (HttpURLConnection) connoU.openConnection();
conno.setRequestProperty("User-Agent", "Java/1.5.0_06/WWW::Connotea");
-Ben(G)
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X