connotea-code-devel

[Connotea-code-devel] =?iso-8859-1?q?Perl_-_Encoding_Problems_whi?= =?iso-8859-1?q?le_posting_with_german_=22Umlaute=22_=F6=E4=FC=DF?=

[Jan L <ein...@go...>]

Hi,
I am trying to post entries to connotea with perl on two ways. I am doing
this with the perl module and with LWP::UserAgent. In both I have problems,
that German Umlaute (diacritical marks) are not stored in the right way. So
there is a problem with the encoding but I currently do not have a clue to
solve this problem.

Does someone have a tipp on this?

Here is an example

http://www.connotea.org/uri/e2f5efbaa57e5a1b9a1567e9c44b8be6


Code for using the perl module:

use WWW::Connotea;

use constant USER =3D> 'xxxxxxx';
use constant PASSWORD =3D> 'xxxxxxx';

my $c =3D WWW::Connotea->new(user =3D> USER, password =3D> PASSWORD);
$c->authenticate;

my $p =3D WWW::Connotea::Post->new;
$p->link('http://www.somelinkwhichdoesnotexist.de');
$p->tags('=F6=E4=FC=DF=D6=C4=DC','testing');
$p->description('=F6=E4=FC=DF=D6=C4=DC');

my $location =3D $c->create($p);

print 'Successfuly created post at ', $location, "\n";


Code doing this with LWP post request:

use LWP::UserAgent;
my $ua =3D LWP::UserAgent->new();

my $url_post=3D'http://username:pas...@ww.../data/add';
my $response =3D $ua->post($url_post,['uri'=3D>'
http://www.somelinkwhichdoesnotexist.de','tags'=3D>'=E4=F6=FC=DF','descript=
ion'=3D>'=E4=F6=FC=DF','comment'=3D>'=E4=F6=FC=DF'],'Referer'
=3D>'');

[Connotea-code-devel] RSS feeds?

[Hilary S. <hil...@st...>]

Hi,

I'm having problems with the RSS feeds in the open source code.  First, 
I noticed that some tags were malformed/empty "<:>".  I fixed this by 
commenting out a line in DBI.pm, although I'm not sure what was causing 
the problem in the first place.

Now, the only problem seems to be that the resulting file is not 
recognized/displayed as an RSS file by Firefox (2.0), so I can't use the 
live bookmarks feature.  This is not a problem on the main Connotea 
site, so I'm hoping someone will be able to help me fix it.

Here's a image of the problem:
  http://cirge.stanford.edu/connoteafiles/rss.html

The top image is what I see when I click the "RSS" link at the top of a 
list of bookmarks. The bottom image is what I should see.  Oddly enough, 
if I use curl to create a local copy of the rss feed (curl -o test.rss 
http://cirge.stanford.edu/bibliotech/rss/user/hilary/tag/aCGH) generated 
by connotea, it is recognized by Firefox to be an rss feed.  This has 
nothing to do with the file extension since "curl -o test http..." also 
generates a recognized rss feed.

I am using Apache's mod_mime (with the default configurations) although 
I don't think this should have any effect.  From my research, I think 
that Firefox guesses if a file is an RSS file or an HTML file by looking 
at the first couple of lines.  If the file begins with <? ?>followed by 
<rdf:RDF> or <rss>, then Firefox should treat the file as an RSS file.  
However, the content of both the file at 
http://cirge.stanford.edu/bibliotech/rss/user/hilary/tag/aCGH and the 
copied file generated by curl are identical.  So theoretically, Firefox 
should interpret them the same way, but doesn't.  I'm confused.

The only difference I can find is that the file header for the 
(correctly recognized) RSS feed says the file type is 
"application/xhtml+xml", while the file header for the unrecognized RSS 
feed says the file type is "application/xml".  Changing this in Const.pm 
so that the RSS feeds have "application/xhtml+xml" as the file type  
doesn't seem to make a difference.

Anyone else having this problem?  Any thoughts?

Please ask if any of the above is not clear.

Thanks so much,
Hilary

Re: [Connotea-code-devel] RSS feeds?

[Martin F. <ma...@ne...>]

Hi Hilary,

Hilary Spencer wrote:
> I'm having problems with the RSS feeds in the open source code.  First, 
> I noticed that some tags were malformed/empty "<:>".  I fixed this by 
> commenting out a line in DBI.pm, although I'm not sure what was causing 
> the problem in the first place.

I think we caught the same bug independently, but can you tell me what 
it was that you changed so I can compare?

> Now, the only problem seems to be that the resulting file is not 
> recognized/displayed as an RSS file by Firefox (2.0), so I can't use the 
> live bookmarks feature.  This is not a problem on the main Connotea 
> site, so I'm hoping someone will be able to help me fix it.
[snip]

We were a bit surprised by Firefox's new behaviour, but as you say, on 
connotea.org it's recognizing the RSS. I can't fathom why your instance 
would behave differently...? Have you tried using the Live Headers 
module for Firefox and compared all the HTTP headers in the exchange?

Martin

Re: [Connotea-code-devel] RSS feeds?

[Hilary S. <hil...@st...>]

Hi Martin,

I managed to fix the problem I mentioned before.  I was also surprised 
to find out that the problem was with the headers, as I had looked at 
them before and didn't see (what I thought were) any significant 
differences.  In addition, the Firefox documentation claims that the 
browser recognizes RSS feeds based on the content of the document, and 
not the headers.

Anyway, according to LiveHeaders, the only difference was that the my 
RSS file had:
"Content-Type: application/xml"
"Content-Encoding: utf-8"

While the RSS file on Connotea.org has:
Content-Type: application/xml; charset=UTF-8"

It seems that the extra header was causing the problem (just changing 
the content type didn't help).  So, I commented out the line in 
Apache.pm which adds the Content-Encoding header (in sub query_handler):
487: #  $r->content_encoding('utf-8');

And in Const.pm, I added the charset to the content-type constant:
30: use constant RSS_MIME_TYPE          => 'application/xml; charset=UTF-8';

And now it works correctly.  Apparently, if the media type given in the 
Content-Type header is 'application/xml', the browser ought to ignore 
the Content-Encoding header and use either the charset specified in the 
content-type header, or in the XML document itself, or default to UTF-8 
(see http://diveintomark.org/archives/2004/02/13/xml-media-types).  But 
either way, the content-encoding header is unnecessary.

In regard to your other question, I commented out the following lines in 
DBI.pm (sub biblio_rdf):
2326: #       $model->addStmt(new RDF::Core::Statement($node,
2327: #                                                new 
RDF::Core::Resource($RDF.'type'),
2328: #                                                new 
RDF::Core::Resource($CONNOTEA.$id->noun)));

Finally, I had problems after updating the cpan module XML::RSS, 
although I'm not sure why.  For some reason, XML::RSS::encode_text (line 
274 in Util.pm) was not a function in the module any more, so I had to 
write my own local function.

Hilary

Martin Flack wrote:
> Hi Hilary,
>
> Hilary Spencer wrote:
>   
>> I'm having problems with the RSS feeds in the open source code.  First, 
>> I noticed that some tags were malformed/empty "<:>".  I fixed this by 
>> commenting out a line in DBI.pm, although I'm not sure what was causing 
>> the problem in the first place.
>>     
>
> I think we caught the same bug independently, but can you tell me what 
> it was that you changed so I can compare?
>
>   
>> Now, the only problem seems to be that the resulting file is not 
>> recognized/displayed as an RSS file by Firefox (2.0), so I can't use the 
>> live bookmarks feature.  This is not a problem on the main Connotea 
>> site, so I'm hoping someone will be able to help me fix it.
>>     
> [snip]
>
> We were a bit surprised by Firefox's new behaviour, but as you say, on 
> connotea.org it's recognizing the RSS. I can't fathom why your instance 
> would behave differently...? Have you tried using the Live Headers 
> module for Firefox and compared all the HTTP headers in the exchange?
>
> Martin
>
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Connotea-code-devel mailing list
> Con...@li...
> https://lists.sourceforge.net/lists/listinfo/connotea-code-devel
>
>   

Re: [Connotea-code-devel] RSS feeds?

[Martin F. <ma...@ne...>]

Ah, yes, this was reported as a bug. The Content-Encoding field is 
apparently only to be used for compression algorithms, not character 
sets. I'm really sorry that did not come to mind and you had to 
rediscover it.

We fixed the biblio_rdf problem by adding another accessor called 
xmlnoun() which is used there.

Sounds like it's time for a new open source release.

I hadn't seen the XML::RSS problem yet but we try not to upgrade the 
CPAN modules very often once things are working.

Martin

Hilary Spencer wrote:
>   Hi Martin,
> 
> I managed to fix the problem I mentioned before.  I was also surprised 
> to find out that the problem was with the headers, as I had looked at 
> them before and didn't see (what I thought were) any significant 
> differences.  In addition, the Firefox documentation claims that the 
> browser recognizes RSS feeds based on the content of the document, and 
> not the headers.
> 
> Anyway, according to LiveHeaders, the only difference was that the my 
> RSS file had:
> "Content-Type: application/xml"
> "Content-Encoding: utf-8"
> 
> While the RSS file on Connotea.org has:
> Content-Type: application/xml; charset=UTF-8"
> 
> It seems that the extra header was causing the problem (just changing 
> the content type didn't help).  So, I commented out the line in 
> Apache.pm which adds the Content-Encoding header (in sub query_handler):
> 487: #  $r->content_encoding('utf-8');
> 
> And in Const.pm, I added the charset to the content-type constant:
> 30: use constant RSS_MIME_TYPE          => 'application/xml; charset=UTF-8';
> 
> And now it works correctly.  Apparently, if the media type given in the 
> Content-Type header is 'application/xml', the browser ought to ignore 
> the Content-Encoding header and use either the charset specified in the 
> content-type header, or in the XML document itself, or default to UTF-8 
> (see http://diveintomark.org/archives/2004/02/13/xml-media-types).  But 
> either way, the content-encoding header is unnecessary.
> 
> In regard to your other question, I commented out the following lines in 
> DBI.pm (sub biblio_rdf):
> 2326: #       $model->addStmt(new RDF::Core::Statement($node,
> 2327: #                                                new 
> RDF::Core::Resource($RDF.'type'),
> 2328: #                                                new 
> RDF::Core::Resource($CONNOTEA.$id->noun)));
> 
> Finally, I had problems after updating the cpan module XML::RSS, 
> although I'm not sure why.  For some reason, XML::RSS::encode_text (line 
> 274 in Util.pm) was not a function in the module any more, so I had to 
> write my own local function.
> 
> Hilary
> 
> Martin Flack wrote:
>> Hi Hilary,
>>
>> Hilary Spencer wrote:
>>   
>>> I'm having problems with the RSS feeds in the open source code.  First, 
>>> I noticed that some tags were malformed/empty "<:>".  I fixed this by 
>>> commenting out a line in DBI.pm, although I'm not sure what was causing 
>>> the problem in the first place.
>>>     
>> I think we caught the same bug independently, but can you tell me what 
>> it was that you changed so I can compare?
>>
>>   
>>> Now, the only problem seems to be that the resulting file is not 
>>> recognized/displayed as an RSS file by Firefox (2.0), so I can't use the 
>>> live bookmarks feature.  This is not a problem on the main Connotea 
>>> site, so I'm hoping someone will be able to help me fix it.
>>>     
>> [snip]
>>
>> We were a bit surprised by Firefox's new behaviour, but as you say, on 
>> connotea.org it's recognizing the RSS. I can't fathom why your instance 
>> would behave differently...? Have you tried using the Live Headers 
>> module for Firefox and compared all the HTTP headers in the exchange?
>>
>> Martin
>>
>>
>> -------------------------------------------------------------------------
>> Take Surveys. Earn Cash. Influence the Future of IT
>> Join SourceForge.net's Techsay panel and you'll get the chance to share your
>> opinions on IT & business topics through brief surveys - and earn cash
>> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV <http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV>
>> _______________________________________________
>> Connotea-code-devel mailing list
>> Con...@li... <mailto:Con...@li...>
>> https://lists.sourceforge.net/lists/listinfo/connotea-code-devel
>>
>>   
>