Re: [dotNetRDF-bugs] Encoding problem with SesameHttpProtocolVersion6Connector and Update(...)

[Rob V. <rv...@do...>]

Matthias

Do you have a specific example query that reproduces the problem?

Looking at the code we already escape code points above 127 into their \u
escaped form so the query texts we send should be entirely within the
Latin 1 range (aka ISO-8859-1), so it's possible that either we are
escaping them incorrectly or Sesame is unescaping incorrectly.  One
possibility is that you are using characters that can't be encoded in a
single UTF-8 byte so they are being encoded as a surrogate pair in UTF-8
(I.e. two bytes) which could be breaking our encoding or Sesame's
decoding. 

If you can provide an example query then I can reproduce the exact
circumstances and investigate further, I have an (old) existing test for
UTF-8 in queries to Sesame which demonstrates that it does work correctly
at least for characters that can be represented as a single UTF-8 byte.
I've added a new test case with some Chinese characters and that functions
correctly for me so having an example that fails for you is necessary for
me to debug further.

Also it would be useful to know what version of Sesame you are
communicating with I.e. what version of the Sesame software is the server
running?

Rob

On 9/12/13 3:23 PM, "Rob Vesse" <rv...@do...> wrote:

>Matthias
>
>Thanks for reporting this, filed as CORE-374 [1]
>
>I'm currently in the midst of relocating countries (and continents) so
>likely won't get to this in the next couple of weeks but will aim to have
>it fixed in early October and released as part of the 1.0.1 release
>
>Thanks,
>
>Rob
>
>[1]: http://dotnetrdf.org/tracker/Issues/IssueDetail.aspx?id=374
>
>On 9/12/13 2:21 AM, "Schwalbe, Matthias (SQAS 1)"
><mat...@cr...> wrote:
>
>>Oops,
>>
>>Found out I should use Copy and Paste with more care:
>>Fixed pastes in original mail further down.
>>
>>Thanks
>>
>>Matthias Schwalbe
>>IT Services
>>+41 44 333 86 10 (*413 8610)
>>
>>From: Schwalbe, Matthias (SQAS 1)
>>Sent: Donnerstag, 12. September 2013 09:47
>>To: 'dot...@li...'
>>Cc: Fichtner, Johann (SQAS 32)
>>Subject: Encoding problem with SesameHttpProtocolVersion6Connector and
>>Update(...)
>>
>>Hi Rob,
>>
>>There is a (simple) problem with SesameHttpProtocolVersion6Connector (and
>>potentially other Connectors)
>>in calls to both the synchronous and asynchronous versions of the
>>Update(...) function.
>>
>>The SPARQL update queries get encoded in url encoded web form format:
>>httpRequest.ContentType = "application/x-www-form-urlencoded";
>>
>>The character set used by:
>>builder.Append(HttpUtility.UrlEncode(this.EscapeQuery(sparqlUpdate)));
>>
>>UrlEncode(...) is utf-8, but the default format for decoding
>>application/x-www-form-urlencoded,
>>according to WEB documentation, is iso8859-1 which leads to garbled
>>literal texts.
>>This happens once text in the query uses characters outside the Latin
>>character range, e.g. Chinese
>>character, accented letters, umlauts.
>>
>>The simplest way to fix this would be to explicitly also state the
>>character encoding used:
>>httpRequest.ContentType = "application/x-www-form-urlencoded;
>>charset=utf-8";
>>
>>This character set it always used by UrlEncode(...)
>>
>>Thanks a lot
>>
>>sincerely yours
>>
>>PS: I used the latest (yesterday) dotNetRdf build from NuGet
>>
>>Matthias Schwalbe
>>CREDIT SUISSE AG
>>CREDIT SUISSE | IT Services, SQAS 1
>>Uetlibergstr. 231 (A/B+ZN) | 8070 Zürich | Switzerland
>>Phone +41 44 333 86 10
>>mat...@cr... | www.credit-suisse.com
>>
>>
>>-------------------------------------------------------------------------
>>-
>>----
>>How ServiceNow helps IT people transform IT departments:
>>1. Consolidate legacy IT systems to a single system of record for IT
>>2. Standardize and globalize service processes across IT
>>3. Implement zero-touch automation to replace manual, redundant tasks
>>http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clkt
>>r
>>k
>>_______________________________________________
>>dotNetRDF-bugs mailing list
>>dot...@li...
>>https://lists.sourceforge.net/lists/listinfo/dotnetrdf-bugs
>
>
>
>
>
>--------------------------------------------------------------------------
>--
>--
>How ServiceNow helps IT people transform IT departments:
>1. Consolidate legacy IT systems to a single system of record for IT
>2. Standardize and globalize service processes across IT
>3. Implement zero-touch automation to replace manual, redundant tasks
>http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktr
>k
>_______________________________________________
>dotNetRDF-bugs mailing list
>dot...@li...
>https://lists.sourceforge.net/lists/listinfo/dotnetrdf-bugs
>