#4 Special characters not correctly presented

[Andreas Nowitzki]

We are using xmlrpc.inc on our website for decoding and encoding server/client requests between 2 server.

We are using version xmlrpc since march 6th 2008 (18:47:24) without changes.

The problem:
German language includes special characters (such as ä, ö, ü, ß). Using our search box and starting a request for one-word-terms with special characters, there are NO search results. In other connections there are terms of special charcters correctly presented, when more than one word is requested and one word is without a special character.

In case we change the procedure in line 231 ($GLOBALS['xmlrpc_internalencoding']='ISO-8859-1';)from 'ISO-8859-1' to 'UTF-8' we get results of single words with special characters, but: in the search result the special characters are replaced by a kind of strange characters such as: A 'ü' stands for “ü” and A 'ö' stands for “ö” etc.

Question: Do you see a chance getting correct characters in the output string?? The content requested and the whole website is using UTF-8.

Please, your answer is very much appreciated at your earliest convenience. Thanks in advance

Andreas Nowitzki, Technical services
P+P Peter Marketing
Waldstraße 15 / D-65329 Hohenstein b. Wiesbaden
Tel.: 06128 / 748 08 77
Fax: 06128 / 748 97 55
eMail Zentrale1: peter.peter@sofind.de
eMail Zentrale2: peter.peter@magicfinding.de
http://www.sofind.de
http://www.magicfinding.de

[Andreas Nowitzki]

[Gaetano Giunta]

[response sent by mail on 2009/04/02]
Hello.

To give useful advice, I think I need a bit more information about your setup:
- the search form posts to a php page that acts as xmlrpc client using the phpxmlrpc library?
- the xmlrpc server is a php page that uses the phpxmlrpc library? where does it get its data from?

The complete chain of requests is, in such a case, the following:

browser ---[post]---> php page acting as client
php page 1 ---[use xmlrpc client to encode data and call]---> php page acting as server
php page 2 ---[use xmlrpc server to decode data]--->
---[fetch response from external source]--->
---[use xmlrpc server to encode data and answer]---> php page acting as client
php page 1 ---[use xmlrpc client to decode data]---> browser

as you can see it is quite long, and there are many encoding/decoding steps involved. Each one is an opportunity for errors ;)

If the website is in utf8, and the external source is in utf8 too, I think the best way to avoid problems is to use utf8 across the whole chain:
- set $GLOBALS['xmlrpc_internalencoding']='UTF-8' on the client
- set $GLOBALS['xmlrpc_internalencoding']='UTF-8' on the server
- set the accept-charset attribute on the "form" element or at least the content-type http header to utf8

This way:
- the browser should send back to the php page its form data in utf8
- the php app will pass utf8 data to the xmlrpc client and the client will know that
- charset handling between xmlrpc client and server works anyway (trust me)
- the xmlrpc server will decode the request received and pass it to the php app as utf8 data
- the php app will do its job and feed back utf8 data to be encoded by the server in an xmlrpc response
- the client, upon receiving it, will decode it to plain php values in utf8 format for the php application
- the php application will then send the utf8 data to the browser for display
- if the content-type header is set to utf8 for the page displaying search results, you should have no problems in displaying them

I hope this helps.

Gaetano

ps: side note: debugging charset problems is generally a bit of a pain, because using printf or var_dump to send debug info to the browsers always incurs in some extra charset encoding/decoding (done by the browser). If in doubt, use a file_put_contents(var_export($var, true)) to see the real bytes that lake up a string at any point in the chain.

[SourceForge Robot]

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 90 days (the time period specified by
the administrator of this Tracker).