Re: [PyWrapper-devel] Unicode

[Roger H. <ro...@td...>]

I think MySQL unicode support if pretty perfect now (version 5) you =20
seem to be able to specify encodings right down to column level and =20
do just about anything you might want to - if you can work out what =20
it is you need to do that is ;).

I believe the big changes came in 4.1

http://dev.mysql.com/tech-resources/articles/4.1/unicode.html

It comes down to doing some detective work. Tracing right through the =20=

data path from the original thing you imported into the db.

Good luck,

Roger



On 15 Jun 2007, at 10:20, D=F6ring, Markus wrote:

> well,
> im not sure about the very latest mysql version, but mysql is very =20
> lazy when it comes to character encodings and to my knowledge it =20
> *doesnt* support unicode at all. What most people do is encode =20
> their data as utf8 and dump it in mysql as latin1. that causes some =20=

> confusion sometimes. for pywrapper I think the utf8 setting would =20
> be correct in this case. have you tried different settings? what =20
> happens? and are you sure the =C3 is not inside your db?
>
> pywrapper has only one setting for character encodings, yes, the =20
> one you pointed to.
> for serious debugging you could try to connect to the db via the =20
> terminal using python interactively. take a look at the dbmod =20
> dmod_mysql.py inside lib/biocase/pywrapper/dbmod (or sth like this) =20=

> how to do a connection. just retype the commands in the python =20
> shell and see what you get.
>
> markus
>
>
> -----Original Message-----
> From:	pyw...@li... on behalf of =20
> Shunde Zhang
> Sent:	Thu 6/14/2007 2:42 AM
> To:	PyWrapper Developers mailing list
> Cc:=09
> Subject:	Re: [PyWrapper-devel] Unicode
>
> Hi Markus,
>
> The raw XML from your server has the correct value, eg. Punb=E4cken. =20=

> However, in
> my raw XML, this character is shown as "=C3=A4", which is not correct.
> Could you tell me how you configured it in detail? Did you just set =20=

> it in:
>
> Home  =BB Configtool  =BB <datasource>  =BB Connection  >> Encoding
>
> Is there any other place I need to look at?
>
> I am using mysql and when I created the database, I didn't set any =20
> character
> set. so it is using the default one, utf-8. since if I dump the =20
> database, I can
> see:
>
> /*!40101 SET NAMES utf8 */;
>
> I think this means my database is using utf8. also I set utf8 in =20
> pywrapper. but
> it doesn't work. is it possible to debug it? thanks a lot.
>
> Cheers,
> Shunde.
>
>
> Quoting Markus D=F6ring <m.d...@bg...>:
>
>> shunde,
>> pywrapper definitely supports unicode.
>> you can configure the character encoding for each datasource (the
>> character encoding used in the database, output should always be
>> utf8) and it works fine for the training dataset where there are at
>> least latin1 characters. see here:
>>
>> http://search.biocase.org/tapir/pywrapper?dsa=3Dtraining&op=3Di&c=3Dhtt=
p://
>> rs.tdwg.org/dwc/dwcore/Locality&limit=3D100
>>
>> --
>> Markus
>>
>>
>>
>> On 13.06.2007, at 07:08, Shunde Zhang wrote:
>>
>>> Hi all,
>>>
>>> Does pywrapper support unicode? I've done a test. In database, my
>>> value is
>>> "Universit=E4t". and in the XML returned from pywrapper it becomes
>>> "Universit=C3=A4t". It seems not correct. Could anybody tell
>>> me how to
>>> configure pywrapper to support Unicode? thanks in advance.
>>>
>>> Cheers,
>>> Shunde.
>>>
>>>
>>>
>>> --------------------------------------------------------------------=20=

>>> --
>>> ---
>>> This SF.net email is sponsored by DB2 Express
>>> Download DB2 Express C - the FREE version of DB2 express and take
>>> control of your XML. No limits. Just data. Click to get it now.
>>> http://sourceforge.net/powerbar/db2/
>>> _______________________________________________
>>> PyWrapper-devel mailing list
>>> PyW...@li...
>>> https://lists.sourceforge.net/lists/listinfo/pywrapper-devel
>>
>>
>> ---------------------------------------------------------------------=20=

>> ----
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> PyWrapper-devel mailing list
>> PyW...@li...
>> https://lists.sourceforge.net/lists/listinfo/pywrapper-devel
>>
>
> ----------------------------------------------------------------------=20=

> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> PyWrapper-devel mailing list
> PyW...@li...
> https://lists.sourceforge.net/lists/listinfo/pywrapper-devel
>
>
>
>
> ----------------------------------------------------------------------=20=

> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> PyWrapper-devel mailing list
> PyW...@li...
> https://lists.sourceforge.net/lists/listinfo/pywrapper-devel