I want to use utf8 encoding. I have read the previous post 'Unicode Issues creating databases'.
I am trying to set the connection encoding to 'utf8' and tried the following:
1) I have set the default encoding of the Mysql server to 'utf8'. The database and tables were created with default encoding 'utf8'.
2) I am using a local my.cnf with 'default-character-set=utf8, when creating the connection.
3) I have set the 'character_set_connection' utf8 using the MySQLdb cursor.
Still, every time I try to execute something like:
cursor.execute('insert into table (field) values (%s)', (u'\u1508'))
I get the error:
exceptions.UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-3: ordinal not in range(256)
I am using the following system:
Mysql 4.1.12 on win XP pro.
MySQLdb 1.37
Python 2.4.1
Am I doing something wrong? Is there something else I can try?
Thanks
Amit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
While setting default-character-set = utf in ~/.my.cnf should work (but remember you need to tell connect() to read it with the read_default_file option), in 1.2.1c7 and 1.2.1 (out tomorrow), there is also a charset option to connect.
The example shown by dobesv won't work in general, because the internal converters for columns need to be reset so they are decoding the right character encoding. Just changing conn.charset doesn't do this. The same applies for trying to change it with SQL: MySQLdb doesn't know you've done this, so it still tries to decode stuff with the default encoding.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
I want to use utf8 encoding. I have read the previous post 'Unicode Issues creating databases'.
I am trying to set the connection encoding to 'utf8' and tried the following:
1) I have set the default encoding of the Mysql server to 'utf8'. The database and tables were created with default encoding 'utf8'.
2) I am using a local my.cnf with 'default-character-set=utf8, when creating the connection.
3) I have set the 'character_set_connection' utf8 using the MySQLdb cursor.
Still, every time I try to execute something like:
cursor.execute('insert into table (field) values (%s)', (u'\u1508'))
I get the error:
exceptions.UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-3: ordinal not in range(256)
I am using the following system:
Mysql 4.1.12 on win XP pro.
MySQLdb 1.37
Python 2.4.1
Am I doing something wrong? Is there something else I can try?
Thanks
Amit
WIth my MySQLdb I set the connection encoding like this:
conn = MySQLdb.connect(ip,user,pass,db, use_unicode=1)
conn.charset = 'Cp1252' # Use Cp1252
Maybe for you, you could set 'utf-8', or at the very least, make sure you're passing use_unicode=1.
While setting default-character-set = utf in ~/.my.cnf should work (but remember you need to tell connect() to read it with the read_default_file option), in 1.2.1c7 and 1.2.1 (out tomorrow), there is also a charset option to connect.
The example shown by dobesv won't work in general, because the internal converters for columns need to be reset so they are decoding the right character encoding. Just changing conn.charset doesn't do this. The same applies for trying to change it with SQL: MySQLdb doesn't know you've done this, so it still tries to decode stuff with the default encoding.