The problem is that it doesn't work when v is a Unicode string that contains characters which can't be converted to iso8859-1/latin-1. This is true even though I gave unicode=True in the .connect() call.
In order to make the above work, I had to use v.encode("iso8859-1", "replace"), which isn't acceptable - the Unicode characters need to be kept.
There must be something stupid I'm missing here, but from everything I've read this should Just Work.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
''The problem is that it doesn't work when v is a Unicode string that contains characters which can't be converted to iso8859-1/latin-1. This is true even though I gave unicode=True in the .connect() call.''
to my understanding unicode param is used to use a particular encoding while storing/loading data from mysql, i use that param as unicode='latin-1' and it works fine, just look at Connection docstring:
''unicode -- If set to a string, character columns are returned as unicode objects with this encoding. If set to None, the default encoding is used. If not set at all, character columns are returned as normal strings.''
sometimes i too receive UTF-8 strings that are cannot be translated to latin-1 strings, but i think this is really a mysql limitation: up to version 4.0.x mysql can't cope with UTF-8 strings. upcoming 4.1 should be able to manage also UTF-8 strings and that would do the trick.
so, if you want latin-1 in mysql you have to discard those unicode characters which cannot be mapped to latin-1.
HTH.
cheers,
deelan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Here is the .execute() statement I am trying to use:
c.execute("""INSERT INTO attrs (fileid, name, value) VALUES (%s, %s, %s)""", (id, k, v))
The problem is that it doesn't work when v is a Unicode string that contains characters which can't be converted to iso8859-1/latin-1. This is true even though I gave unicode=True in the .connect() call.
In order to make the above work, I had to use v.encode("iso8859-1", "replace"), which isn't acceptable - the Unicode characters need to be kept.
There must be something stupid I'm missing here, but from everything I've read this should Just Work.
''The problem is that it doesn't work when v is a Unicode string that contains characters which can't be converted to iso8859-1/latin-1. This is true even though I gave unicode=True in the .connect() call.''
to my understanding unicode param is used to use a particular encoding while storing/loading data from mysql, i use that param as unicode='latin-1' and it works fine, just look at Connection docstring:
''unicode -- If set to a string, character columns are returned as unicode objects with this encoding. If set to None, the default encoding is used. If not set at all, character columns are returned as normal strings.''
sometimes i too receive UTF-8 strings that are cannot be translated to latin-1 strings, but i think this is really a mysql limitation: up to version 4.0.x mysql can't cope with UTF-8 strings. upcoming 4.1 should be able to manage also UTF-8 strings and that would do the trick.
so, if you want latin-1 in mysql you have to discard those unicode characters which cannot be mapped to latin-1.
HTH.
cheers,
deelan