#266 charset='utf8' ignored--unicode objects not returned

MySQLdb-1.3
open
Andy Dustman
MySQLdb (285)
5
2012-09-19
2008-09-25
No

I can't get MySQLdb to return unicode objects to me using the code on trunk. It always return UTF-8 encoded strs.

MySQLdb from today's trunk.

mysql Ver 14.12 Distrib 5.0.51, for apple-darwin9.2.1 (i686) using
EditLine wrapper

Python 2.5.2

I'm using MacPorts on OS X 10.5, but I can repro this on Linux.

Darwin Macintosh-4.local 9.4.0 Darwin Kernel Version 9.4.0: Mon Jun 9
19:30:53 PDT 2008; root:xnu-1228.5.20~1/RELEASE_I386 i386

Attached is a simple script with a simple query that would return a unicode object in 1.2.2, but now returns an encoded string in 1.3:

Discussion

  • Simple script that shows the problem

     
    Attachments
  • jj, I may hunt this down shortly. Can you provide a working CREATE TABLE and row contents related to your example script, too?

     
  • Andy Dustman
    Andy Dustman
    2009-02-05

    Trunk is just not fully functional yet; it does no type conversion at all.

     
  • Sure. Here's a simplified version that will hopefully work for you:

    CREATE TABLE atoms (
    id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(255),
    INDEX (name)
    ) ENGINE = INNODB;

    INSERT INTO atoms (name) VALUES ('Compaq Evo D310\xc2\xb5 - P4 2.4 GHz');

     
  • Trunk is just not fully functional yet; it does no type conversion at all.

    Ah, that would explain it.

     
  • Andy Dustman
    Andy Dustman
    2009-03-08

    I can't be sure because there are no schema details, but if you had a binary collation on a column, these were being returned as a string instead of unicode (1.2 still does this). The latest trunk should work correctly. I'm marking this as fixed, reopen if you find otherwise. Also see bug #2663436

     
  • Here's the schema:

    CREATE DATABASE development
    DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

    ...

    CREATE TABLE atoms (
    id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,

    active TINYINT NOT NULL DEFAULT 1,
    
    url TEXT NOT NULL,
    name VARCHAR(255),
    blurb VARCHAR(255),
    thumbnail TEXT,
    
    UNIQUE INDEX (url(255)),
    INDEX (name)
    

    ) ENGINE = INNODB;

     
  • Just checked trunk and it works. It was originally broken because, as you said, "Trunk is just not fully functional yet; it does no type conversion at all."

    Thanks!