Menu

Linux and Windows Charsets

Help
2006-10-09
2012-09-19
  • Cédric Marfil

    Cédric Marfil - 2006-10-09

    Hello,
    I installed mysql-python v1.2.1_p2 on Windows XP to work with Python 2.4 / Zope 2.9.4 and ZMySQLDA 2.0.8. It has to connect with a MySQL 4.1.21 database, encoded in 'latin1'.
    Everything works great on this configuration.
    I even tested to connect directly in python from the method _mysql.connect() and data are transmitted, encoded in 'latin1' I think ('\xe9' for 'é').
    When I test conn.character_set_name(), it returns 'latin1'. It's exactly what I need.

    I installed exactly the same components on a Linux Gentoo 2006.1. I have problems reading special chars like é or è or à ...
    I tested to connect via the _mysql.connect() method, and this time, I get data, encoded in 'utf8' I think ('\xc3\xa9' for 'é').
    When I test conn.character_set_name(), it returns 'utf8' (which is not what I need).

    How could anybody explain this situation ? Aren't distributions exactly the same between OS ?

    I then tested the v1.2.2b1 on Windows, to which a set_character_set() method was added apparently. But when I installed it on Linux Gentoo, the method wasn't available (Attribute Error : set_character_set).

    Has anyone an idea about how the _mysql client choose it's encoding ? Is there any default value to specify somewhere ?

    I saw a post on (http://www.mail-archive.com/sqlobject-discuss@lists.sourceforge.net/msg01416.html) but that didn't mention the bug n° on sf.net, and I didn't manage to find back. Does anyone know to which bug this post refers ?

    Thanks in advance

     
    • Cédric Marfil

      Cédric Marfil - 2006-10-17

      Thanks Andy,
      I didn't manage to emerge 1.2.1_p2 version as I don't have the ebuild for. But I already installed it using a tarball.
      For all to work, I just added the keyword you gave me to the kwargs passed to the connection like this:
      def init(self,connection):
      self.connection=connection
      self.kwargs = kwargs = self._parse_connection_string(connection)
      kwargs['read_default_file'] = '/etc/mysql/my.cnf'
      self.db=apply(self.Database_Connection, (), kwargs)
      self.transactions = self.db.server_capabilities & CLIENT.TRANSACTIONS
      if self._try_transactions == '-':
      self.transactions = 0
      elif not self.transactions and self._try_transactions == '+':
      raise NotSupportedError, "transactions not supported by this server"

      Then I configured my "my.cnf" file [CLIENT] directive like this:
      [client]

      password = your_password

      port = 3306
      socket = /var/run/mysqld/mysqld.sock
      default-character-set=latin1

      Thanks one more time.
      I hope these explainations will help anybody else.

      Cheers

       
    • Andy Dustman

      Andy Dustman - 2006-10-09

      http://packages.gentoo.org/search/?sstring=mysql-python

      1.2.0 is the current stable version.

      1.2.2_p2 is the current unstable (~) version.

      You probably need to add this line to /etc/portage/package.keywords:

      dev-python/mysql-python ~x86

      and re-emerge mysql-python.

       
    • Andy Dustman

      Andy Dustman - 2006-10-09

      Also, your default character set is determined in /etc/mysql/my.cnf, and on Gentoo is utf-8 by default.

       
    • Cédric Marfil

      Cédric Marfil - 2006-10-10

      Hello,
      A lot of my code is relying on v1.2.1_p2 (for BLOB management) which is the stable one on sf. I only tested the 1.2.2b1 to try to resolve my problem, but it didn't.
      I didn't emerge it, but installed it from the tarball available on sf (python setup.py build / install).
      If I emerge it, it will load the 1.2.0 version which is available for gentoo portages, but I need 1.2.1_p2.
      I tryied to change my my.cnf configuration file, adding this line to [client] directive:
      default-character-set = latin1
      But that didn't work.
      To solve my problem, I wrote this line in ZMySQLDA/db.py in the _begin method:
      self.db.query("SET CHARACTER SET latin1")
      But I don't like this patch. It's obviously hardcoded patch and I don't find it very elegant.
      I was looking for a way to add a charset parameter to the ZMySQLDA adapter, then, setting it when creating the _mysql object by a method on it (just like set_character_set in v1.2.2b1).
      Is there any way to do this ?

       
      • Andy Dustman

        Andy Dustman - 2006-10-10

        Like I said, add this to /etc/portage/package.keywords:

        dev-python/mysql-python ~x86

        Then you can emerge the 1.2.1_p2 version. If you want the beta version as a Gentoo ebuild, make your own overlay, or as a quick fix, copy /usr/portage/dev-python/mysql-python/mysql-python-1.2.1_p2.ebuild to mysql-python-1.2.2_b1.ebuild. See man ebuild to see how to make a digest for it. Note that emerge --sync will wipe this out so you should do this in your own overlay; see man make.conf and PORTDIR_OVERLAY for how to do this.

        Try doing this to ZMySQLDA: Where it is doing the connect() call, add this keyword parameter:

        read_default_file="/etc/mysql/my.cnf"

        or some other configuration file you want to use.

         

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.