encoding problems

Help
s_sbaiz
2005-09-29
2013-06-04
  • s_sbaiz
    s_sbaiz
    2005-09-29

    My MailManager instance crashes getting mails by POP3 very often.
    Errors traceback are usually like these:

    1)
    Module ZPublisher.Publish, line 100, in publish
    Module ZPublisher.mapply, line 88, in mapply
    Module ZPublisher.Publish, line 40, in call_object
    Module Products.MailManager.MailManager, line 639, in getMail
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 421, in getMail
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 450, in _getPOP3
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 155, in process
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 173, in process_addMsgToTicket
    Module Products.MailManager.MailManager, line 792, in addMessageToTicket
    Module Shared.DC.ZRDB.DA, line 436, in __call__
    Module DocumentTemplate.DT_String, line 474, in __call__
    Module Shared.DC.ZRDB.sqlvar, line 135, in render
    Module Products.ZMySQLDA.DA, line 131, in sql_quote__
    Module Products.ZMySQLDA.db, line 317, in string_literal
    UnicodeEncodeError: 'ascii' codec can't encode character u'\u2019' in position 145: ordinal not in range(128)

    2)
    Module ZPublisher.Publish, line 100, in publish
    Module ZPublisher.mapply, line 88, in mapply
    Module ZPublisher.Publish, line 40, in call_object
    Module Products.MailManager.MailManager, line 639, in getMail
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 421, in getMail
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 450, in _getPOP3
    Module /var/lib/zope_2.7.0b/Dati/Products/MailManager/Extensions/AccountPluggableBrain.py, line 134, in process
    Module None, line 337, in getHeader
    Module None, line 345, in _decodeHeader
    AttributeError: 'NoneType' object has no attribute 'Header'

    Can both of them be caused by encoding problems?

    Is it necessary for my mysql database tables to be in UTF-8 format?

    Regards,
    Simone

     
    • Kevin Campbell
      Kevin Campbell
      2005-09-30

      Simone,

      The first problem of the two is likely due to lack of UTF-8 encoding, as we are using unicode through MailManager now. The second problem may be unrelated, and seems rather bizarre. It appears to be due to the email package being broken? Can you try the following in your python interpreter?

      import email.Header
      print email.Header

      let me know how you get on.

      Regards,
      Kevin

       
    • s_sbaiz
      s_sbaiz
      2005-09-30

      This is my result:
      >>> import email.Header
      >>> print email.Header
      <module 'email.Header' from '/usr/local/lib/python2.3/email/Header.pyc'>
      >>>

      But I think to have solved Header error:
      I've changed line: 
      except email.Header.HeaderParseError:
      with line:
      except:
      in MailMixin.py script.
      Can it cause further errors?

      I think main encoding problems are in MailManager.py calling sql.addMessage in the variable  html_body=html_body.
      How can I solve these problems?
      This is critical for me: tickets can be added!

      What should I do?

      Regards,
      Simone

       
    • Kevin Campbell
      Kevin Campbell
      2005-10-04

      Simone,

      That exception is far too general, and I would advise against that modification. I presume the problem is due to some code replacing the email module in the global namespace, but I will need to investigate this further.

      As for the encoding problem, the issue is due to the ZMySQLDA adapter trying to convert the unicode string supplied by the MailManager code to a standard ascii encoding. It should not be doing this, and it should be storing unicode in the database. Unfortunately, I don't have a system with the ZMySQLDA adapter installed here at present, but I'll try and see if we can recreate this here through our test suite by tomorrow.

      Regards,
      Kevin