On 08/29/2013 12:38 AM, Jim Baker wrote:



On Wed, Aug 28, 2013 at 3:53 PM, Fernando Martins <fernando@cmartins.nl> wrote:
....

I'm using jython 2.5 and this works:

x = "José"
toClient.println("<li>%s</li>" % x)

In my example, row[0] is of type PyUnicode and the db uses encoding utf8. So, some part of the pipelining is changing the encoding. After a search, I got to

http://old.nabble.com/Re%3A-On-String,-PyString-and-PyUnicode-p18767753.html

where you state that PyUnicode uses utf-16. I can't test now, but I guess .encode('utf-8') should do the trick, right? And I don't suppose there is a way to avoid this extra conversion?

Correct, this conversion is really required. Sometimes you may be working with input stream/output streams that are character-oriented, but not in this case. But it's just a simple matter of .encode("UTF-8")


For the record, I have it working without .encode("UTF-8"). I think the issue was that the encoding of the response has to be set before getting the writer. I think I got mislead by the browser initially reporting the page as UTF-8, incorrectly because the servlet docs says the default is ISO-8859-1. At this stage I had not explicitly set the encoding and latter on I set it after creating the writer (incorrectly). Note also that creating a string with

x = "josé"

will not work even if the .py file states encoding utf8!! it needs

x = unicode("josé", "UTF-8")

However, it works well (no call to encode) from a text file including UTF-8 characters!!

For the record, complete test code:

# -*- coding: utf-8 -*-
from javax.servlet.http import HttpServlet
from com.ziclix.python.sql import zxJDBC
from java.lang import System
import sys

class JythonServlet1 (HttpServlet):
    def doGet(self,request,response):
        self.doPost (request,response)

    def doPost(self,request,response):
        response.setContentType("text/html; charset=UTF-8")
        # encoding must be set before getWriter
        toClient = response.getWriter()
        d = "jdbc:mysql://localhost/database"
        u, p, v = 'xxxx', 'xxxx', 'com.mysql.jdbc.Driver'
        db = zxJDBC.connect(d, u, p, v)
        cursor = db.cursor()
        cursor.execute('SELECT * FROM TABLE')
        html = unicode("<html><head><title>Servlet Test</title>" +\
          "<body><h1>José</h1><ul>", "UTF-8")
        toClient.println(html)
        for row in cursor.fetchall():
            toClient.println("<li>%s</li>" % row[1])
       toClient.println("</ul></body></html>")
...

This also doesn't count some new functionality that we will be delivering such as ssl support; for what it's worth, I recently pushed up a branch of an experiment on this that allows me to use easy_install (https://bitbucket.org/jimbaker/jython-ssl); pip still fails because of some Unicode issue and the lack of os.O_NOFOLLOW, something we could readily fix if we restrict 2.7 to Java 7 (as I've started to advocate on #jython, given the recent end-of-life for Java 6 for users without Oracle support contracts).
 

I don't know yet if I'm going to use jython for sure but it is likely as I'll have to port a python web site to the official java platform. Jython might give me a cheap way to "port" from python to java ;)

In that case, it will be java 6 and I have no idea if the systems will be upgraded to 7 any time soon. But I don't think supporting older platforms is a good criteria to delay progress of Jython unless it pays your bread and butter :)

Personally I am very glad to have a shot at using python on java and very grateful for your work!!

Cheers,
Fernando