sqlobject-discuss

[SQLObject] Unicode w/ sqlobject

[Markos K. <mk...@gm...>]

OK, so I have a simple object:
class PrimaryDef(SQLObject):
     _connection=sqlobject.connectionForURI("mysql:// 
infoshopkeeper@localhost/lexiko? 
use_unicode=1&charset=utf8&sqlobject_encoding=utf-8")
     class sqlmeta:
         fromDatabase=True
If I try to add an ascii record, it works.
If I try to add a unicode record
	PrimaryDef(lemma=u"λήμμα῾῾)
		or even
	PrimaryDef(lemma=u"λήμμα῾῾).encode("utf-8")
I get the standard
/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/site-packages/sqlobject/dbconnection.pyc in _insertSQL(self,  
table, names, values)
     383         return ("INSERT INTO %s (%s) VALUES (%s)" %
     384                 (table, ', '.join(names),
--> 385                  ', '.join([self.sqlrepr(v) for v in values])))
     386
     387     def transaction(self):

UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position  
1: ordinal not in range(128)

So far so good, I'm still thinking that the connection isn't getting  
my  option keys to know it's supposed to be unicode an then chokes at  
sqlrepr, but if I just do
	PrimaryDef._connection.sqlrepr(u"λήμμα῾῾)
it gives me
	"'\xce\xbb\xce\xae\xce\xbc\xce\xbc\xce\xb1'"

What am I missing here? the table is utf8, the dbencoding is utf8, I'm  
even on OS X, which is utf8.... what gives?
Thanks for any help.
--Marko

Re: [SQLObject] Unicode w/ sqlobject

[Imri G. <lor...@gm...>]

This might be a stupid question, but you didn't include the full source of
PrimaryDef, so I have to ask.
How did you define the column? If you set a StringCol instead of a
UnicodeCol, that might explain it.

Also, noe that your second example,
    "PrimaryDef(lemma=u"λήμμα῾῾).encode("utf-8")"
does the encoding only after trying to add to the table. Maybe you intended
to write
    "PrimaryDef(lemma=u"λήμμα῾῾.encode("utf-8"))"
instead?

Cheers,
Imri



On Sun, Jun 28, 2009 at 4:22 AM, Markos Kapes <mk...@gm...> wrote:

> OK, so I have a simple object:
> class PrimaryDef(SQLObject):
>     _connection=sqlobject.connectionForURI("mysql://
> infoshopkeeper@localhost/lexiko?
> use_unicode=1&charset=utf8&sqlobject_encoding=utf-8")
>     class sqlmeta:
>         fromDatabase=True
> If I try to add an ascii record, it works.
> If I try to add a unicode record
>        PrimaryDef(lemma=u"λήμμα῾῾)
>                or even
>        PrimaryDef(lemma=u"λήμμα῾῾).encode("utf-8")
> I get the standard
> /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/
> python2.5/site-packages/sqlobject/dbconnection.pyc in _insertSQL(self,
> table, names, values)
>     383         return ("INSERT INTO %s (%s) VALUES (%s)" %
>     384                 (table, ', '.join(names),
> --> 385                  ', '.join([self.sqlrepr(v) for v in values])))
>     386
>     387     def transaction(self):
>
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position
> 1: ordinal not in range(128)
>
> So far so good, I'm still thinking that the connection isn't getting
> my  option keys to know it's supposed to be unicode an then chokes at
> sqlrepr, but if I just do
>        PrimaryDef._connection.sqlrepr(u"λήμμα῾῾)
> it gives me
>        "'\xce\xbb\xce\xae\xce\xbc\xce\xbc\xce\xb1'"
>
> What am I missing here? the table is utf8, the dbencoding is utf8, I'm
> even on OS X, which is utf8.... what gives?
> Thanks for any help.
> --Marko
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> sqlobject-discuss mailing list
> sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>



-- 
Imri Goldberg
--------------------------------------
www.algorithm.co.il/blogs/
--------------------------------------
-- insert signature here ----

Re: [SQLObject] Unicode w/ sqlobject

[Markos K. <mk...@gm...>]

Just assumed using the sqlmeta fromDatabase=True formatted columns  
right (they are typed as utf-8 in the database).
Why doesn't this work, I wonder? Should I file it as a bug? UnicodeCol  
works, but the one column is going to be a mighty big text chunk  
sometimes..... Will sqlobject truncate ever?
Thanks,
--Marko

On Jun 28, 2009, at 3:21 AM, Imri Goldberg wrote:

> This might be a stupid question, but you didn't include the full  
> source of PrimaryDef, so I have to ask.
> How did you define the column? If you set a StringCol instead of a  
> UnicodeCol, that might explain it.
>
> Also, noe that your second example,
>     "PrimaryDef(lemma=u"λήμμα῾῾).encode("utf-8")"
> does the encoding only after trying to add to the table. Maybe you  
> intended to write
>     "PrimaryDef(lemma=u"λήμμα῾῾.encode("utf-8"))"
> instead?
>
> Cheers,
> Imri
>
>
>
> On Sun, Jun 28, 2009 at 4:22 AM, Markos Kapes <mk...@gm...>  
> wrote:
> OK, so I have a simple object:
> class PrimaryDef(SQLObject):
>     _connection=sqlobject.connectionForURI("mysql://
> infoshopkeeper@localhost/lexiko?
> use_unicode=1&charset=utf8&sqlobject_encoding=utf-8")
>     class sqlmeta:
>         fromDatabase=True
> If I try to add an ascii record, it works.
> If I try to add a unicode record
>        PrimaryDef(lemma=u"λήμμα῾῾)
>                or even
>        PrimaryDef(lemma=u"λήμμα῾῾).encode("utf-8")
> I get the standard
> /opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/
> python2.5/site-packages/sqlobject/dbconnection.pyc in _insertSQL(self,
> table, names, values)
>     383         return ("INSERT INTO %s (%s) VALUES (%s)" %
>     384                 (table, ', '.join(names),
> --> 385                  ', '.join([self.sqlrepr(v) for v in  
> values])))
>     386
>     387     def transaction(self):
>
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position
> 1: ordinal not in range(128)
>
> So far so good, I'm still thinking that the connection isn't getting
> my  option keys to know it's supposed to be unicode an then chokes at
> sqlrepr, but if I just do
>        PrimaryDef._connection.sqlrepr(u"λήμμα῾῾)
> it gives me
>        "'\xce\xbb\xce\xae\xce\xbc\xce\xbc\xce\xb1'"
>
> What am I missing here? the table is utf8, the dbencoding is utf8, I'm
> even on OS X, which is utf8.... what gives?
> Thanks for any help.
> --Marko
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> sqlobject-discuss mailing list
> sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
>
>
>
> -- 
> Imri Goldberg
> --------------------------------------
> www.algorithm.co.il/blogs/
> --------------------------------------
> -- insert signature here ----

Re: [SQLObject] Unicode w/ sqlobject

[Oleg B. <ph...@ph...>]

On Sun, Jun 28, 2009 at 02:31:53PM -0400, Markos Kapes wrote:
> Just assumed using the sqlmeta fromDatabase=True formatted columns right 
> (they are typed as utf-8 in the database).
> Why doesn't this work, I wonder?

   Because nobody has implemented that feature. Wanna work on it?

> UnicodeCol  
> works, but the one column is going to be a mighty big text chunk  
> sometimes..... Will sqlobject truncate ever?

   SQLObject - no. Database - yes, they could. Depends on the column's
type.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.