Re: [Modeling-users] type('') and unicode

[Sebastien B. <sbi...@us...>]

Yannick Gingras <yan...@sa...> wrote:
> > Okay, so Soif was right, database logging makes it fails (unsurprisingly
> > I must admit).
> >
> > Could you tell what happens if you disable MDL_ENABLE_DATABASE_LOGGING?
> > Does it fail and if yes, where? (traceback as well would be ok)
> > [given that you disable the encoding in your makeMatchQual]
>=20
> Sure !
>=20
[...]
> "/usr/lib/python2.2/site-packages/Modeling/DatabaseAdaptors/AbstractDBAPI=
2AdaptorLayer/AbstractDBAPI2AdaptorChannel.py",
> line 288, in selectAttributes
>     db_info('Evaluating: %s'%statement)
>   File "/usr/lib/python2.2/site-packages/Modeling/logging.py", line 56, in
> log_stderr
>     sys.stderr.write('%s\n'%msg)
> UnicodeError: ASCII encoding error: ordinal not in range(128)
>=20
> It sounds pretty much the same to me...

Okay, sorry, didn't read it right. I suspect that you *enabled* it
there, and that it was disabled before.

  Could you re-try this (without MDL_ENABLE_DATABASE_LOGGING) with:

------------------------------------------------------------------------
--- logging.py  20 Feb 2003 11:48:58 -0000      1.5
+++ logging.py  16 Jul 2003 19:28:49 -0000
@@ -53,6 +53,8 @@
 import os, sys
=20
 def log_stderr(msg):
+  if type(msg) is type(u''):
+    msg=3Dmsg.encode('utf-8')
   sys.stderr.write('%s\n'%msg)
 no_log=3Dlambda msg, severity=3D0: None
 trace=3Ddebug=3Dinfo=3Dlog=3Dwarn=3Derror=3Dfatal=3Dno_log
------------------------------------------------------------------------

The first traceback you gave was referring to db_error(), and it would
be interesting to see what kind of errors this was, wouldn't it ?-)

> > Ok. Sorry I did not understand you right. So this is as simple as that,
> > just encode the attributes' value and that's it? I must try that on of
> > these days.
>=20
> Indeed it does the job but as it was discussed some time ago on the
> mailling list, it does not enable case insensitive match.  A case
> insentivice match with u"=E9=E9" encoded in utf-8 with look for "=C3=A9=
=C3=A9",
> "=E3=A9=C3=A9", "=C3=A9=E3=A9" and "=E3=A9=E3=A9" wich does not make any =
sens once put back in
> unicode.  "=E9=E9", "=C9=E9" and "=C9=C9" are respectivly "=C3=A9=C3=A9",=
 "=C3=C3=A9" and "=C3=C3"
> once encoded.
>=20
> So it may be wise to let the user make the utf-8 trick.  That way he
> won't blame you for the weird result of case insensitive match.  On
> the other hand, some databases like Postgresql detect encoding and
> perform a descent case insitive match with utf-8 data.

This needs investigation. If some of you could provide working python
code with unicode and psycopg/pypgsql/pgdb/mysqldb/sqlitedb, please
share. I've not time for this now.

However, speaking of case-insensitive match: if postgresql supports it,
then it should work, since the SQL WHERE clause behind is UPPER(...)
LIKE UPPER(...) --pure theory and not tested, so if someone feels like
testing it, go ahead :)


-- S=E9bastien.