#51 UTF8 can probably be re-enabled

[Derrick Miller]

The props-mysql-upgrade-0.7-to-0.8.sql script was changed on 27 December 2007 so that it no longer converts the database to UTF8. This was done because strange characters were showing up in stories after conversion.

In upgrading the HM site, we did some more research on the issue and found out the real problem. The problem is that, prior to conversion, many of the stories in the database contained high-bit ASCII characters. This is because editors had pasted in stories directly from the Mac-based Baseview NewsEdit system.

We wrote a script which converts all the high-bit characters (dashes, smartquotes, etc.) to proper low-bit ASCII characters. We were then able to convert the database to UTF8 without any weird characters showing up.

Because this problem is apparently specific to the HM's PROPS installation and is not likely to be encountered by other sites, does it make sense to restore the UTF8 conversion in the default MySQL upgrade script? And also to re-enable UTF8 by default in config.php?