Okay, I released a new version of Data Transfer (0.3.3) that has the discussed dropdown - the file gets UTF-8-encoded only if it's not using UTF-8 already. Anyone who was having problems with this before can upgrade to the latest version, and let me know if that improves things.


On Wed, Jul 22, 2009 at 3:21 AM, Patrick Nagel <mail@patrick-nagel.net> wrote:
Hi Yaron,

On 2009-07-22 12:34, Yaron Koren wrote:
> I'm looking now into creating this dropdown in the 'ImportCSV' special
> page (that would let the user specify what encoding the file is in).
> Does anyone have an opinion on what a reasonable set of values would be?
> Are UTF-8 and UTF-16 enough, or should it also allow UTF-32 and other,
> more obscure (I think) encodings?

I think UTF-8 (= no conversion), UTF-16LE (= from Windows) and UTF-16BE (= from
Mac, AFAIK, not 100% sure) should suffice. Adding locally used (IMO obsolete)
encodings like GBK / GB2312 for Chinese, for example, would be more work for
you, and would only help a few very lazy users who still don't use Unicode,
while they should. ;)


Key ID: 0x86E346D4            http://patrick-nagel.net/key.asc
Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4