#159 Can't change encoding while "Save UTF-8 BOM" option selected

[Wayne Sierke]

With the "Save UTF-8 byte order mark" option enabled, I am unable to change the encoding for an open file. The action stops with the error:
Cannot set encoding: Error at line 1, column 1: Start tag expected, '<' not found

I have:
XML Copy Editor 1.2.0.4
Framework version: wxWidgets 2.8.10
OS: FreeBSD-7.2

[gnschmidt]

I wonder why that's happening. Which encoding are you converting from (presumably UTF-8) and to?

-Gerald

[Wayne Sierke]

Initially it was attempting to switch to ISO-8859-1 but a quick test revealed that it happens regardless of the from/to encoding selections. As far as I can tell any attempt to change encoding fails while the "Save BOM" option is enabled.
It almost seems as though when that option is enabled that it prefixes its internal copy of the document with the BOM regardless of the current encoding setting.
In fact I just attempted to save a pre-existing document which is set to ISO-8859-1, after enabling the "Save BOM" option it refuses to save the document with the error: Cannot save document in ISO-8859-1: Error at line 1, column 1: Start tag expected, '<' not found (saved in default encoding UTF-8)
and indeed the saved file now has the BOM added.
Probably there is a logic error which is causing the "Save BOM" condition to be applied regardless of whether the current encoding is UTF-8 or not, or the current encoding is not detected properly, etc.

[gnschmidt]

Thanks Wayne, that's as precise a bug report as I can hope to have! Will investigate and fix ASAP.
-Gerald

[gnschmidt]

Arrgh I can't reproduce this one either. On my Ubuntu 9.04 setup conversion seems to work fine regardless of the UTF8 BOM setting.

The thing is: the program doesn't store the UTF-8 BOM in memory at all. Whatever happens, the BOM is discarded when the document is opened (and an initial test checks if the file is valid UTF-8 in which case it's moved as a raw chunk of data into the document buffer). It's only when the document is written to disk that the setting makes a difference (and then only if certain conditions are met, e.g. the file isn't a DTD).

-Gerald

[Zane U. Ji]

Here is the patch [b03707] if anybody is interested.

[Zane U. Ji]

Please let me know if I missed anything.