bad display of UTF-8 encoded files with "illegal" chars
Windows visual diff and merge for files and directories
Brought to you by:
christianlist,
grimmdp
Menu
▾
▴
#2126 bad display of UTF-8 encoded files with "illegal" chars
i had a file with a single 0xB0 in it.
this char resolved to whatever might be but not to what it should.
as UTF-8 recommends and as it is commonly implement such a char should resolve to the illegal char marker U+FFFD.
A deeper check showed that indeed there is much fuzz that does not resolve as it should when using the test file from this location:
http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
when using e.g. programmers notepad then its quite visible to me that the right edge line end markers are all one above the other. this is not the case for WinMerge and several other deviations do appear including the lack of use of the illegal char marker.
PS: in all test cases my install of WinMerge 2.12.4.0 Unicode showed UTF-8 on its bottom line
oops, the PN2 (programmers notepad test) was done with copy&paste from the text beeing open in the firefox (waterfox for windows 64 bits v16.0.1) browser. so it might be the clipboard and the browser were the more important part of the conversion. PN2 alone does not automatically do unicode but simply selects ascii on the file.