Looks like diffuse does have some issues with encoding detection. The algorithm it uses is ether pretty dumb (looking at the bug in the Feature request tracker this seem to be the case) or pretty much broken.
The problem I have is that it doesn't correctly detect UTF-16 files (the file content looks like this A@B@C@D@..., ie the 00 hex sequence is replaced by @). If I add uft_16_le as the first entry to the regional settings (same as done for utf-8 in the other bug), then uft-16 files open correctly, but the problem is that doing this causes other (non utf-16) files to open as garbage randomly (some open correctly, some open as one long line of random chars, mostly chinese).
One would thing that detecting utf-16 would be the simplest thing, because the file always starts with a BOM (FFFE in the case of utf-16le), so there is no need to guess.
Log in to post a comment.