Reading UTF-8 with and without BOM

One important feature of JDecisiontable is that it stores you decision tables as text rather than as binaries. This enables 3rd party developers to create or alter decision tables with tools they're familiar with. The format is JSON a very common format so you may read your decision tables as objects in many programming languages, i.e. Javascript. The encoding is UTF-8 without BOM for reading and writing.

A BOM (= Byte Order Mark) is some extra information at the beginning a UTF-x encoded string. You can see it as bytes EF BB BF before very first char of text in a hex editor.

Learning more about UTF-8 and encodings by enhancing the underlying library MyFlipFlops (MyFlipFlops version 1.0007 released) raised the question what would happen if a decision table got a BOM?

To find out a test were prepared. The test uses Windows® “Editor” editor (notepad.exe) to open a *.5dt file containing characters “ABCxyz123ÄÖÜäöüß@€|µ~¹²³¼½¬{[]}\” and saved as text file with encoding “UTF-8”. “Editor” added a BOM visible as byte EF BB BF at the beginning of the file in a hex editor (Gnome ghex).

JDecisiontable version 1.1.1 (as jar-with-dependencies using MyFlipFlops version 1.0006 as delivered) could successfully open the prepared file so that is test along with a second test using the original file to run automatically each build.

It's very useful to define "test" as "Activity to compare an expected result with the actual result" avoiding definitions related to "finding bugs" because you were unable to find any bugs because JDecisiontable version 1.1.1 was never expected to read UTF-8 with BOM. Thus were would be no bug if the test would fail because it was never expected to do this. This would be a feature-request leading maybe to a change.

By the way:
1. UTF-8 stores also chinese, japanese and korean characters see RFC 3629.
2. The encoding named “ANSI” by Windows® “Editor” is known as “windows-1252” according to IANA.