#263 Strip UTF-8/16 BOM

[SuperPat]

OpenCSV does not strip the UTF-8/16 with BOM csv files

[Scott Conway]

Sorry this must be a duplicate or something because I swore I responded to this already.

I am rejecting this because the purpose of opencsv is to parse csv data. Not csv in utf form or ascii or ebcdic or whatever character formats. It is the duty of the developer using opencsv to handle the vulgarities of whatever file/stream/character types they are using.

For UTF characters you can use the apache-commons commons-io BOMInputStream. If you check out the latest code there is an example in BomHandlingTest.

    @Test
    @DisplayName("Show how to handle a utf file with a bom character.")
    public void testBomHandling() throws IOException {
        BOMInputStream b = BOMInputStream.builder()
                .setPath(Paths.get(UTF_FILE_NAME))
                .setByteOrderMarks(ByteOrderMark.UTF_8)
                .setInclude(false)
                .get();

        InputStreamReader ff = new InputStreamReader(b);

        List<Job> jobs = new CsvToBeanBuilder(ff)
                .withType(Job.class).build().parse();

        assertEquals(40, jobs.size());
    }