#258 Invalid Numeric Values Default to Zero Without Captured Exceptions

[Adrian Haenni]

There is still unexpected behavior related to the following issues in OpenCSV:

• Bug #253
• Bug #242

I encountered this issue after upgrading to OpenCSV 5.11, and it appears to be caused by the default behavior of BeanUtils.

Example Setup

Assume the following CSV Java class:

public class Csv {

    @CsvBindByName(column = "ColumnA")
    private String colA;

    @CsvBindByName(column = "ColumnB")
    private Integer colB;

    @CsvBindByName(column = "ColumnC")
    private String colC;

    // getters/setters...
}

And this CSV file:

ColumnA;ColumnB;ColumnC
a;1;xyz
b;invalidDataType;zzz

Now consider the following code snippet to parse the CSV:

HeaderColumnNameMappingStrategy<T> mapper = new HeaderColumnNameMappingStrategy<>();
mapper.setType(getCsvType()); // => Csv.class

CsvToBean<T> csvToBean = new CsvToBeanBuilder<T>(reader)
        .withMappingStrategy(mapper)
        .withSeparator(';')
        .withThrowExceptions(false)
        .build();

List<T> result = csvToBean.parse();
importResult.logParsingExceptions(csvToBean.getCapturedExceptions());

Behavior Before OpenCSV 5.11

Prior to version 5.11, this setup behaved as expected:

• The first row was parsed correctly.
• The second row failed to parse due to the invalid integer format ("invalidDataType"), and an exception was captured.

Result:
1. ColumnA = a, ColumnB = 1, ColumnC = xyz
2. - (skipped)
Captured exceptions: 1 (org.apache.commons.beanutils.ConversionException for "invalidDataType")

Behavior in OpenCSV 5.11
With version 5.11, parsing still returns correct data for the first row. However, the second row is not rejected. Instead:

• ColumnB is silently set to 0
• No exception is recorded in getCapturedExceptions()

Result:
1. ColumnA = a, ColumnB = 1, ColumnC = xyz
2. ColumnA = b, ColumnB = 0, ColumnC = zzz
Captured exceptions: none

Root Cause
The issue stems from the following internal behavior in OpenCSV:

• When a ConverterPrimitiveTypes instance is created without a locale, it calls:
• this.readConverter = BeanUtilsBean.getInstance().getConvertUtils();
• This leads to instantiating ConvertUtilsBean, which calls deregister() in its constructor.
• Inside deregister(), the method registerPrimitives(false) is invoked.
• The false flag means exceptions are not thrown for errors ("throwException"). Instead:
• register(Integer.TYPE, throwException ? new IntegerConverter() : new IntegerConverter(ZERO));
• Thus, if an invalid string is encountered (e.g. "invalidDataType"), NumberFormatException is silently swallowed, and the default value 0 is used.

Workarounds
1. Set Locale explicitly - You could define a locale attribute in each @CsvBindByName annotation. However, this is impractical for applications with many CSV definitions and fields.
2. Register a custom converter - Another option is to register a custom integer converter:

BeanUtilsBean beanUtilsBean = BeanUtilsBean.getInstance();
ConvertUtilsBean convertUtils = beanUtilsBean.getConvertUtils();
convertUtils.register(new MyIntegerConverter(), Integer.class);

However, this affects global application behavior and may have unintended side effects.

A robust fix would require OpenCSV to respect the .withThrowExceptions(false) flag more explicitly and avoid defaulting silently to zero unless explicitly configured to do so.

[Scott Conway]

Good catch. I think you have given me enough to be able to create a unit test to replicate the issue and see if I can find a fix.

The issue I have with using the workarounds internally is the change I made was an attempt to only call the register if and only if there was not a converter register for that type already. That was the issue - another user using opencsv within an application where converters were already set were found that they were being unset. My original thought was to just check for null - I did not dig deep enough into the apache beanutils to see that the original creation of the converter was already creating a set of defaults and thus my null check was always going to return false and the correct register was not going to be run. So I really need to find a way of querying the BeanUtilsBean to see if a value was registered or if it is the default that was created on construction. If it is the later I want to override it. but if not I want to leave what is there.

[Scott Conway]

Short term I have rolled back the changes I made and reopened bug #253. I will keep this open though until I can find a way of getting the converter utils without it deregistering existing converters.