#9 Tab characters in answers lead to a bad parse
Using GetResults.java I would occasionally see: "Error processing HIT results for HIT <hitid>: 25"
I traced this to an ArrayIndexOutOfBounds Exception at line 109 of HitResultProcessor.java in processAnswers()
The real culprit is in HitResults.java HitResults.getAnswers(). It builds a tab-delimited list without first checking if the answer strings already contain the delimiter.
I've patched the problem with the following lines of code. However, I assume that the tab characters are not meaningful and can be removed. A better solution may be to enclose everything in quotes.
In HitResults.getAnswers() at ~line: 203 of HitResults.java:
if (answerValue != null) {
//PATCH - Tabs in Data Bug - Patched 17Dec2008 By kbcarle@gmail.com//
answerValue = answerValue.replaceAll(Character.toString(DELIMITER), " ");
//optional: also replace newline chars
answerValue = answerValue.replaceAll("\n", " ");
//END PATCH
result += answerValue + DELIMITER;
}
I've discovered that when the parameter includeQuestionId RequesterService.getAnswerValue() is true, then it is RequesterService.getAnswerValue() that must strip any pre-existing tab chars from the answer. RequesterService.getAnswerValue() uses the hardcoded value "\t" to separate the questionID from the answer string.
Therefore the above patch is incorrect and will cause corrupted results.
New patch is at line 1125 of RequesterService.getAnswerValue():
if (val.length()==0) {
result += HITResults.EMPTY_ANSWER; // Feature 1816806 (missing columns when value is NULL)
}
else {
//BEGIN PATCH//
//- Tabs in Data Bug - Patched 17Dec2008 By Kris Carle kbcarle@gmail.com //
val = val.replaceAll("\t", " ");
//optional: also replace newline chars
val = val.replaceAll("\n", " ");
//END PATCH//
result += val;
}