#156 Multi-line property parsing fails - fix provided

[Anonymous]

Greetings.
I've had a problem parsing calendars that have multi-line properties like:

DESCRIPTION:This is a lo
ng description
that exists on a long line.

The parsing logic seems to expect each property to occupy exactly one line - which is not a correct assumption. Multi-line properties transport have to be supported (see iCal RFC section 4.1 content lines - http://www.ietf.org/rfc/rfc2445.txt ).

My fix for iCal4j 1.0.4 (added check for newline and subsequent space, if this is the case, newline and space are skipper and property parsing continues):
net.fortuna.ical4j.data.CalendarParserImpl.PropertyParser.parse, line 228 and onward:
boolean continueParsing = true;
while (continueParsing) {
while (nextToken != StreamTokenizer.TT_EOL && nextToken != StreamTokenizer.TT_EOF) {

if (tokeniser.ttype == StreamTokenizer.TT_WORD) {
value.append(tokeniser.sval);
} else {
value.append((char) tokeniser.ttype);
}

nextToken = tokeniser.nextToken();
}
continueParsing = false;
if (nextToken == StreamTokenizer.TT_EOL) {
int afterNextToken = tokeniser.nextToken();
if (afterNextToken != StreamTokenizer.TT_EOF && afterNextToken != StreamTokenizer.TT_EOL && tokeniser.sval.startsWith(" ")) {
nextToken = afterNextToken;
tokeniser.sval = tokeniser.sval.substring(1);
continueParsing = true;
} else {
tokeniser.pushBack();
}
}
}

[Anonymous]

P.S. Since it's not visible in comment, the actual content of the property is this:
DESCRIPTION:This is a lo
[SPACE]ng description
[SPACE]that exists on a long line.

Where [SPACE] is space symbol.
This is in accordance with WebDAV RFC (link provided above), so has to be fixed in iCal4J parser.

[Anonymous]

P.P.S.
> This is in accordance with WebDAV RFC
I meant iCal RFC, sorry.

[Ben Fortuna]

Hi,

Apologies for the delayed response. The logic for handling folded (multiline) properties is actually provided in the UnfoldingReader class. So when you use the parser you should wrap your input in an UnfoldingReader to handle multi-line properties. You can see an example of this in CalendarBuilder.build(Reader).

regards,
ben

[Anonymous]

"when you use the parser"
I don't!

iCal4j Connector calls the parser, so the connector should be fixed then I suppose.

[Anonymous]

P.S. Actually I see that iCal4j Connector uses UnfoldingReader. I suppose somehow it doesn't work correctly then. I'll check in details later.

[Anonymous]

P.P.S. It all seems to be not working - I've tried an UnfoldingReader and it's not unfolding anything.
I see that pattern "\n " is considered to be "relaxed" somehow, but I suspect there's not way to tell Connector to create UnfoldingReader-s with relaxed=true (it is a constructor parameter for UnfoldingReader).
Moreover, when I manually create UnfoldingReader with relaxed = true, it fails with buffer overflow, so I have to set another constructor parameter to make it have bigger buffer - and by bigger I mean bigger than size of the Calendar property it's trying to read.
And even then it doesn't unfold anything!

Here's the sample code I used.
public static void main(String[] args) throws Exception {

String str = "DESCRIPTION:\\n\\n\\nDeelnemers testsѐëöäüї!@#$^ Deelnemers Deelnemers Deelneme\n"
+ " rs Deelnemers Deelnemers Deelnemers Deelnemers Deelnemers Deelnemers Deelnem\n"
+ " ers Deelnemers Deelnemers Deelnemers Deelnemers v v Deelnemers Deelnemers De\n"
+ " elnemers Deelnemers Deelnemers v Deelnemers Deelnemers Deelnemers \\n\nBlah\n";
// TODO Auto-generated method stub
UnfoldingReader reader = new UnfoldingReader(new StringReader(str), 65535, true);
char[] buffer = new char[65535];
reader.read(buffer);
System.out.println(new String(buffer));
reader.close();
}

The result outputs same string as input!

[Ben Fortuna]

Ok, first the specification states that a line is folded by the string "\r\n ", whereas you are using "\n ". It is fine to use that but you must enable relaxed unfolding using either CompatibilityHints or an ical4j.properties file. See here for more details:

http://wiki.modularity.net.au/ical4j/index.php?title=Compatibility

regards,
ben