#156 Multi-line property parsing fails - fix provided

Release
open-works-for-me
Parsing (53)
5
2013-01-14
2012-12-27
Anonymous
No

Greetings.
I've had a problem parsing calendars that have multi-line properties like:

DESCRIPTION:This is a lo
ng description
that exists on a long line.

The parsing logic seems to expect each property to occupy exactly one line - which is not a correct assumption. Multi-line properties transport have to be supported (see iCal RFC section 4.1 content lines - http://www.ietf.org/rfc/rfc2445.txt ).

My fix for iCal4j 1.0.4 (added check for newline and subsequent space, if this is the case, newline and space are skipper and property parsing continues):
net.fortuna.ical4j.data.CalendarParserImpl.PropertyParser.parse, line 228 and onward:
boolean continueParsing = true;
while (continueParsing) {
while (nextToken != StreamTokenizer.TT_EOL && nextToken != StreamTokenizer.TT_EOF) {

if (tokeniser.ttype == StreamTokenizer.TT_WORD) {
value.append(tokeniser.sval);
} else {
value.append((char) tokeniser.ttype);
}

nextToken = tokeniser.nextToken();
}
continueParsing = false;
if (nextToken == StreamTokenizer.TT_EOL) {
int afterNextToken = tokeniser.nextToken();
if (afterNextToken != StreamTokenizer.TT_EOF && afterNextToken != StreamTokenizer.TT_EOL && tokeniser.sval.startsWith(" ")) {
nextToken = afterNextToken;
tokeniser.sval = tokeniser.sval.substring(1);
continueParsing = true;
} else {
tokeniser.pushBack();
}
}
}

Discussion

  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2012-12-27

    P.S. Since it's not visible in comment, the actual content of the property is this:
    DESCRIPTION:This is a lo
    [SPACE]ng description
    [SPACE]that exists on a long line.

    Where [SPACE] is space symbol.
    This is in accordance with WebDAV RFC (link provided above), so has to be fixed in iCal4J parser.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2012-12-27

    P.P.S.
    > This is in accordance with WebDAV RFC
    I meant iCal RFC, sorry.

     
  • Ben Fortuna

    Ben Fortuna - 2013-01-14
    • status: open --> pending-works-for-me
     
  • Ben Fortuna

    Ben Fortuna - 2013-01-14

    Hi,

    Apologies for the delayed response. The logic for handling folded (multiline) properties is actually provided in the UnfoldingReader class. So when you use the parser you should wrap your input in an UnfoldingReader to handle multi-line properties. You can see an example of this in CalendarBuilder.build(Reader).

    regards,
    ben

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-01-14

    "when you use the parser"
    I don't!

    iCal4j Connector calls the parser, so the connector should be fixed then I suppose.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-01-14
    • status: pending-works-for-me --> open-works-for-me
     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-01-14

    P.S. Actually I see that iCal4j Connector uses UnfoldingReader. I suppose somehow it doesn't work correctly then. I'll check in details later.

     
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2013-01-14

    P.P.S. It all seems to be not working - I've tried an UnfoldingReader and it's not unfolding anything.
    I see that pattern "\n " is considered to be "relaxed" somehow, but I suspect there's not way to tell Connector to create UnfoldingReader-s with relaxed=true (it is a constructor parameter for UnfoldingReader).
    Moreover, when I manually create UnfoldingReader with relaxed = true, it fails with buffer overflow, so I have to set another constructor parameter to make it have bigger buffer - and by bigger I mean bigger than size of the Calendar property it's trying to read.
    And even then it doesn't unfold anything!

    Here's the sample code I used.
    public static void main(String[] args) throws Exception {

    String str = "DESCRIPTION:\\n\\n\\nDeelnemers testsѐëöäüї!@#$^ Deelnemers Deelnemers Deelneme\n"
    + " rs Deelnemers Deelnemers Deelnemers Deelnemers Deelnemers Deelnemers Deelnem\n"
    + " ers Deelnemers Deelnemers Deelnemers Deelnemers v v Deelnemers Deelnemers De\n"
    + " elnemers Deelnemers Deelnemers v Deelnemers Deelnemers Deelnemers \\n\nBlah\n";
    // TODO Auto-generated method stub
    UnfoldingReader reader = new UnfoldingReader(new StringReader(str), 65535, true);
    char[] buffer = new char[65535];
    reader.read(buffer);
    System.out.println(new String(buffer));
    reader.close();
    }

    The result outputs same string as input!

     
  • Ben Fortuna

    Ben Fortuna - 2013-01-14

    Ok, first the specification states that a line is folded by the string "\r\n ", whereas you are using "\n ". It is fine to use that but you must enable relaxed unfolding using either CompatibilityHints or an ical4j.properties file. See here for more details:

    http://wiki.modularity.net.au/ical4j/index.php?title=Compatibility

    regards,
    ben

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks