Phrase error recovery always throws away unexpected token
Brought to you by:
alder
Menu
▾
▴
#31 Phrase error recovery always throws away unexpected token
When the parser reaches phrase-level error recovery, it never retries the unexpected token after shifting the error token. This is problematic when that token begins a valid phrase -- the result is that a valid phrase following an error phrase is also thrown away as a part of the error phrase.
The attached patch addresses this issue, as well as improving the accuracy of the source range of the error token. (I also changed the termination condition for the simulator since reinserting the unexpected token can make the stream buffer full.)
Actually, there is a mistake in that patch -- the simulator loop should initialize with "int shifted = 1;".
Err.. actually that's wrong. I'll update the patch when I fix it. :/
Probably the most performance efficient solution is to just increase the buffer size, since a capacity of 3 is one too few for the case when the third token encountered cannot be shifted during delete-the-token error recovery.