#77 Empty error messages in parsing exceptions

[dkasak]

Sometimes pyparsing will give an empty message in parsing exceptions in certain cases, even for very simple grammars. It seems this behaviour is triggered when a line ends with whitespace.

Consider the following simple parser:

from pyparsing import *

foo = Keyword("foo")
bar = Keyword("bar")

line = foo - bar

When I run:

line.parseString("foo")

I get the following output:

Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 989, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1641, in parseImpl
    if (instring[loc] == self.firstMatchChar and
IndexError: string index out of range

During handling of the above exception, another exception occurred:

pyparsing.ParseException: Expected "bar" (at char 3), (line:1, col:4)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 8, in <module>
    line.parseString("foo")
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1125, in parseString
    raise exc
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1115, in parseString
    loc, tokens = self._parse( instring, 0 )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 989, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 2377, in parseImpl
    raise ParseSyntaxException(pe)
pyparsing.ParseSyntaxException: Expected "bar" (at char 3), (line:1, col:4)

Notice that it correctly tells me that the keyword "bar" is expected. However, if I run this instead:

line.parseString("foo ")

I get this output:

Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 2372, in parseImpl
    loc, exprtokens = e._parse( instring, loc, doActions )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 993, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1641, in parseImpl
    if (instring[loc] == self.firstMatchChar and
IndexError: string index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "test.py", line 8, in <module>
    line.parseString("foo ")
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1125, in parseString
    raise exc
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 1115, in parseString
    loc, tokens = self._parse( instring, 0 )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 989, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
File "/usr/lib/python3.4/site-packages/pyparsing.py", line 2379, in parseImpl
    raise ParseSyntaxException( ParseException(instring, len(instring), self.errmsg, self) )
pyparsing.ParseSyntaxException:  (at char 4), (line:1, col:5)

The error message of the exception is now empty instead of telling me that "bar" is expected. I tried looking at the source code a bit and found that in pyparsing.py:1586, the Literal class sets mayIndexError to False. Since the IndexError from the traceback was being raised from that class, I tried changing this to True and then the correct error message is display in both cases. I'm not sure if this is the proper fix since I haven't looked at the code in close detail.

I'm using pyparsing 2.0.3.

[dkasak]

Actually, the exception is raised in the Keyword class, so the relevant mayIndexError which changes the behaviour is in that class on line 1624.