exception wrongly caught

[Tim Coote]

I use fedora and the current version of pyparsing is still 1.5.0, so this issue may have changed/gone away. Should the fedora project be bumping the pyparsing version?

I'm parsing some logs with email addresses embedded in <>. Sometimes, there's no email address. A simplistic test case of my initial code (hacked out of much more code) looked like this:

from pyparsing import Keyword, Word,   Group, Literal, alphanums,    ZeroOrMore,  ParseResults
rejectedMsgs = set ('')
def addRejFrom (s,l,t):
    print "add rej from", s, l, t, s[l-2:l+3]
#    if len (t) > 0: rejectedMsgs.add (t[0])    rejectedMsgs.add (t[0])
embeddedMailAddress = Literal ("<").suppress() + ZeroOrMore (Word (alphanums)) + Literal (">").suppress()
fromInfo = Literal ("from=").suppress () + embeddedMailAddress ("embeddedMailAddress")
noQueueFrom = fromInfo ("qfrominfo").setParseAction (addRejFrom)
logLine= Group ( fromInfo.setParseAction (addRejFrom) + Literal ("l")) ("Line")
broken="""from=<> l"""
print logLine.parseString (broken).dump ()

This code fails on the string broken as follows:

[tim@mercury mailout]$ python test2.py
add rej from from=<> l 0 [] 
Traceback (most recent call last):
  File "test2.py", line 21, in <module>
    print logLine.parseString (broken).dump ()
  File "/usr/lib/python2.5/site-packages/pyparsing.py", line 1049, in parseString
    loc, tokens = self._parse( instring, 0 )
  File "/usr/lib/python2.5/site-packages/pyparsing.py", line 925, in _parseNoCache
    loc,tokens = self.parseImpl( instring, preloc, doActions )
  File "/usr/lib/python2.5/site-packages/pyparsing.py", line 2560, in parseImpl
    return self.expr._parse( instring, loc, doActions, callPreParse=False )
  File "/usr/lib/python2.5/site-packages/pyparsing.py", line 927, in _parseNoCache
    raise ParseException( instring, len(instring), self.errmsg, self )
pyparsing.ParseException:  (at char 9), (line:1, col:10)

What is not obvious from this traceback is that the error was caught in addRejFrom because the list of tokens (t) is empty, so t should be throwing a list index out of range. In fact, if I make embeddedMailAddress a Group (…), I get the exception that I'd expect.

Is this a bug? (It certainly makes it hard to debug the grammar). Is it fixed in newer versions of pyparsing, and, as I implied, how can I encourage the fedora project to use the newer versions if it is? (are 1.5.x releases regarded as experimental, or something)?

Lot of questions. Sorry, I'm new here :-)

[Paul McGuire]

Tim -

The short answer is, this behavior still exists in the latest version of pyparsing.  You might get some insights by using the traceParseAction decorator that is included in pyparsing, to help troubleshoot issues like this.  This has been reported before, and I'm on the fence about just how (or even if) to fix it.  Parse actions are there for *any* kind of logic the developer might want, and I'm not sure that I want pyparsing to trap exceptions thrown by them.

I can't speak for Fedora.  I've had a sketchy history with some of my 1.5.x versions (took me 3 versions to finally work out the kinks in the Python3 compatibility), but 1.5.5 should be pretty stable, and should be safe for them to adopt.  I don't have too much experience with Fedora, but "easy_install pyparsing upgrade" (perhaps with root privs) should get you the latest and greatest for your own system.

• Paul

[Tim Coote]

Thanks, Paul. I'll try that decorator: it sounds like it could help me with a lot of pain. Grammar's can be a mess to debug as a simple change can dramatically impact all sorts of behaviour, and I've not been disciplined enough to put in place all the tests that I should.

My suggestion would be that the default behaviour doesn't catch the exceptions with some mechanism to change that default. That makes the default behaviour safe for naive / intermittent users (like me :-) ) with a simple extension for those who are more full-on.

I personally try to stick with stock packages from distros, otherwise migration through versions is very difficult. I'll see what I can do to get the version bumped.

Tim