how to get more info about parsing failure

2005-04-07
2013-05-14
  • Klaas Hofstra
    Klaas Hofstra
    2005-04-07

    Hi all,

    I'm parsing a design language. Syntax errors in the input result in Parse Exceptions or an empty result list. I would like to get more info about which part of the input didn't match (i.e. the syntax error). I can imagine that the parser could return the tokens that it could not match. This would help tracking down the syntax error. Are there ways to do such a thing?

    Cheers,
    Klaas

     
    • Paul McGuire
      Paul McGuire
      2005-04-07

      Klaas -

      Well, picking thru ParseExceptions isn't always the easiest thing.  I've been reminded several times that pyparsing needs quite a bit more help in documenting the debugging, diagnostic, and troubleshooting processes.  pyparsing has always had the setDebug() option, which you can set on individual elements to track when that element is about to be matched, and then whether the match succeeded or threw an exception.  In 1.3, I added another method called setDebugActions() that allows you to customize these routines to do your own diagnostic logging, but I never got around to documenting them very much.  In looking at the implementation of setDebugActions(), I'm actually a little underwhelmed, it seems this is an item I only implemented partially and never got back to.  (For instance, it is difficult to specify only 1 of the 3 debug actions start/match/no-match, you have to specify all three.  Also, there is no indication or documentation of what the debug actions are supposed to accept in terms of arguments.)  Also, I left out a suggested enhancement to ParseException that provides a marked-up text line when an exception is raised, so this will have to go out in 1.3.1.

      Meanwhile, I can suggest that you either use setDebug( ) on elements that don't match; use the ParseException handler in the idlParse.py example as a template for extracting a bit more information from a ParseException than just the location, line, and column numbers; or open up the pyparsing.py code, check out the default debug actions, and try writing one of your own and using the new setDebugActions method.

      The other thing to note is that pyparsing is not always very good at identifying what you and I would consider the true exception location.  I've looked at this a number of times and have not come up with a solution, so the ParseExceptions you get may report exceptions that appear to be somewhat removed from the actual syntax error.

      -- Paul