Re: [Pyparsing] Using pyparsing to implement a matching engine for a regexp-like matching language?

[Paul M. <pt...@au...>]

Andrew -

Welcome to Pyparsing!  This is a very interesting application.

There are two examples in the pyparsing wiki that may help you in your work:
an EBNF parser and generator - parses EBNF and generates a pyparsing grammar
that implements that BNF
(http://pyparsing.wikispaces.com/space/showimage/ebnf.py and
http://pyparsing.wikispaces.com/space/showimage/ebnftest.py)
a regex inverter - parses a regex (restricted forms only) and returns a
generator that generates all input strings that would match that regex
(http://pyparsing-public.wikispaces.com/space/showimage/invRegex.py)

Unfortunately, I'd say these are both fairly advanced examples, and maybe a
bit overwhelming to a new pyparsing user.  Please work through some of the
simpler examples (marked with a "Start" icon) and then take a look at these
others.  

The basic "engine" or structure of a pyparsing program is:
- compose the parsing expressions
- call parseString
- process the returned ParseResults object (can be thought of like a list,
but advanced usages will return nested lists, and also dict-style results,
with matched tokens retrievable by name)

Here is the basic "Hello, World!" (or any greeting of the form "<word> ,
<word> <punctuation>") parser:

from pyparsing import Word, alphas, oneOf
# compose the parsing expression, using pyparsing classes and builtins
greet = Word(alphas) + "," + Word(alphas) + oneOf("! . ?")

# parse the input string
greetingTokens = greet.parseString("Howdy, Pardner!")

# process the returned results
for i,token in enumerate(greetingTokens):
    print i,token

Prints:
0 Howdy
1 ,
2 Pardner
3 !

I'd also recommend following some of the documentation links for more
detailed examples and discussions.

I have considered converting pyparsing to a streaming-type (or push) parser,
but as it turns out, this would be a radical redesign, maybe in version 2
someday.  I don't know of any other modules to refer you to that might
support push parsing.

-- Paul