Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Paul M. <pt...@au...>]

This performance problem has come up a couple of times in the past month.
For a start, you could try enabling packrat parsing - change your import and
following lines to:


from pyparsing import (Word, Literal, CaselessLiteral,
                       Combine, Optional, nums, StringEnd,
operatorPrecedence, 
                       opAssoc,ParserElement)
ParserElement.enablePackrat()


This will speed up your parser about 500X (not kidding!).

-- Paul
 

-----Original Message-----
From: pyp...@li...
[mailto:pyp...@li...] On Behalf Of Ralf
Schmitt
Sent: Wednesday, April 02, 2008 12:44 PM
To: Joshua J. Kugler
Cc: pyp...@li...
Subject: Re: [Pyparsing] parsing with operatorPrecedence takes too much time

On Wed, Apr 2, 2008 at 7:29 PM, Joshua J. Kugler <jk...@sk...>
wrote:

> On Wed, 2008-04-02 at 18:33 +0200, Ralf Schmitt wrote:
> > Hi all,
> >
> > I'm attaching a short test program that tries to parse mathematical 
> > expressions using operatorPrecedence.
> > It takes nearly one second to parse a simple expression like 
> > 1+1+1-(5+2)
> (on
> > a 2.4Ghz machine).
>
> You forgot to attach it. :)
>

hmm. gmail says it's there. I'm pasting it at the end of this email.


> When you say "takes nearly one second" do you mean for the python 
> script to start up, initialize, parse, and exit?  Or one second just 
> for parsing?  Have you tried *just* the parsing routine under 
> something like timeit?
>

I mean only the parsing.


> Check my code against the timeit docs, but something like this:
>
> t = timeit("parse_exp.parseString('1+1+1-(5+2)')",
>    setup='from __main__ import parse_exp)
>
> print t.timeit()
>
> That should give you an idea of how long it's actually taking.
>

no need for timeit. look at this:

#! /usr/bin/env python
# -*- coding: utf-8 -*-

from pyparsing import (Word, Literal, CaselessLiteral,
                       Combine, Optional, nums, StringEnd,
operatorPrecedence, opAssoc)

# define grammar
point = Literal('.')
number=integer = Word(nums)
floatnumber = Combine( (integer +
                       Optional( point + Optional(number) ) +
                       Optional(  CaselessLiteral('e') + integer ))
                       | (point + Optional(number) + Optional(
CaselessLiteral('e') + integer ))
                     )

operand = floatnumber | integer

plus  = Literal("+")
minus = Literal("-")
mult  = Literal("*")
div   = Literal("/") | CaselessLiteral("div") | CaselessLiteral("mod")
addop  = plus | minus
multop = mult | div

cmpop = Literal("<>") | Literal("!=") | Literal("=") | Literal("<=") |
Literal(">=") | Literal(">") | Literal("<")

expr = operatorPrecedence(operand, [
        (Literal("+") | Literal("-") | CaselessLiteral("not"), 1,
opAssoc.RIGHT),
        (multop, 2, opAssoc.LEFT),
        (addop, 2, opAssoc.LEFT),
        (Literal("round"), 2, opAssoc.LEFT),
        (cmpop, 2, opAssoc.LEFT),
        (CaselessLiteral("and"), 2, opAssoc.LEFT),
        (CaselessLiteral("or"), 2, opAssoc.LEFT),
        ]) + StringEnd()

import time
stime=time.time()
res=expr.parseString("1+1+1-(5+2)")
print time.time()-stime, res