pyparsing-users — User notes and help on the pyparsing module

Re: [Pyparsing] automatic naming of parse results?

[Paul M. <pt...@au...>]

> 
> is it possible to do sth. like .setResultsName globally and automatically?
E.g.:
> 
>     a = pyparsing.Group(b + c)
> 
> could work like:
> 
>     a = pyparsing.Group(b + c).setResultsName("a")
> 

Results names are tricky, but useful.  Part of the nuisance of results names
is the ".setResultsName" syntax, so that is why in 1.4.7 I added support for
code like this:

    a = pyparsing.Group(b + c)("a")

That is, I made all ParserElements callable, and __call__ calls
setResultsName.

Also, despite the leading "set" in "setResultsName", you do know I hope that
this method does an implicit copy, and the copy gets the results name, not
the original.  This is so that you can create a basic expression like
integer, and then use it in multiple places with different names, as in:

integer = Word(nums)
personData = Word(alphas)("name") + integer("age") + integer("salary")

(This actually looks almost like a constructor syntax, which is consistent
with the concept that we are creating new sub-expressions, using the
definition of integer as a prototype.)  But you don't really want integer to
have a results name of "integer", since you need different names for the
different data items, so automatic naming, which seems like a good idea at
first, doesn't hold up - sometimes it would be helpful, but other times it
gets in your way.

If you had a grammar with many sub-expressions, you could do something like
this (where '...' is whatever pyparsing definition you want):

Aexpr = ...
Bexpr = ...
Cexpr = ...
grammar = (Aexpr | Bexpr) + OneOrMore(Cexpr)

for exprName in ("Aexpr Bexpr Cexpr".split()):
    # note that this won't work, since it makes and then discards a copy of
the original
    # locals()[exprName].setResultsName(exprName)

    # have to set the attribute directly, like this
    locals()[exprName].resultsName = exprName

This might be more in line with what you were asking.  I've never set an
expression's results name this way, but I don't see anything wrong with it.

-- Paul

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

On Fri, Apr 4, 2008 at 5:16 PM, Paul McGuire <pt...@au...> wrote:

>
> I've poked a bit at your example, and I don't see any other low-hanging
> fruit to crank more speed out of this parser.  What you might do is try
> converting by hand to an explicit infix notation parser, using the style
> in
> fourFn.py that comes with pyparsing.  Other than that, I would say this
> may
> be as much a pyparsing performance issue as it is operatorPrecedence.
>

thanks for having a look at it. I will be implementing this parser without
pyparsing then.

-ralf

Re: [Pyparsing] (no subject)

[Gre7g L. <haf...@ya...>]

--- "W. Martin Borgert" <de...@de...> wrote:

> I will try this. The main problem is, that my
> grammar is HUGE
> (> 600 productions), so it is a lot of work. Also, I
> already
> suffer both performance and memory problems with
> large input
> files, so I'm curious about the consequences of this
> change :~)

Oof. If you're looking at a huge grammar, then you
might consider the "hacking pyparsing.py" route
instead.

My grammar is only about 45 statements and I was
already parsing nearly everything into objects (all
derived from a single base class) anyhow. So it really
wasn't bad for me to make a single change to my base
class and let that effect everything. You may be
making a much larger gamble with your time by changing
so much code.

Would you like me to take a crack at this hack?

Gre7g

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Paul M. <pt...@au...>]

Ralf -

Part of what you are seeing is the result of a bug-fix in pyparsing 1.4.9 (I
think), in which the operands of an operatorPrecedence were evaluated once
for each level in the precedence chain, including the calling of parse
actions.  To correct this, each level now does a lookahead to see if that
level applies, and only if successful then parses that level, otherwise
descends to the next level of the list.  So now every level is prefixed by a
"FollowedBy" with that level's expression.  Since the levels nest, this gets
geometrically larger as more levels are added to the list.  I ran your code
with a modified version of operatorPrecedence that does not do this, and the
time to parse your given string dropped from .09 to .06 seconds, and the
expression string representation dropped to 202600 chars.

I've poked a bit at your example, and I don't see any other low-hanging
fruit to crank more speed out of this parser.  What you might do is try
converting by hand to an explicit infix notation parser, using the style in
fourFn.py that comes with pyparsing.  Other than that, I would say this may
be as much a pyparsing performance issue as it is operatorPrecedence.

-- Paul

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

On Thu, Apr 3, 2008 at 12:15 AM, Paul McGuire <pt...@au...> wrote:

> This performance problem has come up a couple of times in the past month.
> For a start, you could try enabling packrat parsing - change your import
> and
> following lines to:
>
>
> from pyparsing import (Word, Literal, CaselessLiteral,
>                       Combine, Optional, nums, StringEnd,
> operatorPrecedence,
>                        opAssoc,ParserElement)
> ParserElement.enablePackrat()
>
>
> This will speed up your parser about 500X (not kidding!).
>


Paul,

operatorPrecedence isn't working for me. I think something must be wrong,
it's just much too slow.
I've turned on packrat parsing. The current code can be viewed here:
http://code.pediapress.com/hg/mwlib/file/ce563c294ad1/mwlib/e.py
or downloaded here:
http://code.pediapress.com/hg/mwlib/raw-file/ce563c294ad1/mwlib/e.py

It reads expressions from stdin.

Parsing something like:
5-(((63+459.67)/1.8)<100000)-(((63+459.67)/1.8)<10000)-(((63+459.67)/1.8)<1000)-(((63+459.67)/1.8)<100)-(((63+459.67)/1.8)<10)

takes nearly 0.09 seconds. (i.e. one can only parse around 11 such
expressions, which is unbeliavably slow for a 2.4 Ghz processor).

The string representation of the expr parser element is huge:
In [2]: len(str(e.expr))
Out[2]: 4317534

For my old parser this is length is 22076.

What's going on?

- Ralf

Re: [Pyparsing] (no subject)

[W. M. B. <de...@de...>]

On Thu, Apr 03, 2008 at 08:41:03PM -0700, Gre7g Luterman wrote:
> The solution may not be ideal (since it requires that
> everything be parsed and instantiated), but it works
> and requires no hacking pyparsing.py.  I am using this
> currently in my code.

I will try this. The main problem is, that my grammar is HUGE
(> 600 productions), so it is a lot of work. Also, I already
suffer both performance and memory problems with large input
files, so I'm curious about the consequences of this change :~)

Many thanks!

Re: [Pyparsing] (no subject)

[Gre7g L. <haf...@ya...>]

--- Eike Welk <eik...@gm...> wrote:

> Hello Martin!
> 
> Just recently "Gre7g Luterman" had a very similar
> question. He also 
> proposed a kind of decent solution:

Thanks, "Eike".

You can find this post in the archive:

http://tinyurl.com/yqpn7d

And then there was a follow-up with more refined code
at:

http://tinyurl.com/25gacz

The solution may not be ideal (since it requires that
everything be parsed and instantiated), but it works
and requires no hacking pyparsing.py.  I am using this
currently in my code.

Alternatively, you'd need to hook in somewhere and
track the maximum parse position and return it in the
final object.

Hope that helps,

"Gre7g"

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pyparsing] (no subject)

[Eike W. <eik...@gm...>]

Hello Martin!

Just recently "Gre7g Luterman" had a very similar question. He also 
proposed a kind of decent solution:
He stores the location (in the parsed text) of successfully parsed 
parts of the program. The code for storing is in the parse actions. 
When a syntax error happens he searches the error with special code, 
starting from the position of the last part, that was successfully 
parsed. 

I have a similar but much more simple solution in my compiler:
I store the postion of the last statement that was successfully 
parsed. When a syntax error happens I print: Error below line XXX.

I always wanted to propose some kind of backtracking break for 
Pyparsing. However it is difficult to describe (for me) what exactly 
should happen, so that you would get meaningful error messages. 

Regards, Eike.

[Pyparsing] better parse error reporting possible?

[W. M. B. <de...@de...>]

(sorry, deleted subject accidently...)

[Pyparsing] (no subject)

[<de...@de...>]

Hi,

I still have to find out how to report parse errors better. E.g.
if I pass the following code to the example parser idlParse.py
(in the pyparsing distribution or on the pyparsing web page):

    test(
        """
        module Test1
          {
        module Test2
          {
        module Test3
          {
        module Test4
          {
        module Test5
          {
        module Test6
          {
          xception TestException
            {
            string msg;
            };
          };
          };
          };
          };
          };
          };
        """
        )

I get an error message that does not point to the typo in the
keyword:

        module Test2
        ^
Expected "}" (at char 42), (line:4, col:9)

Any idea on how to behave friendlier and more helpful to the
users of my parser?

Thanks in advance!

[Pyparsing] automatic naming of parse results?

[<de...@de...>]

Hi,

is it possible to do sth. like .setResultsName globally and
automatically? E.g.:

    a = pyparsing.Group(b + c)

could work like:

    a = pyparsing.Group(b + c).setResultsName("a")

if some flag is set or whatever. Reason: I don't like to add a
lot of redundant code to my parser.

Thanks in advance for any hint!

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

On Thu, Apr 3, 2008 at 2:08 PM, Paul McGuire <pt...@au...> wrote:

> Ralf -
>
> Thanks! I can make this patch easily in the next release.
>
> Is there any harm in just defining this function for all versions, instead
> of conditionalizing for v2.6?
>

It might be a bit slower than the builtin hash. Other than that it should be
no problem.

- ralf

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Paul M. <pt...@au...>]

Ralf -

Thanks! I can make this patch easily in the next release.

Is there any harm in just defining this function for all versions, instead
of conditionalizing for v2.6?

-- Paul 

-----Original Message-----
From: pyp...@li...
[mailto:pyp...@li...] On Behalf Of Ralf
Schmitt
Sent: Thursday, April 03, 2008 6:26 AM
To: Paul McGuire
Cc: pyp...@li...
Subject: Re: [Pyparsing] parsing with operatorPrecedence takes too much time

>
> python 2.6 won't let you call hash() on an object which implements 
> it's own __eq__, but doesn't implement __hash__. This is something you 
> might like to fix in an upcoming release..
> I will try to fix that issue and will send you a patch...
>

I'm now using the following workaround:

 if sys.version_info>=(2,6):
    ParserElement.__hash__ = lambda self: hash(id(self))

- ralf
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for just about anything Open
Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Pyparsing-users mailing list
Pyp...@li...
https://lists.sourceforge.net/lists/listinfo/pyparsing-users

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

>
> python 2.6 won't let you call hash() on an object which implements it's
> own __eq__, but doesn't implement __hash__. This is something you might like
> to fix in an upcoming release..
> I will try to fix that issue and will send you a patch...
>

I'm now using the following workaround:

 if sys.version_info>=(2,6):
    ParserElement.__hash__ = lambda self: hash(id(self))

- ralf

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

On Thu, Apr 3, 2008 at 12:15 AM, Paul McGuire <pt...@au...> wrote:

> This performance problem has come up a couple of times in the past month.
> For a start, you could try enabling packrat parsing - change your import
> and
> following lines to:
>
>
> from pyparsing import (Word, Literal, CaselessLiteral,
>                       Combine, Optional, nums, StringEnd,
> operatorPrecedence,
>                        opAssoc,ParserElement)
> ParserElement.enablePackrat()
>
>
> This will speed up your parser about 500X (not kidding!).
>

fine, thanks. Actually I was thinking pyparsing was kidding me when it was
that slow.
(I think this is still rather slow if pyparsing can only parse 500 of those
very basic expressions in one second). I was using a parser, which didn't
use operatorPrecedence before and it never felt slow. Am I better off
writing the parsing rules myself? (like in
http://code.pediapress.com/hg/mwlib/file/tip/mwlib/expr.py)
It's similar to the mathematical expression parser from the examples
directory.


btw, I am running python 2.6 and with this change I get the following error:
~/ python e.py                                                    ralf@rat64ok
Traceback (most recent call last):
  File "e.py", line 41, in <module>
    res=expr.parseString("1+1+1-(5+2)")
  File
"/home/ralf/py26/lib/python2.6/site-packages/pyparsing-1.4.11-py2.6.egg/pyparsing.py",
line 980, in parseString
    loc, tokens = self._parse( instring.expandtabs(), 0 )
  File
"/home/ralf/py26/lib/python2.6/site-packages/pyparsing-1.4.11-py2.6.egg/pyparsing.py",
line 907, in _parseCache
    if lookup in ParserElement._exprArgCache:
TypeError: unhashable type: 'And'

python 2.6 won't let you call hash() on an object which implements it's own
__eq__, but doesn't implement __hash__. This is something you might like to
fix in an upcoming release..
I will try to fix that issue and will send you a patch...

- ralf

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Paul M. <pt...@au...>]

This performance problem has come up a couple of times in the past month.
For a start, you could try enabling packrat parsing - change your import and
following lines to:


from pyparsing import (Word, Literal, CaselessLiteral,
                       Combine, Optional, nums, StringEnd,
operatorPrecedence, 
                       opAssoc,ParserElement)
ParserElement.enablePackrat()


This will speed up your parser about 500X (not kidding!).

-- Paul
 

-----Original Message-----
From: pyp...@li...
[mailto:pyp...@li...] On Behalf Of Ralf
Schmitt
Sent: Wednesday, April 02, 2008 12:44 PM
To: Joshua J. Kugler
Cc: pyp...@li...
Subject: Re: [Pyparsing] parsing with operatorPrecedence takes too much time

On Wed, Apr 2, 2008 at 7:29 PM, Joshua J. Kugler <jk...@sk...>
wrote:

> On Wed, 2008-04-02 at 18:33 +0200, Ralf Schmitt wrote:
> > Hi all,
> >
> > I'm attaching a short test program that tries to parse mathematical 
> > expressions using operatorPrecedence.
> > It takes nearly one second to parse a simple expression like 
> > 1+1+1-(5+2)
> (on
> > a 2.4Ghz machine).
>
> You forgot to attach it. :)
>

hmm. gmail says it's there. I'm pasting it at the end of this email.


> When you say "takes nearly one second" do you mean for the python 
> script to start up, initialize, parse, and exit?  Or one second just 
> for parsing?  Have you tried *just* the parsing routine under 
> something like timeit?
>

I mean only the parsing.


> Check my code against the timeit docs, but something like this:
>
> t = timeit("parse_exp.parseString('1+1+1-(5+2)')",
>    setup='from __main__ import parse_exp)
>
> print t.timeit()
>
> That should give you an idea of how long it's actually taking.
>

no need for timeit. look at this:

#! /usr/bin/env python
# -*- coding: utf-8 -*-

from pyparsing import (Word, Literal, CaselessLiteral,
                       Combine, Optional, nums, StringEnd,
operatorPrecedence, opAssoc)

# define grammar
point = Literal('.')
number=integer = Word(nums)
floatnumber = Combine( (integer +
                       Optional( point + Optional(number) ) +
                       Optional(  CaselessLiteral('e') + integer ))
                       | (point + Optional(number) + Optional(
CaselessLiteral('e') + integer ))
                     )

operand = floatnumber | integer

plus  = Literal("+")
minus = Literal("-")
mult  = Literal("*")
div   = Literal("/") | CaselessLiteral("div") | CaselessLiteral("mod")
addop  = plus | minus
multop = mult | div

cmpop = Literal("<>") | Literal("!=") | Literal("=") | Literal("<=") |
Literal(">=") | Literal(">") | Literal("<")

expr = operatorPrecedence(operand, [
        (Literal("+") | Literal("-") | CaselessLiteral("not"), 1,
opAssoc.RIGHT),
        (multop, 2, opAssoc.LEFT),
        (addop, 2, opAssoc.LEFT),
        (Literal("round"), 2, opAssoc.LEFT),
        (cmpop, 2, opAssoc.LEFT),
        (CaselessLiteral("and"), 2, opAssoc.LEFT),
        (CaselessLiteral("or"), 2, opAssoc.LEFT),
        ]) + StringEnd()

import time
stime=time.time()
res=expr.parseString("1+1+1-(5+2)")
print time.time()-stime, res

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

On Wed, Apr 2, 2008 at 7:29 PM, Joshua J. Kugler <jk...@sk...>
wrote:

> On Wed, 2008-04-02 at 18:33 +0200, Ralf Schmitt wrote:
> > Hi all,
> >
> > I'm attaching a short test program that tries to parse mathematical
> > expressions using operatorPrecedence.
> > It takes nearly one second to parse a simple expression like 1+1+1-(5+2)
> (on
> > a 2.4Ghz machine).
>
> You forgot to attach it. :)
>

hmm. gmail says it's there. I'm pasting it at the end of this email.


> When you say "takes nearly one second" do you mean for the python script
> to start up, initialize, parse, and exit?  Or one second just for
> parsing?  Have you tried *just* the parsing routine under something like
> timeit?
>

I mean only the parsing.


> Check my code against the timeit docs, but something like this:
>
> t = timeit("parse_exp.parseString('1+1+1-(5+2)')",
>    setup='from __main__ import parse_exp)
>
> print t.timeit()
>
> That should give you an idea of how long it's actually taking.
>

no need for timeit. look at this:

#! /usr/bin/env python
# -*- coding: utf-8 -*-

from pyparsing import (Word, Literal, CaselessLiteral,
                       Combine, Optional, nums, StringEnd,
operatorPrecedence, opAssoc)

# define grammar
point = Literal('.')
number=integer = Word(nums)
floatnumber = Combine( (integer +
                       Optional( point + Optional(number) ) +
                       Optional(  CaselessLiteral('e') + integer ))
                       | (point + Optional(number) + Optional(
CaselessLiteral('e') + integer ))
                     )

operand = floatnumber | integer

plus  = Literal("+")
minus = Literal("-")
mult  = Literal("*")
div   = Literal("/") | CaselessLiteral("div") | CaselessLiteral("mod")
addop  = plus | minus
multop = mult | div

cmpop = Literal("<>") | Literal("!=") | Literal("=") | Literal("<=") |
Literal(">=") | Literal(">") | Literal("<")

expr = operatorPrecedence(operand, [
        (Literal("+") | Literal("-") | CaselessLiteral("not"), 1,
opAssoc.RIGHT),
        (multop, 2, opAssoc.LEFT),
        (addop, 2, opAssoc.LEFT),
        (Literal("round"), 2, opAssoc.LEFT),
        (cmpop, 2, opAssoc.LEFT),
        (CaselessLiteral("and"), 2, opAssoc.LEFT),
        (CaselessLiteral("or"), 2, opAssoc.LEFT),
        ]) + StringEnd()

import time
stime=time.time()
res=expr.parseString("1+1+1-(5+2)")
print time.time()-stime, res

Re: [Pyparsing] parsing with operatorPrecedence takes too much time

[Joshua J. K. <jk...@sk...>]

On Wed, 2008-04-02 at 18:33 +0200, Ralf Schmitt wrote:
> Hi all,
> 
> I'm attaching a short test program that tries to parse mathematical
> expressions using operatorPrecedence.
> It takes nearly one second to parse a simple expression like 1+1+1-(5+2) (on
> a 2.4Ghz machine).

You forgot to attach it. :)

When you say "takes nearly one second" do you mean for the python script
to start up, initialize, parse, and exit?  Or one second just for
parsing?  Have you tried *just* the parsing routine under something like
timeit?

Check my code against the timeit docs, but something like this:

t = timeit("parse_exp.parseString('1+1+1-(5+2)')",
    setup='from __main__ import parse_exp)

print t.timeit()

That should give you an idea of how long it's actually taking.

j

-- 
Joshua Kugler
VOC/SigNet Provider (aka Web App Programmer)
S&K Aerospace Alaska
1

[Pyparsing] parsing with operatorPrecedence takes too much time

[Ralf S. <sc...@gm...>]

Hi all,

I'm attaching a short test program that tries to parse mathematical
expressions using operatorPrecedence.
It takes nearly one second to parse a simple expression like 1+1+1-(5+2) (on
a 2.4Ghz machine).

Any hints? Am I doing something wrong?

Regards,
- Ralf

Re: [Pyparsing] Salvaging a failed parse

[Paul M. <pt...@au...>]

Greg -

>>>
Does PyParsing expose a method to convert this to a line/column like the
ParseException does, or should I just count up the CR's and do the math
myself?
>>>

Yes, pyparsing has lineno(loc,string) and col(loc,string) methods for
converting an absolute loc into line number and column number.  You can see
them in use in the body of this FAQ:
http://pyparsing-public.wikispaces.com/FAQs#toc2.  The line(loc,string)
function will give you the line of text corresponding to the loc value.
(Beware when using col() if there are tabs in your input text.  Pyparsing
expands tabs to 8-space tabstops by default so column numbers may end up
different than you expect.  Use parseWithTabs to change this default
behavior.  The html docs that come with pyparsing have more details on this
issue.)

There are also usage examples in
http://pyparsing.wikispaces.com/space/showimage/linenoExample.py.

Sorry to not respond sooner, I was out of the country on vacation, and just
getting my head back together.

-- Paul

Re: [Pyparsing] Salvaging a failed parse

[Gre7g L. <haf...@ya...>]

--- Gre7g Luterman <haf...@ya...> wrote:

<snipped>

> Here's a short example program illustrating this:
> 
> http://pastie.textmate.org/167541

Here's a more refined version of what I had in mind:

http://pastie.textmate.org/167851

This version converts EVERYTHING to objects (even
literals). Then, if the parse fails, it reparses the
last item successfully parsed, skips over any
whitespace or comments, and then finds the next
interesting bit of text.

This allows me to report...

Parse error: did not expect 'else' on line 10 (column
5)

Ideally, I'd like the parser to report that it was
hoping to find a ";" in the assignment statement, but
that would require hacking into PyParsing since I
can't guarantee the parent object has been
instantiated yet (this won't happen until all of the
required bits parse successfully).

Regardless, it's better than reporting an error on the
" 7" (the last thing parsed successfully) and far more
desirable than reporting an error on the "for"
statement, since that couldn't be parsed entirely.

Does anyone have any thoughts on this? I'll gladly
stick with what I have since it works, but if there's
a better way, I'd love to talk about it.

Thanks,

Gre7g

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pyparsing] problem installing pyparsing

[Paul M. <pt...@au...>]

-----Original Message-----
From: pyp...@li...
[mailto:pyp...@li...] On Behalf Of Isabelle
Cabrera
Sent: Tuesday, March 18, 2008 11:56 AM
I've installed rpm libpython2.5-devel and now it works.
Thank you :)

Isabelle

================

Well, I'm glad this resolved your problem, but the real question is, why
does the pyparsing setup.py require the devel install of Python?  I don't
know what in the pyparsing setup.py is trying to run the Python makefile,
any distutils experts out there?

-- Paul

[Pyparsing] Salvaging a failed parse

[Gre7g L. <haf...@ya...>]

First off, let me just say that the compiler I wrote
with PyParsing is actually functional and is working
amazingly well. It has left me absolutely speechless.
I can't even begin to imagine how much work it would
have taken to get to this point without an incredible
tool like this.

Secondly, I apologize for my earlier post asking about
something that I should have seen in the FAQs. I
didn't even notice that there was an FAQ section on
the website! Doh.

Finally, on to the question at hand... How can I best
recover from a failed parse? I realize that's a really
open-ended question, so let me take a moment to
explain what's happening, how my gut says I should
handle this, and then I hopefully I can get some
feedback on whether this is the best way to go about
it or not.

My compiler is really good at detecting logic errors,
like trying to assign a value to a constant, but if
the user's error causes the parse to fail, then I
don't have much useful feedback I can give them.

For example, here's a chunk of invalid code:

a=5;
for (i = 5; i < 10; i = 7)
{
    c=9;

    if (a < 6)
        a = 7
    else
        b = 9;
}
x = 5;

Note that "a = 7" is missing a ";". The parser will
decide that the assignment doesn't match the pattern I
specified, so therefor the "if" statement doesn't
match the pattern I specified, so therefore the "for"
statement doesn't match the pattern I specified.

The parser is 100% correct, but if I tell the user
"Expected end of text (at char 6), (line:3, col:1)"
then I'm not being helpful. The problem is farther
down and if I tell them the problem is on the for
statement, they will never find it.

Here's a short example program illustrating this:

http://pastie.textmate.org/167541

My gut says that the way to "fix" this is to look at
the position argument passed to my setParseAction
instances and keep track of the farthest position
through the file that the parser managed to get. The
logic here being that farthest parse position was
*PROBABLY* the closest to being successful.

I won't really have a good idea of why the parse
failed, if it does, but at least I'll have a position
in the file, and that's a good start.

Here's some more example code, illustrating the
process:

http://pastie.textmate.org/167542

Is this how you'd handle this? Is there a better way?
Is there a pitfall I'm ignoring?

Also, since this will only give me a linear position
into the file, does PyParsing expose a method to
convert this to a line/column like the ParseException
does, or should I just count up the CR's and do the
math myself?

Many thanks again,

Gre7g

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Pyparsing] problem installing pyparsing

[Isabelle C. <isa...@in...>]

Joshua J. Kugler a écrit :
> On Tue, 2008-03-18 at 16:55 +0100, Isabelle Cabrera wrote:
>   
>> Hi,
>>
>> I have a problem installing pyparsing (v 1.4.11):
>>
>> $ sudo python setup.py install
>> running install
>> error: invalid Python installation: unable to open 
>> /usr/lib/python2.5/config/Makefile (No such file or directory)
>>
>> I do have a directory /usr/lib/python2.5, but I don't have directory 
>> config/ under it.
>>
>> Do you know what's wrong ?
>>     
>
> That file isn't in the main python RPM.  I can't find a copy right now
> (I'm not running on Mandriva), but I would guess that install
> python-devel would give you what you need.
>
> Hope that helps.
>
> j
>   
I've installed rpm libpython2.5-devel and now it works.
Thank you :)

Isabelle


-- 
Isabelle Cabrera
Projet Alpage - INRIA Rocquencourt
http://alpage.inria.fr
Tel : 01 3963 5270

Re: [Pyparsing] problem installing pyparsing

[Joshua J. K. <jk...@sk...>]

On Tue, 2008-03-18 at 16:55 +0100, Isabelle Cabrera wrote:
> Hi,
> 
> I have a problem installing pyparsing (v 1.4.11):
> 
> $ sudo python setup.py install
> running install
> error: invalid Python installation: unable to open 
> /usr/lib/python2.5/config/Makefile (No such file or directory)
> 
> I do have a directory /usr/lib/python2.5, but I don't have directory 
> config/ under it.
> 
> Do you know what's wrong ?

That file isn't in the main python RPM.  I can't find a copy right now
(I'm not running on Mandriva), but I would guess that install
python-devel would give you what you need.

Hope that helps.

j

-- 
Joshua Kugler
VOC/SigNet Provider (aka Web App Programmer)
S&K Aerospace Alaska