Re: [Pyparsing] pyparsing, AST and named results

[Paul M. <pt...@au...>]

Alexey -

I'm so sorry not to have responded earlier, and apologize in advance that
I'm still not sending a substantive reply.  I just started a new job, and my
pyparsing time is very limited.

I'll try to look at your suggestion this weekend.

-- Paul

(Unfortunately, some of my unit test cases use proprietary data formats that
I can't give out publicly.)


> -----Original Message-----
> From: Alexey Borzenkov [mailto:sn...@gm...]
> Sent: Wednesday, August 26, 2009 3:58 AM
> To: Pyp...@li...
> Subject: [Pyparsing] pyparsing, AST and named results
> 
> Hi all,
> 
> I've sent following letters to Paul 20 days ago, but unfortunately
> didn't receive any reply. :( So, I'm forwarding it to this list, maybe
> someone else would be interested in the following patch...
> 
> ---------- Forwarded message ----------
> From: Alexey Borzenkov <sn...@gm...>
> Date: Tue, Aug 4, 2009 at 8:43 PM
> Subject: pyparsing, AST and named results
> To: Paul McGuire <pt...@us...>
> 
> Hi Paul,
> 
> I've been using pyparsing to generate a simple AST, and among others I
> had AST.Body class for sequence of statements. The problem was that as
> soon as I implemented __len__, __iter__ and __getitem__ on AST.Body I
> started having weird problems (all statements but the first one
> disappearing from compiled code), which I traced to this patch:
> 
> [...older patch snipped...]
> 
> Basically, because I implemented __getitem__, and because I was using
> named parameters, only the first element (the first statement) was
> getting assigned under that name. This is a very quick (and possibly
> incomplete fix), but it worked in my case.
> 
> [...other irrelevant info snipped...]
> 
> Thanks,
> Alexey.
> 
> ---------- Forwarded message ----------
> From: Alexey Borzenkov <sn...@gm...>
> Date: Wed, Aug 5, 2009 at 2:31 PM
> Subject: Re: pyparsing, AST and named results
> To: Paul McGuire <pt...@us...>
> 
> 
> Hi Paul,
> 
> It's me again. I've been looking at ParseResults even more, and wonder
> if under "if name:" the intension is not to assign empty results under
> a name? Because as I see it, the code doesn't check for empty
> ParseResults, and I'm wondering if that's intentional or not. If empty
> ParseResults don't have any more special meaning than empty lists,
> then perhaps it could be patched this way:
> 
> diff --git a/src/pyparsing.py b/src/pyparsing.py
> index 57e938a..8dbdec1 100644
> --- a/src/pyparsing.py
> +++ b/src/pyparsing.py
> @@ -277,14 +277,15 @@ class ParseResults(object):
>     # constructor as small and fast as possible
>     def __init__( self, toklist, name=None, asList=True, modal=True ):
>         if self.__doinit:
> +            if isinstance(toklist, list):
> +                toklist = toklist[:]
> +            else:
> +                toklist = [toklist]
>             self.__doinit = False
>             self.__name = None
>             self.__parent = None
>             self.__accumNames = {}
> -            if isinstance(toklist, list):
> -                self.__toklist = toklist[:]
> -            else:
> -                self.__toklist = [toklist]
> +            self.__toklist = toklist
>             self.__tokdict = dict()
> 
>         if name:
> @@ -293,9 +294,7 @@ class ParseResults(object):
>             if isinstance(name,int):
>                 name = _ustr(name) # will always return a str, but
> use _ustr for consistency
>             self.__name = name
> -            if not toklist in (None,'',[]):
> -                if isinstance(toklist,basestring):
> -                    toklist = [ toklist ]
> +            if toklist and toklist[0] != '':
>                 if asList:
>                     if isinstance(toklist,ParseResults):
>                         self[name] =
> _ParseResultsWithOffset(toklist.copy(),0)
> @@ -303,10 +302,7 @@ class ParseResults(object):
>                         self[name] =
> _ParseResultsWithOffset(ParseResults(toklist[0]),0)
>                     self[name].__name = name
>                 else:
> -                    try:
> -                        self[name] = toklist[0]
> -                    except (KeyError,TypeError,IndexError):
> -                        self[name] = toklist
> +                    self[name] = toklist[0]
> 
>     def __getitem__( self, i ):
>         if isinstance( i, (int,slice) ):
> 
> The way I see it, it might even have some performance improvement,
> because toklist now is always ParseResults or a list (just like in
> __toklist), and I checked that toklist should never be None, this
> leaves only one comparison (with '') instead of three, and no
> unnecessary indexing or try/except. The question is will it break
> anything?
> 
> I can't complete all unitTests (examples are missing in svn, and not
> everything is in 1.5.2 release), but my changes don't make non-failing
> ones fail.
> 
> Also, about empty ParseResults as names, the only case I could come up
> with is something like this:
> 
> from pyparsing import *
> 
> s = "()"
> g = (Suppress('(') + ZeroOrMore('.') + Suppress(')'))('dots') +
> StringEnd()
> print repr(g.parseString(s).dots)
> 
> With my changes dots will now disappear, but it seems more consistent
> to me, because ZeroOrMore('.')('dots') would not appear under the
> name, why a bunch of suppressed tokens should make a difference?
> 
> Thanks,
> Alexey.
> 
> --------------------------------------------------------------------------
> ----
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-
> Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Pyparsing-users mailing list
> Pyp...@li...
> https://lists.sourceforge.net/lists/listinfo/pyparsing-users