Re: [Pyparsing] left-recursion again: expression of attribute

[spir ☣ <den...@gm...>]

On Tue, 30 Mar 2010 22:43:11 +0200
Eike Welk <eik...@gm...> wrote:

> Hello Denis!
> 
> On Tuesday March 30 2010 21:31:58 spir ☣ wrote:
> > Hello,
> > 
> > This time, that's me having a left-recursion issue. I'm trying to parse var
> >  names that can possibly refer to attributes, like "a.b.c". I can parse it
> >  as simpleName + ZeroOrMore(extension)
> > but then I need to reformat the result resursively, to get the real
> >  semantics of: getattr(getattr(container, name), name)
> > i.e. in the case of "a.b.c":
> > 	((a).b).c
> 
> It is impossible to do what you want in an elegant way, you must reformat the 
> parse result.

Right, I knew it in fact ;-) but was expecting kind of a miracle!

> I had a similar question some time ago, this is the simplified 
> summary of Paul's answer. 
> 
> For code that gives you the right operator precedence look at Pyparsing's 
> operatorPrecedence(...) or at the calculator example. It also gives you a 
> parse result that can be relatively easily reformatted. 
> 
> By the way, if you really implement a programming language with Pyparsing use 
> operatorPrecedence(...) it can parse nearly all of Python's expressions 
> including function calls. 
[...]

I don't, in fact (even may swith to Lua for several reasons). And there is no operator precedence in the language I'm parsing.

For possible interest: As of now, operators map to class methods, meaning eg "+" does not map to
   self.add(other)
but to
   Number.add(numbers)
and this scheme may (as of now, it does) mirror in a prefix notation syntax:
   +(n1 n2 n3 ...)
instead of
   n1 + n2 + n3 ...

Thus there is no operator precedence at all.
(Anyway, I wanted no arbitrary precedence between operators of the same "kind" (arithmetics, logical...), so I would have imposed parens if not used prefix notation.)

I have always found operators mapping to instance methods simply wrong (less wrong for unary operators, but still). So, this scheme brings me both advantages at once.

> If you are interested I can talk you though my still very big parser 
> implementation, which can be found here (look at line 979):
> http://tinyurl.com/yc3gyqc
> 
> Original URL, but Kmail messes this URL up:
> http://bazaar.launchpad.net/~vcs-
> imports/freeode/trunk/annotate/head:/freeode_py/freeode/simlparser.py

I have had a look, thank you. But your app is really big ;-) I'm exploring it superficially, looks interesting and the implementation is really clean :-)

* custom nodes *
Just a side note you may find interesting, since you yield nodes of custom type, to represent chunks of source code, via parse actions:
For my present app, I introduce a modif in my parsing library that allows specifying a class, instead of a func, as match (parse) action. In this case, instead of instantiating a standard node (parse result) from the match result, and then applying an action on it --that will possibly yield a custom node, as you do--, the matching method directly calls the class (with the same args as if it were a standard node), and returns it normally -- so that it will be inserted in the tree like any other node.
Hem, seems I'm not clear, so the sketch is:

class KindOfPattern(Pattern):
    ...
    def _match(source):
        # (the source holds its current pos, after matching it holds the range)
        result = <try and match, else MatchFailure exception>
        # result is either matched source snippet
        # or a sequence of child nodes
        action = self.action
        if isinstance(action, type):
            # case custom
            node = action(self, result, source)
            # needed transformation, if any, should be done in __init__
        else:
            # case standard
            node = Node(self, result, source)
            # node applies possible action itself, if not None
        return node

[A disadvantage is I cannot use anymore builtin match actions that reformat result nodes according to common needs (drop, extract sub result, join, flatten leaves, debug output...). So, I'll have to reintroduce the possibility to specify several actions (which I removed before, because one action can simply call others: "def action2(node): action1(node); ...").]

Every custom node is regarded (from a higher-level node's point of view) as a single / simple node, meaning it becomes a leaf whatever it represents. But it may be useful that custom nodes _act_ like composite ones when they conceptually are, by implementing indexing/iteration, treeview, whatever...

Don't know if Paul likes this option and would like to introduce it in pyparsing.

An intermediate solution is to modify the lib itself so as to have only custom nodes ;-) I did this first, first by making Node a subtype of my top Data type (so that the door is open to homoiconicity). But this is not very practicle, since usually lower-level matches are well dealt with as standard nodes, before they become input data for higher-level ones.


Denis
________________________________

vit esse estrany ☣

spir.wikidot.com