RE: [Pyparsing] Combining strings

[Paul M. <pa...@al...>]

Mike -

Welcome to the world of pyparsing!  This is a good start at implementing a
little shell language (perhaps to provide a safe scripting environment?).

Your program has just about all the necessary pieces to solve your problem.
Actually, I think most of the solution lies in *removing* code!

I went through 2 stages in looking at your problem.  First I thought of
doing a 2-pass parse on this input: first pass to just do the symbol
substitution, then the second pass to interpret the modified command string.
Second stage was to incorporate the symbol substitution into the overall
command parsing, as your program had been doing.  I'm still deciding which
approach I like better, maybe we'll come back to that.

So in stage 1, I looked just at your pyvar definition.  (It looks like you
want to replace '%(xxx)' or '%xxx' with the contents of some previously
defined value 'xxx'):

  pyvar       = Combine( Optional(printables) + Literal('%').suppress() +
Optional('(').suppress()
                + Word(variable) + Optional(')').suppress() +
Optional(printables) )

There's too much included in this definition.  I suspect you started with
something simple, then tried to add the leading and trailing
Optional(printables) when your pyvar was embedded in something larger, like
a path.  If we leave pyvar this way, then the parse action will drop the
surrounding data on the floor.  (This COULD be fixed by updating the parse
action, but I'd prefer to keep pyvar to be just about the '%xxx' syntax.)

  pyvar       = Combine( Literal('%').suppress() + Optional('(').suppress()
                + Word(variable) + Optional(')').suppress() )

Your input string is:
 echo /home/%(user)/path/%(prog)/bin/%prog

With your simplified pyvar, you can just use it to do the variable
substitution in a first pass:
  print pyvar.transformString(cmdstring)
Prints out:
 echo /home/mgmiller/path/foo/bin/foo

So if you add:
  cmdstring = pyvar.transformString(cmdstring)
This will do all the symbol substitution, up front.

Now your definition of statement can focus on what your intended command
syntax should be:
  statement = keyword + ZeroOrMore(args)

Which parses as you expect,
  print statement.parseString(cmdstring)
Gives:
  ['echo', '/home/mgmiller/path/foo/bin/foo']


I am a fan of using results names, so I jazzed up your statement definition
to read:

  statement = (
    keyword.setResultsName("command") +
ZeroOrMore(args).setResultsName("args")
    )

Which now allows me to write:
  inputCmd = statement.parseString(cmdstring)
  print inputCmd.command
  print inputCmd.args
And I get:
  echo
  ['/home/mgmiller/path/foo/bin/foo']


Stage 2 required changing around the definition of args, to comprehend that
it migh include some embedded pyvars:

  args        = Combine( OneOrMore( Word(alphanums + '_-=/"\'') | pyvar ) )

This give us a working single pass parser, but I'm not keen on this
'pollution' of the args definition.   This grammar is fairly simple, so it's
not a big deal.  But in general, I think variable or macro substitution
needs to happen in its own transformation pass first - otherwise, the
grammar will need to include lots of "| pyvar" type elements, since you
can't predict where they will show up.  By separating into two passes, the
substitution part is very simple and clean, and the application grammar is
simple and clean.

Please check out the adventure game presentation I gave at the last PyCon -
you can get the source code and see how I implemented a Command pattern to
structure the execution of the parsed commands.

Good luck!

-- Paul




-----Original Message-----
From: pyp...@li...
[mailto:pyp...@li...] On Behalf Of
pyp...@mg...
Sent: Friday, March 24, 2006 3:56 AM
To: pyp...@li...
Subject: [Pyparsing] Combining strings

Hi,

I'm trying to get a string back in one piece and the Combine class
doesn't seem to be working.  I'm not sure how to fix it.  Can anyone help?

Here's the output:
 echo /home/%(user)/apps/%(prog)/bin/%prog
['echo', '/home/', 'mgmiller', '/apps/', 'foo', '/bin/', 'foo']

I'd like to get the path back as one string ... how can I do that? 

Thanks if you can help,
-Mike

----------------------

#!/bin/env python
from pyparsing import *
import string, os
namespace = { 'a': 5, 'user':'mgmiller', 'prog':'foo' }

cmdstring =  r''' echo /home/%(user)/path/%(prog)/bin/%prog '''


def getpyvar(s, loc, match):
    'return a python type variable from our own namespace'
    if namespace.has_key(match[0]):  return namespace[match[0]]


# define grammar
keyword     = oneOf('alias dir echo ver')
args        = Word(alphanums + '_-=/"\'')
variable    = alphas + '_'

# expansions
pyvar       = Combine( Optional(printables) + Literal('%').suppress() +
Optional('(').suppress()
                + Word(variable) + Optional(')').suppress() +
Optional(printables) )
pyvar.setParseAction(getpyvar)


statement = (
    keyword + ZeroOrMore(pyvar | args)
    )

print cmdstring
print statement.parseString(cmdstring)

----------------------

-Mike


 -------------------------------------------------------------
 Mike Miller                  Earth, Sol, Orion Arm, Milky Way




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Pyparsing-users mailing list
Pyp...@li...
https://lists.sourceforge.net/lists/listinfo/pyparsing-users