pyparsing-users

[Pyparsing] Parsing multiple items into the same Dict

[George P. <ge...@ga...>]

Hi all,
    Given the following:

s = \
"""
a.clk  = "----____----____"
a.rst  = "--------________"

b.clk  = "____----____----"
b.x    = "________--------"
"""

TIMING_SPEC = OneOrMore(Regex('[-_]'))
GROUPNAME   = Word(alphanums) + Suppress('.')
SIGNAME     = Word(alphanums) + Suppress('=')

GRAMMAR     = ???

# --------

What would the grammar have to look like so I can get the
following kind of parse results?

In particular, I'm stuck on how to parse multiple items into
the same dict. In this case, I want to have a dict containing
GROUPNAMEs as keys. Each GROUPNAME dict contains one or more
'SIGNAME':TIMINGSPEC key/value pairs:

>>>d = GRAMMAR.parseString(s).asDict()

>>>d['a'].keys()
['clk', 'rst']

>>>d['b'].keys()
['clk', 'x']

>>>d['a']['clk']
----____----____

>>>d['a']['rst']
--------________

>>>d['b']['clk']
____----____----

>>>d['b']['x']
________--------

Thanks,
George

Re: [Pyparsing] Parsing multiple items into the same Dict

[Paul M. <pa...@al...>]

> -----Original Message-----
> From: pyp...@li... 
> [mailto:pyp...@li...] On 
> Behalf Of George Pantazopoulos
> Sent: Monday, October 23, 2006 11:31 PM
> To: pyp...@li...
> Subject: [Pyparsing] Parsing multiple items into the same Dict
> 
> 
> Hi all,
>     Given the following:
> 
> s = \
> """
> a.clk  = "----____----____"
> a.rst  = "--------________"
> 
> b.clk  = "____----____----"
> b.x    = "________--------"
> """
> 
> TIMING_SPEC = OneOrMore(Regex('[-_]'))
> GROUPNAME   = Word(alphanums) + Suppress('.')
> SIGNAME     = Word(alphanums) + Suppress('=')
> 
> GRAMMAR     = ???
> 
> # --------
> 
> What would the grammar have to look like so I can get the 
> following kind of parse results?
> 

George -

Is it really necessary to create the dict directly in the ParseResults?
This could probably be done in some tricky parse actions, but things might
be simpler if your ParseResults just reflect the expression structure, and
then walk the structure to build up your nested dict.  Below is one way.

-- Paul


from pyparsing import *

s = \
"""
a.clk  = "----____----____"
a.rst  = "--------________"

b.clk  = "____----____----"
b.x    = "________--------"
"""

TIMING_SPEC = Combine('"' + Word('_-') + '"' )
ident = Word(alphas, alphanums) 

SIGNAL_DEF = ident.setResultsName("GROUP") + \
           '.' + ident.setResultsName("SIGNAME") + \
           '=' + \
           TIMING_SPEC.setResultsName("TIMING_SPEC")

GRAMMAR = OneOrMore( Group(SIGNAL_DEF) )

results = GRAMMAR.parseString(s)

d = {}
for toks in results:
    if toks.GROUP not in d:
        d[toks.GROUP] = {}
    d[toks.GROUP][toks.SIGNAME] = toks.TIMING_SPEC

print d
print d.keys()
for k in d.keys():
    print k, d[k].keys()


Prints out:
{'a': {'rst': '"--------________"', 'clk': '"----____----____"'}, 'b': {'x':
'"________--------"', 'clk': '"____----____----"'}}
['a', 'b']
a ['rst', 'clk']
b ['x', 'clk']
d['a']['rst'] -> "--------________"
d['a']['clk'] -> "----____----____"
d['b']['x'] -> "________--------"
d['b']['clk'] -> "____----____----"

-- Paul

Re: [Pyparsing] Parsing multiple items into the same Dict

[George P. <ge...@ga...>]

> George -
>
> Is it really necessary to create the dict directly in the ParseResults?
> This could probably be done in some tricky parse actions, but things might
> be simpler if your ParseResults just reflect the expression structure, and
> then walk the structure to build up your nested dict.  Below is one way.
>
> -- Paul
>
>
> from pyparsing import *
>
> s = \
> """
> a.clk  = "----____----____"
> a.rst  = "--------________"
>
> b.clk  = "____----____----"
> b.x    = "________--------"
> """
>
> TIMING_SPEC = Combine('"' + Word('_-') + '"' )
> ident = Word(alphas, alphanums) 
>
> SIGNAL_DEF = ident.setResultsName("GROUP") + \
>            '.' + ident.setResultsName("SIGNAME") + \
>            '=' + \
>            TIMING_SPEC.setResultsName("TIMING_SPEC")
>
> GRAMMAR = OneOrMore( Group(SIGNAL_DEF) )
>
> results = GRAMMAR.parseString(s)
>
> d = {}
> for toks in results:
>     if toks.GROUP not in d:
>         d[toks.GROUP] = {}
>     d[toks.GROUP][toks.SIGNAME] = toks.TIMING_SPEC
>
> print d
> print d.keys()
> for k in d.keys():
>     print k, d[k].keys()
>
>
> Prints out:
> {'a': {'rst': '"--------________"', 'clk': '"----____----____"'}, 'b': {'x':
> '"________--------"', 'clk': '"____----____----"'}}
> ['a', 'b']
> a ['rst', 'clk']
> b ['x', 'clk']
> d['a']['rst'] -> "--------________"
> d['a']['clk'] -> "----____----____"
> d['b']['x'] -> "________--------"
> d['b']['clk'] -> "____----____----"
>
>   
Paul,
    Thanks for your fantastic help! Your detailed example taught me a 
lot. No, I didn't need to create the dict in the parse results directly. 
I'm glad you questioned that because I was starting to go insane ;-)

Here's an excerpt of the solution you helped guide me to. (Note: I 
accidentally used quotes in my timing spec string in the previous email).

If you're curious what I'm doing this for, I'm making a tool for MyHDL 
to increase the fun factor in the design and testing of complex 
electronic systems: 
http://myhdl.jandecaluwe.com/doku.php/users:george_pantazopoulos

Kudos,
George
http://www.gammaburst.net

# ---------------------
from pyparsing import *

def parse_ats_block(ats_block):

    """
    Example input:

    a.clk  = ----____----____
    a.rst  = --------________

    b.clk  = ____----____----
    b.x    = ________--------

    For the Timing Diagram portion:
   
    Input        Parser output            
    ---------------------------------------
    "--__"   =>  ['-', '-', '_', '_']

    "1..72." =>  ['1..', '72.']      
   
    """

    # pyparsing grammar
    # Special thanks to Paul McGuire

    # The OneOrMore() will split up the timing spec into a list of tokens
    TIMING_SPEC = OneOrMore(Regex('[-_]') | Regex('[0-9]+\.+'))
   
    identifier  = Word(alphas, alphanums)
       
    SIGNAL_DEF  = identifier.setResultsName("GROUP") + \
                  '.' + identifier.setResultsName("SIGNAME") + \
                  '=' + TIMING_SPEC.setResultsName("TIMING_SPEC")

    ATS_GRAMMAR = OneOrMore(Group(SIGNAL_DEF))


    # The parse results can be treated as a list of lists of the following
    # form: (Each sublist represents a signal definition)
    # [groupname, '.', signame, '=', timingspec]

    # In addition, thanks to the use of setResultsName()
    # the components can also be accessed using attribute syntax
   
    # Each signal definition entry in the results will contain
    # a .GROUP, .SIGNAME, and .TIMING_SPEC attribute.
   
    # Parse
    results = ATS_GRAMMAR.parseString(ats_block)
   
    # From the parse results create a nested dict.
    # The top level dict contains a dict for each group, keyed by the
    # group name.
    #
    # Each group dict contains key/value pairs where key = signal name
    # and value = the signal's timing spec
   
    d = dict()
    for toks in results:
        if toks.GROUP not in d:
            d[toks.GROUP] = dict()
        d[toks.GROUP][toks.SIGNAME] = toks.TIMING_SPEC
       
    results_dict = d

    return results_dict