Thread: [Pyparsing] Returning an actual `list`
Brought to you by:
ptmcg
From: Athanasios A. <ath...@gm...> - 2017-10-04 10:44:35
|
Hello I have set up a very simple "primitive data type" parsing system to parse numbers, quoted strings, lists and dictionaries. The last two are following Python's convention. While this is working, I am having trouble returning specific data types from the "action" of the LIST definition. Here is a snippet: LIST = pyparsing.Forward() DICT = pyparsing.Forward() VALUE = (NUMBER|IDENTIFIER|DICT|pyparsing.Group(LIST)) KEY_VALUE_PAIR = pyparsing.Group(IDENTIFIER("key") + pyparsing.Suppress(":") + VALUE("value")) LIST << pyparsing.Suppress("[") + pyparsing.delimitedList(VALUE) + pyparsing.Suppress("]") DICT << pyparsing.Suppress("{") + pyparsing.delimitedList(KEY_VALUE_PAIR) + pyparsing.Suppress("}") So, when you are trying to parse something like: [1,2,3,4,[5,6]], this is returned as a pyparsing.ParseResults type of object, rather than a `list`. I have tried to set a simple parseAction with `lambda s,l,t: list(t)` or `lambda s,l,t:list(t[0])` but I am still getting back a ParseResults object. Ideally, I would like the rule to return a list, just like the INT rule returns a proper Python integer. Any ideas about what am I missing? All the best AA |
From: Athanasios A. <ath...@gm...> - 2017-10-20 10:08:03
|
Hello everyone Any ideas on the attached? All the best AA ---------- Forwarded message ---------- From: Athanasios Anastasiou <ath...@gm...> Date: Wed, Oct 4, 2017 at 11:44 AM Subject: Returning an actual `list` To: pyp...@li... Hello I have set up a very simple "primitive data type" parsing system to parse numbers, quoted strings, lists and dictionaries. The last two are following Python's convention. While this is working, I am having trouble returning specific data types from the "action" of the LIST definition. Here is a snippet: LIST = pyparsing.Forward() DICT = pyparsing.Forward() VALUE = (NUMBER|IDENTIFIER|DICT|pyparsing.Group(LIST)) KEY_VALUE_PAIR = pyparsing.Group(IDENTIFIER("key") + pyparsing.Suppress(":") + VALUE("value")) LIST << pyparsing.Suppress("[") + pyparsing.delimitedList(VALUE) + pyparsing.Suppress("]") DICT << pyparsing.Suppress("{") + pyparsing.delimitedList(KEY_VALUE_PAIR) + pyparsing.Suppress("}") So, when you are trying to parse something like: [1,2,3,4,[5,6]], this is returned as a pyparsing.ParseResults type of object, rather than a `list`. I have tried to set a simple parseAction with `lambda s,l,t: list(t)` or `lambda s,l,t:list(t[0])` but I am still getting back a ParseResults object. Ideally, I would like the rule to return a list, just like the INT rule returns a proper Python integer. Any ideas about what am I missing? All the best AA |
From: <pt...@au...> - 2017-10-20 12:41:04
|
I looked at this briefly last week, I thought I had a working example of this in the parsePythonValue.py example, but it seems to have the same problem. I thought I had worked out a working version at one time, using ungroup, but have not succeeded with that. Here is an example of an expression and parse action that creates a list. Bear in mind that the return value from parseString is *always* a ParseResults, even if it is just `pp.pyparsing_common.integer.parseString("123")`. Without the parse action, the parsed list is a ParseResults returned as the 0'th element of a ParseResults: import pyparsing as pp LBRACK, RBRACK = map(pp.Suppress, "[]") item = pp.Word(pp.alphas) list_expr = LBRACK + pp.delimitedList(item) + RBRACK ret = pp.Group(list_expr).parseString("[a,b,c,d]") print(ret) print(type(ret)) print(ret[0]) print(type(ret[0])) Gives: [['a', 'b', 'c', 'd']] <class 'pyparsing.ParseResults'> ['a', 'b', 'c', 'd'] <class 'pyparsing.ParseResults'> Adding this parse action, we get a list returned as the 0'th element of a ParseResults. Note *no* Group around the list expression: def make_list(tokens): contents = tokens.asList() tokens[:] = (contents,) list_expr.addParseAction(make_list) ret = list_expr.parseString("[a,b,c,d]") print(ret) print(type(ret)) print(ret[0]) print(type(ret[0])) Gives: [['a', 'b', 'c', 'd']] <class 'pyparsing.ParseResults'> ['a', 'b', 'c', 'd'] <class 'list'> Does that get you closer? -- Paul ---- Athanasios Anastasiou <ath...@gm...> wrote: > Hello everyone > > Any ideas on the attached? > > All the best > AA > > > ---------- Forwarded message ---------- > From: Athanasios Anastasiou <ath...@gm...> > Date: Wed, Oct 4, 2017 at 11:44 AM > Subject: Returning an actual `list` > To: pyp...@li... > > > Hello > > I have set up a very simple "primitive data type" parsing system to parse > numbers, quoted strings, lists and dictionaries. The last two are following > Python's convention. > > While this is working, I am having trouble returning specific data types > from the "action" of the LIST definition. > > Here is a snippet: > > LIST = pyparsing.Forward() > DICT = pyparsing.Forward() > > VALUE = (NUMBER|IDENTIFIER|DICT|pyparsing.Group(LIST)) > > KEY_VALUE_PAIR = pyparsing.Group(IDENTIFIER("key") + > pyparsing.Suppress(":") + VALUE("value")) > > LIST << pyparsing.Suppress("[") + pyparsing.delimitedList(VALUE) + > pyparsing.Suppress("]") > DICT << pyparsing.Suppress("{") + pyparsing.delimitedList(KEY_VALUE_PAIR) + > pyparsing.Suppress("}") > > So, when you are trying to parse something like: [1,2,3,4,[5,6]], this is > returned as a pyparsing.ParseResults type of object, rather than a `list`. > I have tried to set a simple parseAction with `lambda s,l,t: list(t)` or > `lambda s,l,t:list(t[0])` but I am still getting back a ParseResults object. > > Ideally, I would like the rule to return a list, just like the INT rule > returns a proper Python integer. > > Any ideas about what am I missing? > > All the best > AA > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > Pyparsing-users mailing list > Pyp...@li... > https://lists.sourceforge.net/lists/listinfo/pyparsing-users |
From: Athanasios A. <ath...@gm...> - 2017-11-03 11:37:17
|
Hello Paul Thank you very much for your response. I am sorry about the delay in replying, my attention was diverted to a report that required my input in the last few weeks but I am back into this now. I think that this is what I may be missing: > Bear in mind that the return value from parseString is *always* a ParseResults, even if it is just `pp.pyparsing_common.integer. parseString("123")`. My Integer class has a parseAction that returns an integer and that is that. The integer does not identify itself as a ParseResults which BTW has only one field of type integer. When I am trying to do the same with my Dict and List classes, I always get back a ParseResults. Now, knowing that this will always be a ParseResults which can be interpreted as whatever is one thing. What I would "ideally" expect is for the item that is returned from the parseAction to be emitted as is. The second example is closer to what I am after, but there does not seem to be a "return" on make_list (?). Is a return implied here? All the best Athanasios Anastasiou On Fri, Oct 20, 2017 at 1:24 PM, <pt...@au...> wrote: > I looked at this briefly last week, I thought I had a working example of > this in the parsePythonValue.py example, but it seems to have the same > problem. > > I thought I had worked out a working version at one time, using ungroup, > but have not succeeded with that. > > Here is an example of an expression and parse action that creates a list. > Bear in mind that the return value from parseString is *always* a > ParseResults, even if it is just `pp.pyparsing_common.integer. > parseString("123")`. > > Without the parse action, the parsed list is a ParseResults returned as > the 0'th element of a ParseResults: > > import pyparsing as pp > > LBRACK, RBRACK = map(pp.Suppress, "[]") > > item = pp.Word(pp.alphas) > > list_expr = LBRACK + pp.delimitedList(item) + RBRACK > > ret = pp.Group(list_expr).parseString("[a,b,c,d]") > print(ret) > print(type(ret)) > print(ret[0]) > print(type(ret[0])) > > Gives: > [['a', 'b', 'c', 'd']] > <class 'pyparsing.ParseResults'> > ['a', 'b', 'c', 'd'] > <class 'pyparsing.ParseResults'> > > > > Adding this parse action, we get a list returned as the 0'th element of a > ParseResults. Note *no* Group around the list expression: > > def make_list(tokens): > contents = tokens.asList() > tokens[:] = (contents,) > > list_expr.addParseAction(make_list) > ret = list_expr.parseString("[a,b,c,d]") > print(ret) > print(type(ret)) > print(ret[0]) > print(type(ret[0])) > > Gives: > > [['a', 'b', 'c', 'd']] > <class 'pyparsing.ParseResults'> > ['a', 'b', 'c', 'd'] > <class 'list'> > > Does that get you closer? > > -- Paul > > ---- Athanasios Anastasiou <ath...@gm...> wrote: > > Hello everyone > > > > Any ideas on the attached? > > > > All the best > > AA > > > > > > ---------- Forwarded message ---------- > > From: Athanasios Anastasiou <ath...@gm...> > > Date: Wed, Oct 4, 2017 at 11:44 AM > > Subject: Returning an actual `list` > > To: pyp...@li... > > > > > > Hello > > > > I have set up a very simple "primitive data type" parsing system to parse > > numbers, quoted strings, lists and dictionaries. The last two are > following > > Python's convention. > > > > While this is working, I am having trouble returning specific data types > > from the "action" of the LIST definition. > > > > Here is a snippet: > > > > LIST = pyparsing.Forward() > > DICT = pyparsing.Forward() > > > > VALUE = (NUMBER|IDENTIFIER|DICT|pyparsing.Group(LIST)) > > > > KEY_VALUE_PAIR = pyparsing.Group(IDENTIFIER("key") + > > pyparsing.Suppress(":") + VALUE("value")) > > > > LIST << pyparsing.Suppress("[") + pyparsing.delimitedList(VALUE) + > > pyparsing.Suppress("]") > > DICT << pyparsing.Suppress("{") + pyparsing.delimitedList(KEY_VALUE_PAIR) > + > > pyparsing.Suppress("}") > > > > So, when you are trying to parse something like: [1,2,3,4,[5,6]], this is > > returned as a pyparsing.ParseResults type of object, rather than a > `list`. > > I have tried to set a simple parseAction with `lambda s,l,t: list(t)` or > > `lambda s,l,t:list(t[0])` but I am still getting back a ParseResults > object. > > > > Ideally, I would like the rule to return a list, just like the INT rule > > returns a proper Python integer. > > > > Any ideas about what am I missing? > > > > All the best > > AA > > ------------------------------------------------------------ > ------------------ > > Check out the vibrant tech community on one of the world's most > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > _______________________________________________ > > Pyparsing-users mailing list > > Pyp...@li... > > https://lists.sourceforge.net/lists/listinfo/pyparsing-users > > |
From: <pt...@au...> - 2017-11-03 12:30:11
|
---- Athanasios Anastasiou <ath...@gm...> wrote: > The second example is closer to what I am after, but there does not seem to > be a "return" on make_list (?). Is a return implied here? > If None is returned (which is what Python does as a default if no return statement is present), then pyparsing will retain the tokens passed in as the parsed data. Otherwise, pyparsing will replace the parsed data with whatever is returned by the parse action. (If you really want to return None, return [None]). This makes it easy to write parse actions that update the parsed tokens in place, such as when adding new named results. Here's an example that parses a hex number of the form "#xxxxxx" which could represent an RGB color tuple: hexnumber = pp.Word('#', pp.hexnums) hexnumber.addParseAction(lambda t: int(t[0][1:], 16)) color = "#7fff00" print(hexnumber.parseString(color).dump()) And it prints: [8388352] If we add a second parse action (which has no return statement, and thus returns None): def get_rgb(t): # update parsed tokens in-place, adding values for red, green, and blue int_value = t[0] int_value, t['b'] = divmod(int_value, 256) int_value, t['g'] = divmod(int_value, 256) int_value, t['r'] = divmod(int_value, 256) hexnumber.addParseAction(get_rgb) print(hexnumber.parseString(color).dump()) The parse tokens are updated in-place and we get: [8388352] - b: 0 - g: 255 - r: 127 (Note that you must use the dict-style access in the body of the parse action in order to set new results names.) -- Paul |