pyparsing-users

[Pyparsing] Combine problem

[Sebastian S. <ssc...@gm...>]

Hello there!

This is my first post to this list - first of all thanks for this awesome
module, I could not live without it these days!


My goal is to parse the following structure:
s = """
struct VS_OUTPUT {
    float4 pos        : SV_Position;
    float2 UV    : TEXCOORD0;
    float3 vVec    : TEXCOORD1;
    float3 lVec    : TEXCOORD2;
    float3 nor     : TEXCOORD3;
    float4 col     : TEXCOORD4;
    float3 tan     : TEXCOORD5;
    float4 lpos     : TEXCOORD6;
    float depth     : TEXCOORD7;
    float2 dof     : TEXCOORD8;
    float2 VelocityUV      : TEXCOORD9;
};"""

entryString = "float3 tan     : TEXCOORD5;"

def readStruct(s):
    header = Literal("struct VS_OUTPUT")+"{"

    regs = Combine("TEXCOORD"+Word(nums)) | "SV_Position"
    vType = Combine("float"+Optional(Word(nums)))
    entry = vType+Word(alphanums)+":"+regs+";"
    struct = header + OneOrMore(entry) + "};"

    print entry.parseString(entryString)

This is working as is - but when i change

entry = vType+Word(alphanums)+":"+regs+";"  to entry =
Combine(vType+Word(alphanums)+":"+regs+";")

to make this into a single string, I get a parse Exception - it's not a big
problem but I just don't understand why it doesn't work in this case :)
Somehow I can't see the difference to where I use Combine in the lines
above, where it seems just fine.

Maybe I'm missing something really simple.. thanks for looking!

seb

-- 
Sebastian Schoellhammer

Sr. Technical Artist
Square Enix LTD
www.square-enix.com

Re: [Pyparsing] Combine problem

[Paul M. <pt...@au...>]

This question comes up every so often.  Please see this writeup on the
Discussion page of the pyparsing wiki:
http://pyparsing.wikispaces.com/message/view/home/1089023

In addition to the solution posted there, you could try one of these options
instead of using Combine:

Use a parse action to join the parsed tokens with an intervening ' '
character:

entry = vType+Word(alphanums)+":"+regs+";"
entry.setParseString( lambda tokens: ' '.join(tokens) )

Or use the recent originalTextFor helper method:

entry = originalTextFor( vType+Word(alphanums)+":"+regs+";"  )

Write back if that doesn't answer your question.

(Also, check out the C struct parser on the wiki on the
http://pyparsing.wikispaces.com/UnderDevelopment page.)

-- Paul


> -----Original Message-----
> From: Sebastian Schoellhammer [mailto:ssc...@gm...]
> Sent: Wednesday, June 10, 2009 10:01 PM
> To: pyp...@li...
> Subject: [Pyparsing] Combine problem
> 
> Hello there!
> 
> This is my first post to this list - first of all thanks for this awesome
> module, I could not live without it these days!
> 
> 
> My goal is to parse the following structure:
> s = """
> struct VS_OUTPUT {
>     float4 pos        : SV_Position;
>     float2 UV    : TEXCOORD0;
>     float3 vVec    : TEXCOORD1;
>     float3 lVec    : TEXCOORD2;
>     float3 nor     : TEXCOORD3;
>     float4 col     : TEXCOORD4;
>     float3 tan     : TEXCOORD5;
>     float4 lpos     : TEXCOORD6;
>     float depth     : TEXCOORD7;
>     float2 dof     : TEXCOORD8;
>     float2 VelocityUV      : TEXCOORD9;
> };"""
> 
> entryString = "float3 tan     : TEXCOORD5;"
> 
> def readStruct(s):
>     header = Literal("struct VS_OUTPUT")+"{"
> 
>     regs = Combine("TEXCOORD"+Word(nums)) | "SV_Position"
>     vType = Combine("float"+Optional(Word(nums)))
>     entry = vType+Word(alphanums)+":"+regs+";"
>     struct = header + OneOrMore(entry) + "};"
> 
>     print entry.parseString(entryString)
> 
> This is working as is - but when i change
> 
> entry = vType+Word(alphanums)+":"+regs+";"  to entry =
> Combine(vType+Word(alphanums)+":"+regs+";")
> 
> to make this into a single string, I get a parse Exception - it's not a
> big
> problem but I just don't understand why it doesn't work in this case :)
> Somehow I can't see the difference to where I use Combine in the lines
> above, where it seems just fine.
> 
> Maybe I'm missing something really simple.. thanks for looking!
> 
> seb
> 
> --
> Sebastian Schoellhammer
> 
> Sr. Technical Artist
> Square Enix LTD
> www.square-enix.com
> --------------------------------------------------------------------------
> ----
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Pyparsing-users mailing list
> Pyp...@li...
> https://lists.sourceforge.net/lists/listinfo/pyparsing-users

Re: [Pyparsing] Combine problem

[spir <den...@fr...>]

Le Thu, 11 Jun 2009 12:00:32 +0900,
Sebastian Schoellhammer <ssc...@gm...> s'exprima ainsi:

> Hello there!
> 
> This is my first post to this list - first of all thanks for this awesome
> module, I could not live without it these days!
> 
> 
> My goal is to parse the following structure:
> s = """
> struct VS_OUTPUT {
>     float4 pos        : SV_Position;
>     float2 UV    : TEXCOORD0;
>     float3 vVec    : TEXCOORD1;
>     float3 lVec    : TEXCOORD2;
>     float3 nor     : TEXCOORD3;
>     float4 col     : TEXCOORD4;
>     float3 tan     : TEXCOORD5;
>     float4 lpos     : TEXCOORD6;
>     float depth     : TEXCOORD7;
>     float2 dof     : TEXCOORD8;
>     float2 VelocityUV      : TEXCOORD9;
> };"""
> 
> entryString = "float3 tan     : TEXCOORD5;"
> 
> def readStruct(s):
>     header = Literal("struct VS_OUTPUT")+"{"
> 
>     regs = Combine("TEXCOORD"+Word(nums)) | "SV_Position"
>     vType = Combine("float"+Optional(Word(nums)))
>     entry = vType+Word(alphanums)+":"+regs+";"
>     struct = header + OneOrMore(entry) + "};"
> 
>     print entry.parseString(entryString)
> 
> This is working as is - but when i change
> 
> entry = vType+Word(alphanums)+":"+regs+";"  to entry =
> Combine(vType+Word(alphanums)+":"+regs+";")
> 
> to make this into a single string, I get a parse Exception - it's not a big
> problem but I just don't understand why it doesn't work in this case :)
> Somehow I can't see the difference to where I use Combine in the lines
> above, where it seems just fine.

You should always post error messages to get help people help you. In this case:
  [.............]
  File "/usr/lib/python2.5/site-packages/pyparsing-1.5.1-py2.5.egg/pyparsing.py", line 1657, in parseImpl
    raise exc
pyparsing.ParseException: Expected W:(abcd...) (at char 6), (line:1, col:7)

In other words: pyparsing steps on the space, expecting the second part of the pattern.

If I remember well, Combine() invalidates the default "separator-skipping" mode of pyparsing (someone confirms?). Which is rather sensible, for otherwise it would join together bit that are separated in the source, which is probably not what we want in most cases.

If you wish this anyway, just join the result manually:
	result = entry.parseString(entryString)
 	print "".join(result)
or rather set a post-parse action on entry:
	entry.setParseAction(lambda node: ''.join(node))
==>
['float3tan:TEXCOORD5;']

Is this really what you expect?

Denis
------
la vita e estrany

Re: [Pyparsing] Combine problem

[Sebastian S. <ssc...@gm...>]

Thanks a lot for the quick replies!!

I didn't know about the , adjacent=False) argument for Combine.

['float3tan:TEXCOORD5;'] wasn't really what I expected but I was simply
confused by the exception - thanks for the explanations!

Nice work in the  "Under development" section too :)


All the best,

seb



On Thu, Jun 11, 2009 at 1:09 PM, spir <den...@fr...> wrote:

> Le Thu, 11 Jun 2009 12:00:32 +0900,
> Sebastian Schoellhammer <ssc...@gm...> s'exprima ainsi:
>
> > Hello there!
> >
> > This is my first post to this list - first of all thanks for this awesome
> > module, I could not live without it these days!
> >
> >
> > My goal is to parse the following structure:
> > s = """
> > struct VS_OUTPUT {
> >     float4 pos        : SV_Position;
> >     float2 UV    : TEXCOORD0;
> >     float3 vVec    : TEXCOORD1;
> >     float3 lVec    : TEXCOORD2;
> >     float3 nor     : TEXCOORD3;
> >     float4 col     : TEXCOORD4;
> >     float3 tan     : TEXCOORD5;
> >     float4 lpos     : TEXCOORD6;
> >     float depth     : TEXCOORD7;
> >     float2 dof     : TEXCOORD8;
> >     float2 VelocityUV      : TEXCOORD9;
> > };"""
> >
> > entryString = "float3 tan     : TEXCOORD5;"
> >
> > def readStruct(s):
> >     header = Literal("struct VS_OUTPUT")+"{"
> >
> >     regs = Combine("TEXCOORD"+Word(nums)) | "SV_Position"
> >     vType = Combine("float"+Optional(Word(nums)))
> >     entry = vType+Word(alphanums)+":"+regs+";"
> >     struct = header + OneOrMore(entry) + "};"
> >
> >     print entry.parseString(entryString)
> >
> > This is working as is - but when i change
> >
> > entry = vType+Word(alphanums)+":"+regs+";"  to entry =
> > Combine(vType+Word(alphanums)+":"+regs+";")
> >
> > to make this into a single string, I get a parse Exception - it's not a
> big
> > problem but I just don't understand why it doesn't work in this case :)
> > Somehow I can't see the difference to where I use Combine in the lines
> > above, where it seems just fine.
>
> You should always post error messages to get help people help you. In this
> case:
>  [.............]
>  File
> "/usr/lib/python2.5/site-packages/pyparsing-1.5.1-py2.5.egg/pyparsing.py",
> line 1657, in parseImpl
>    raise exc
> pyparsing.ParseException: Expected W:(abcd...) (at char 6), (line:1, col:7)
>
> In other words: pyparsing steps on the space, expecting the second part of
> the pattern.
>
> If I remember well, Combine() invalidates the default "separator-skipping"
> mode of pyparsing (someone confirms?). Which is rather sensible, for
> otherwise it would join together bit that are separated in the source, which
> is probably not what we want in most cases.
>
> If you wish this anyway, just join the result manually:
>        result = entry.parseString(entryString)
>        print "".join(result)
> or rather set a post-parse action on entry:
>        entry.setParseAction(lambda node: ''.join(node))
> ==>
> ['float3tan:TEXCOORD5;']
>
> Is this really what you expect?
>
> Denis
> ------
> la vita e estrany
>
>
> ------------------------------------------------------------------------------
> Crystal Reports - New Free Runtime and 30 Day Trial
> Check out the new simplified licensing option that enables unlimited
> royalty-free distribution of the report engine for externally facing
> server and web deployment.
> http://p.sf.net/sfu/businessobjects
> _______________________________________________
> Pyparsing-users mailing list
> Pyp...@li...
> https://lists.sourceforge.net/lists/listinfo/pyparsing-users
>



-- 
Sebastian Schoellhammer

Sr. Technical Artist
Square Enix LTD
www.square-enix.com