RE: [Pyparsing] RE: python indentation grammar

[Paul M. <pa...@al...>]

Michel -

This part of pyparsing is not well-documented at all, since I typically
discourage people from writing whitespace-sensitive parsers.  Very often,
people come from writing regexp's and try to figure out how to explicitly
handle whitespace between tokens, and I have to explain that pyparsing
doesn't require explicit whitespace handling, that whitespace is assumed to
be a token delimiter, but that the whitespace itself is skipped/ignored by
default.

However, your grammar is *by its nature* whitespace-sensitive.  So you
probably need to call the leaveWhitespace() method on your root parse
object, self.block, as in:

        self.block.leaveWhitespace()

Do this right at the end of your initGrammar method, after assigning
self.block:

        self.block = ZeroOrMore( ...etc, etc. )
        self.block.leaveWhitespace()

This will recursively set whitespace handling through the whole bnf, not
just for the root node, and your whitespace handling should be more
predictable.  After adding this line, I reran your test, and I think I got
all the between-tag whitespace you were looking for.

I'm also glad asXML() seems to be working adequately for you.  As I
mentioned before, this method is a bit iffy, so it is fortunate that you are
getting such good results.

Congratulations on such a sophisticated parsing application!  

-- Paul