[Pyparsing] ParseResults as AST?

[Aubrey B. <ba...@cs...>]

Pyparsing users,

I am having trouble figuring out how to make an abstract syntax tree 
(AST) from a ParseResults object. I do not understand the data model 
that is the ParseResults object as it relates to holding the information 
of an AST. (I have spent a whole day on this so far, so it is time to 
ask for help.)

I am writing a Prolog parser, so I really do need an AST as opposed to 
some flattened representation like ParseResults. Order matters. Type 
matters. Depth matters.

Sample Input
------------

This is only Datalog with atoms, integers, and variables.

'''
% Facts
r1(a). r1(b). r1(c).
r2(a, 1). r2(a, 2). r2(d, 3). r2(e, 4).
% Clauses
r3(A, B) :-
     r1(A),
     r2(A, B).
'''

Desired Output
--------------

I would like to get something back from parsing like the following:

('datalog',
[
('comment', '% Facts'),
('fact', [('name', 'r1'), ('body', [('atom', 'a')])]),
('fact', <result>),
...
('clause', [('head', <result>), ('body', [<result>...])]),
]
)

where <result> stands for the parse result from deeper recursion. The 
structure is just lists of pairs. Each pair is the name (type) of the 
parsed object and its content, the result.

Parsing
-------

I believe I have my grammar implemented correctly in pyparsing as I get 
the expected AST back as XML by calling ParseResults.asXML(). (Except 
that I have an extra top-level element, 
"<datalog><datalog>...</datalog></datalog>", but that is easy to 
workaround.)

Tried
-----

I have tried to understand the ParseResults object. Clearly the AST 
information exists as ParseResults.asXML() gives me an AST in XML. 
However, some of the information used in asXML() is not part of the 
public API.

I have tried to use parse actions to construct my own AST but I couldn't 
figure this out either (in part because the product of parsing is still 
a ParseResults object).

Options
-------

* Use asXML() and reparse the XML. Not performant. Too much code. Misses 
the point.
* Write asAST() into pyparsing.py as a near clone of asXML(). This is my 
current favorite option. (I'm on a deadline.)
* Use parse actions effectively.
* Use a different parsing library (I have also tried pyPEG.)
* Other?

Any suggestions and/or help would be very much appreciated.

Sincerely,

Aubrey