#3 Use of Generators is not thread safe

[Robert Brewer]

An instance of generator.Generator is not safe to use in multiple threads simultaneously, because its buildParser method mutates its parserList state. This in itself is not a problem, but simpleparsegrammar.Parser.buildTagger and objectgenerator.LibraryElement.toParser re-use the same generators even among separate parser.Parser instances.

I'm no expert in simpleparse, but my testing shows that this can be remedied with two small changes: first, for simpleparsegrammar.Parser.buildTagger, copy the SPGenerator object instead of using it directly:

def buildTagger( self, name=None, processor = None ):
"""Build the tag-table for parsing the EBNF for this parser"""
gen = generator.Generator()
gen.names = SPGenerator.names[:]
gen.rootObjects = SPGenerator.rootObjects[:]
gen.methodSource = SPGenerator.methodSource
gen.definitionSources = SPGenerator.definitionSources[:]

return gen.buildParser( name, processor )

Second, for objectgenerator.LibraryElement.toParser, make a new Parser instance for each LibraryElement() instance created in common/*.py, instead of re-using a common "_p" variable. For example, in numbers.py:

for name in ["int","hex", "int_unsigned", "number", "float", "binary_number", "float_floatexp", "imaginary_number", "number_full"]:
c[ name ] = objectgenerator.LibraryElement(
generator = Parser( declaration )._generator,
production = name,
)

The project is a bit complicated for me to know if this preserves semantics, but it works in my tests.

[Robert Brewer]

Hmm. Perhaps it doesn't work so well in my tests. A better approach might be to create a Builder object instead of mutating the state of the Generator itself.

[Robert Brewer]

Indeed. Creating a new Builder object on each call to buildParser works. Working code in a fork at https://bitbucket.org/fumanchu/simpleparse