Search Results for "treebank"

...It is intended for use in the preparation of annotated linguistic corpora, where differences in tokenization may arise (i) following corrections or modifications to the source text or (ii) through the creation of different layers of annotation (part-of-speech, treebank) requiring different tokenization. In its default implementation, it produces a human-readable CSV table associating tokens in text A with tokens in text B, and can also inject token-level annotation from text B to text A. The Aligner class on which the default implementation is based can be incorporated into more complex workflows.

This is a Python package designed to process Penn Treebank Release II-style combined trees (.mrg files) into useful objects for tree traversal, feature extraction, and statistical analysis. For more information, go to http://mrgutils.sourceforge.net