The order of state tokens in the token trees returned
as part of a recognition result can vary depending upon
the topology of the search space as generated by the
various linguist. In particular, the position of the
word state (at the beginning of the word for a flat
linguist, at the end of the word for a lex tree
linguist) is irregular. This irregular order can make
post processing of the results somewhat difficult.
Normalizing the token lists so that the token order is
identical regardless of the linguist used will reduce
the complexity involved in post processing results.
Log in to post a comment.