Doctrine Lexer
Base library for a lexer that can be used in Recursive Descent Parsers
...These methods define the lexical catchable and non-catchable patterns and a method for returning the type of a token and filtering the value if necessary. The Lexer is responsible for giving you an API to walk across a string one character at a time and analyze the type of each character, value and position of each token in the string. The low level API of the lexer is pretty simple. The setInput($input) option sets the input data to be tokenized. The Lexer is immediately reset and the new input tokenized. And the option glimpse(), peeks at the next token, returns it and immediately resets the peek.