Search Results for "token generator"

The SoulNG project contains two tools: a lexical analyzer generator slg and a parser generator spg. The generators and the produced lexical analyzers and parsers use four C++ libraries that are also included in the project. The slg tool takes a .lexer file that contains the description of a lexical analyzer as input and produces C++ source code for a lexical analyzer as output. The produced lexical analyzer is a finite state machine that recognizes patterns described as regular expressions and return corresponding token identifiers to the parser. ...

ULS is a class library for creating lexical analyzer from language specification file. It's provided as C/C++/C# and Java (class) libraries with a few of tools on Linux/Windows platform.

The goal of this project is to provide a generator for lexical analyzers of maximum computational efficiency and maximum range of applications. This includes the support for Unicode (UTF8, UTF16, ...) and a large variety of other encodings directly and via nested converters such as ICU(tm) and IConv. Sophisticated buffer handling allows to operate on plain file streams, on sockets, or manually fed buffer content. 'Ready-to-build' examples explain related concepts and facilitate practical...

...When the lexer identifies a token that has a named event associated with it, it passes the token to the parser method associated with that event. A lexer can dispatch token events to any number of parsers. JPlex is steadily being improved, and the Subversion repo will almost always be newer and better than the binary download.