Search Results for "lexer"

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library. This is the source code repository for RE2, a regular expression library. There is a fair amount of documentation (including code snippets) in the re2.h header file. Unless otherwise noted, the RE2 source files are distributed under the BSD-style license found in the LICENSE file. In POSIX mode, RE2 accepts standard POSIX (egrep) syntax...

A C++ high-performance regex library and Flex-compatible lexical analyzer generator with full Unicode support, new indentation anchors, lazy quantifiers, and many other modern features. Accepts Flex lexer specification syntax and is compatible with Bison/Yacc parsers. Generates reusable source code that is easy to understand. Supports fast scanning of UTF-8/16/32 files, strings, and streams. The reflex scanner generator generates clean C++ lexer class code that is thread-safe. Generates Graphviz files to visualize state machine DFAs. ...

...The Lexer is responsible for giving you an API to walk across a string one character at a time and analyze the type of each character, value and position of each token in the string. The low level API of the lexer is pretty simple. The setInput($input) option sets the input data to be tokenized. The Lexer is immediately reset and the new input tokenized.

javalang is a pure Python library for working with Java source code. javalang provides a lexer and parser targeting Java 8. The implementation is based on the Java language spec.

Simple header-only library written in C++, for lexical analysis of files in PDF format