Search Results for "lexer"

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It's widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build and walk parse trees. It’s widely used in academia and industry to build all sorts of languages, tools, and frameworks. Twitter search uses ANTLR for query parsing, with over 2 billion queries a day. The languages for...

This project has moved to GitHub ! The version here at SourceForge will remain for historic purpose. Koopa is a parser generator, made for COBOL. It can handle source files in isolation (no preprocessing required) and doesn't mind the presence of CICS/SQL fragments. The grammar is easily extensible in a way which minimizes the impact on the overall code.

Provides basic support to Velocity's *.vm and *.vsl files. Syntax coloring, basic error highlighting and braces matching is achieved with a lexer and parser based on Apache Velocity's 1.6.2 specification and compiled with JavaCC 5.0.

JFlex is a flex-like lexer generator for Java with emphasis on speed and full Unicode support. It has some not so usual features like negation in regexps and nested input streams. Also reads JLex specifications unchanged.

Coco/S is a compiler generator that takes plain EBNF grammar files and features a SAX style call back API. It is written in Java and produces a Java Scanner (Lexer) and Parser for the language described by the input grammar. Coco/S is a branch of the 2010/11 release of Coco/R for Java. While Coco/R takes an attributed grammar file, Coco/S has a callback API. This has the advantage of a clean separation between grammar (EBNF) and Compiler/Interpreter.

PHParser 1.2 generates a pure Java parser for PHP programs. Invoking this parser yields an explicit parse tree (AST) and a tree walker suitable for further analysis. This tool package is based upon: - ANTLR 3.2 or higher (www.antlr.org). - JDK 1.6 or higher (java.sun.com). - Grammar specifications of PHP 5.3.

...I have not given much thought to whether it would even make sense to preserve the 'info' field through some of the other operations. This project also includes an extension of RunAutomaton, called TokenAutomaton, that can be used as a lexer, as well as its support classes and examples of its use. Lexers generated by JPlex depend on this library. I separated it from JPlex so that projects using JPlex would not depend on GPL code. Revision 1 of this project is dual-licensed, as it is the same code that was formerly part of JPlex. All code in this library was either originally published under the BSD license or was exclusively authored by me.

...Their state machines are precompiled and serialized, so run time initialization is fast no matter how large the rule set. JPlex is not a parser generator; it only generates an interface for its parsers, and it does not build a parse tree. When the lexer identifies a token that has a named event associated with it, it passes the token to the parser method associated with that event. A lexer can dispatch token events to any number of parsers. JPlex is steadily being improved, and the Subversion repo will almost always be newer and better than the binary download.

KFlex is an extension to JFlex that makes it easy to generate lexers whose actions call methods of a listener interface. This simplifies parser development and makes the lexer a reusable drop-in component.

Java Cobol Lexer take a cobol source program and return it as a list of lexical tokens. Library takes source code in fixed, free and mixed formats. It recognizes standard Cobol 85.

Annotation based lexer and parser generator suitable for processing wide range of languages, from small domain specific languages to big ones like Java. Actions may be written directly as annotated java methods. Examples of expression language and Java5.

A run-time configurable character stream tokenizer that allows the user to define token classes via regular expressions. The developer is not limited to predefined notions of whitespace, commenting, or word modalities.

A simple to use, easy to be extended lexer.