Search Results for "lexer"
Sort By:
Showing 54 open source projects for "lexer"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Add Two Lines of Code. Get Full APM.Works out of the box for Rails, Django, Express, Phoenix, and more. Monitoring exceptions and performance in no time.
-
1
Doctrine Lexer
Base library for a lexer that can be used in Recursive Descent Parsers
...The Lexer is responsible for giving you an API to walk across a string one character at a time and analyze the type of each character, value and position of each token in the string. The low level API of the lexer is pretty simple. The setInput($input) option sets the input data to be tokenized. The Lexer is immediately reset and the new input tokenized. -
2
Boa
Embeddable and experimental Javascript engine written in Rust
This is an experimental Javascript lexer, parser and interpreter written in Rust. Currently, it has support for some of the language. -
3
Red Programming Language
Red is a next-generation programming language
Red is a next-generation programming language strongly inspired by Rebol, but with a broader field of usage thanks to its native-code compiler, from system programming to high-level scripting and cross-platform reactive GUI, while providing modern support for concurrency, all in a zero-install, zero-config, single 1MB file! During the work on the low-level parts of the new Red lexer, the need arised for intra-function factorization abilities to keep the lexer code as DRY as possible. Subroutines were introduced to solve that. They act as the GOSUB directive from Basic language. They are defined as a separate block of code inside a function's body and are called like regular functions (but without any arguments). ... -
4
Logos
Create ridiculously fast Lexers
Logos is a lexer generator for Rust, designed to create fast and efficient lexers with minimal code. It leverages Rust's procedural macros to define token patterns, streamlining the process of lexical analysis. -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
5
CCC
Claude Opus 4.6 wrote a dependency-free C compiler in Rust
Claude’s C Compiler is a remarkable experiment in automated software generation: a full C compiler written from scratch in Rust entirely by an AI model (Claude Opus 4.6) with minimal human intervention, capable of targeting multiple architectures like x86-64, i686, ARM, and RISC-V. The project implements every part of a traditional compiler — the lexer, parser, intermediate representation, optimizer, code generator, assembler, linker, and debug info emitter — without relying on any external toolchain, producing standalone executables in the ELF format. While still early and not fully validated for correctness, it demonstrates the frontier of what sophisticated AI can build in complex, systems-level software domains, including the potential to compile real operating systems and large software projects. -
6
ANTLR
Parser generator to read, process, or translate structured text
ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It's widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build and walk parse trees. It’s widely used in academia and industry to build all sorts of languages, tools, and frameworks. Twitter search uses ANTLR for query parsing, with over 2 billion queries a day. The languages for... -
7
gojq
Pure Go implementation of jq
This is an implementation of jq command written in Go language. You can also embed gojq as a library to your Go products. gojq is purely implemented with Go language and is completely portable. jq depends on the C standard library so the availability of math functions depends on the library. jq also depends on the regular expression library and it makes building scripts complex. gojq implements nice error messages for invalid query and JSON input. The error message of jq is sometimes... -
8
re2
Alternative to backtracking regular expression engines
RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library. This is the source code repository for RE2, a regular expression library. There is a fair amount of documentation (including code snippets) in the re2.h header file. Unless otherwise noted, the RE2 source files are distributed under the BSD-style license found in the LICENSE file. In POSIX mode, RE2 accepts standard POSIX (egrep) syntax... -
9
GraphQL for .NET
This is an implementation of Facebook's GraphQL in .NET
GraphQL is a query language for your API, and a server-side runtime for executing queries by using a type system you define for your data. GraphQL isn't tied to any specific database or storage engine and is instead backed by your existing code and data. A GraphQL service is created by defining types and fields on those types, then providing functions for each field on each type. All packages generated from this repository come with embedded pdb and support Source Link. If you are having... -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
10
...Unlike any other such tool, re2c focuses on generating high efficient code for regular expression matching. As a result this allows a much broader range of use than any traditional lexer.
-
11
RE/flex lexical analyzer generator
The regex-centric, fast lexical analyzer generator for C++
A C++ high-performance regex library and Flex-compatible lexical analyzer generator with full Unicode support, new indentation anchors, lazy quantifiers, and many other modern features. Accepts Flex lexer specification syntax and is compatible with Bison/Yacc parsers. Generates reusable source code that is easy to understand. Supports fast scanning of UTF-8/16/32 files, strings, and streams. The reflex scanner generator generates clean C++ lexer class code that is thread-safe. Generates Graphviz files to visualize state machine DFAs. ... -
12
...Recognizes particularities of several Pascal dialects: Turbo/Borland/Delphi (mostly), CodeWarrior, ISO, ISO Extended (a bit). Includes a full ayacc / aflex parser & lexer for Ada 95+ with sources. Mirror: https://github.com/zertovitch/pascal-to-ada
-
13
soulng
Lexer and parser generator tool for C++
The SoulNG project contains two tools: a lexical analyzer generator slg and a parser generator spg. The generators and the produced lexical analyzers and parsers use four C++ libraries that are also included in the project. The slg tool takes a .lexer file that contains the description of a lexical analyzer as input and produces C++ source code for a lexical analyzer as output. The produced lexical analyzer is a finite state machine that recognizes patterns described as regular expressions and return corresponding token identifiers to the parser. The spg tool takes .parser files that contain descriptions of parsers as input and produces C++ source code for parser classes as output. ... -
14
ARTCOMParser
Parser library that uses an EBNF like grammar to generate a parse tree
Parser library that uses an EBNF like grammar to generate a parse tree You can use it to create a quick parser It uses backtracking so it may be slow for some grammar rules -
15
MLton
A whole-program optimizing compiler for Standard ML
...In addition, MLton provides a feature rich Standard ML programming environment, with full support for SML97 as given in The Definition of Standard ML (Revised), a number of useful language extensions, a complete implementation of the Standard ML Basis Library, various useful libraries, a simple and fast C foreign function interface, the ML Basis system for programming with source libraries, and tools such as a lexer generator, a parser generator, and a profiler. -
16
KAGSA
KAGSA PROGRAMMING LANGUAGE
...It has a flexible syntax and allows some things that other languages prohibit, such as using certain symbols in variable names and starting variable names with numbers. KAGSA contains several main components, including a lexer, syntax checker, parser, and compiler. It supports object-oriented programming, iteration, and other features commonly used in programming languages. -
17
LR Parser and Lexer Generators: Fortran
LR parser generator and lexer generator for use in Fortran
...An infrastructure to use a generated lexer and parser is therefore included as part of these programs. -
18
8cc C Compiler
Compiler for the C programming language
...It was created as a hobby project with the goal of demonstrating how a compiler works internally, making it a valuable resource for learning compiler design. The project includes core components such as a lexer, parser, and preprocessor, allowing users to explore each stage of the compilation process. One of its notable features is its ability to compile itself, showcasing its completeness and serving as a practical example of bootstrapping in compiler development. The code is intentionally written to be concise and readable, making it accessible to developers who want to study compiler internals. ... -
19
javalang
Pure Python Java parser and tools
javalang is a pure Python library for working with Java source code. javalang provides a lexer and parser targeting Java 8. The implementation is based on the Java language spec. -
20
9cc C compiler
A Small C Compiler
9cc is a small and educational C compiler designed to demonstrate the internal workings of a compiler while still being capable of compiling meaningful programs. It serves as a continuation of Rui Ueyama’s series of compiler projects, focusing on clarity and incremental development rather than production-level optimization. The compiler implements essential stages such as lexical analysis, parsing, semantic analysis, and code generation, providing a complete pipeline from source code to... -
21
This project has moved to GitHub ! The version here at SourceForge will remain for historic purpose. Koopa is a parser generator, made for COBOL. It can handle source files in isolation (no preprocessing required) and doesn't mind the presence of CICS/SQL fragments. The grammar is easily extensible in a way which minimizes the impact on the overall code.
-
22
CodeConics
CodeConics is a C# lexer/parser/syntax tree generatior
CodeConics is simple, intuitive, easy to use and debug code-manipulation solution, that is grammar independent and recursion based, as opposed to other DFA(Deterministic Finite-state Automata) based solutions. -
23
NaiveLanguageTools
C# lexer and parser
NLT is free, open-source C# lexer and GLR parser suite -- which translates to ability to parse ambiguous grammars. Grammar can be defined directly in code or (preferably) in separate file (lex/yacc-like) for included generator. If you understand/guess the meaning of grammar in C#: // scanning lexer.AddStringRule(")", match => SymbolEnum.RPAREN); // parsing prod_builder.AddProduction(SymbolEnum.exp, SymbolEnum.LPAREN, SymbolEnum.exp, SymbolEnum.RPAREN, (_1, e, _3) => (AstNode)e); or grammar in NLT format: // scanning /[A-Za-z_][A-Za-z_0-9]*/ -> IDENTIFIER, IdentifierSymbol.Create($text); // parsing program -> list:namespace_list { new Program(currCoords(), (Namespaces)list) }; you should be able to use it :-). ... -
24
Chevrotain
Parser Building Toolkit for JavaScript
Chevrotain is a powerful and fast parser-building toolkit for JavaScript. It supports the construction of LL(k) parsers and is suitable for building interpreters, compilers, and DSLs. -
25
pylangparser
pylangparser - Simple language parsing from Python
pylangparser - Simple language parsing from Python. Project provides classes for parsing formal languages in an easy way. Without using any external libraries, only unittest, re and pprint. There is a Lexer and a Parser class. The lexer produces list of tokens that the Parser then uses to build the AST. The lexer can also be used as a stand alone component. There is support for building customized AST's. The grammars are defined directly into the Python code. In the examples folder you will find both simple example scripts demonstrating basic usage of the parser and some more useful and complex ones. ...
