Search Results for "lexical analysis"
Sort By:
Showing 40 open source projects for "lexical analysis"
View related business solutions-
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
HanLP
Han Language Processing
...Built on TensorFlow 2.0, it was designed to advance state-of-the-art deep learning techniques and popularize the application of natural language processing in both academia and industry. HanLP is capable of lexical analysis (Chinese word segmentation, part-of-speech tagging, named entity recognition), syntax analysis, text classification, and sentiment analysis. It comes with pretrained models for numerous languages including Chinese and English. It offers efficient performance, clear structure and customizable features, with plenty more amazing features to look forward to on the roadmap. -
2
Tencent Cloud Code Analysis
Static code analysis
Tencent Cloud Code Analysis (TCA for short, used internally by the R&D code CodeDog ) is a cloud-native, distributed, high-performance comprehensive code analysis and tracking platform that integrates many analysis tools, including server, web and client The three components have integrated a number of self-developed tools, and also support the dynamic integration of analysis tools of various programming languages in the industry. Obtain the Tencent Cloud code analysis platform by... -
3
SimpRead
An extension that allows you to instantly enter immersive reading
...Do not change the structure of the current page, just highlight the part that needs to be read, suitable for temporary reading or websites that are not adapted to the reading mode. Have pictures you don’t want to show? no problem! Any element on the page can be re-edited. The brand-new lexical analysis engine can not only recognize Wordpress, Hexo, Ghost, and Discuz, as long as it is a well-structured page, it can automatically generate a reading mode. Not only that, but also has a variety of recognition modes. -
4
SuperCollider
Audio server, programming language, and IDE for sound synthesis
SuperCollider is a platform for audio synthesis and algorithmic composition, used by musicians, artists, and researchers working with sound. It is free and open source software available for Windows, macOS, and Linux. scsynth, a real-time audio server, forms the core of the platform. It features 400+ unit generators (“UGens”) for analysis, synthesis, and processing. Its granularity allows the fluid combination of many known and unknown audio techniques, moving between additive and... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
5
Logos
Create ridiculously fast Lexers
Logos is a lexer generator for Rust, designed to create fast and efficient lexers with minimal code. It leverages Rust's procedural macros to define token patterns, streamlining the process of lexical analysis. -
6
Scala Parser Combinators
simple combinator-based parsing for Scala
scala-parser-combinators is a library that offers simple combinator-based parsing for Scala. Originally part of the Scala standard library, it allows developers to construct parsers using combinators directly in Scala code. -
7
MoarVM
A runtime built for the 6model object system
Short for "Metamodel On A Runtime", MoarVM is a modern virtual machine built for the Rakudo Perl 6 compiler and the NQP Compiler Toolchain. MoarVM is used by the majority of Perl 6 programmers. Highlights include: Great Unicode support, with strings represented at grapheme level, dynamic analysis of running code to identify hot functions and loops, and perform a range of optimizations, including type specialization and inlining, support for threads, a range of concurrency control constructs,... -
8
Ilcg2
The Ilcg2 compiler compiler
...The resulting tool allows complete compilers to be generated with only a relatively small translator class in Java being required to map abstract syntax trees to abstract semantic trees. The lexical analysis, parsing, and generation of vectorised assembler code are all automated. -
9
Tokenize.jl
Tokenization for Julia source code
Tokenize is a Julia package that serves a similar purpose and API as the tokenize module in Python but for Julia. This is to take a string or buffer containing Julia code, perform lexical analysis and return a stream of tokens. -
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
10
...Cobalt provides support for all major OOP features, including classes, objects, inheritance, polymorphism, abstraction, encapsulation, constructors, and reusable modular code design. The language uses a custom interpreter that performs lexical analysis, parsing, and runtime execution. It includes built-in standard libraries such as time, random, and math for common computational tasks without external dependencies. Cobalt also comes with a custom Windows-based IDE built using Tkinter, providing code editing, execution, and output viewing in one environment. The project is distributed as a ZIP package containing the language runtime and IDE. ...
-
11
Ring
Simple and flexible programming language for applications development
The Ring is a practical general-purpose multi-paradigm language. The supported programming paradigms are imperative, procedural, object-oriented, declarative using nested structures, functional, meta programming and natural programming. The language is portable (MS-DOS, Windows, Linux, macOS, Android, WebAssembly, Microcontrollers, etc.) and can be used to create Console, GUI, Web, Games and Mobile applications. The language is designed to be simple, small and flexible. Ring is... -
12
TXM
Unicode XML TEI text analysis platform
TXM is a free and open-source cross-platform Unicode & XML based text analysis environment and graphical client, supporting Windows, Linux and Mac OS X. It can also be used online as a J2EE standard compliant web portal (GWT based) with access control built in. DOWNLOAD LATEST VERSION OF TXM : http://textometrie.ens-lyon.fr/spip.php?rubrique61&lang=en TXM offers a comprehensive range of analysis tools (concordances, collocate search, frequency lists, etc.) based on the powerfull CQP... -
13
MTBook
Machine Translation: Foundations and Models
This is a tutorial, the purpose is to introduce the basic knowledge and modeling methods of machine translation systematically, and on this basis, discuss some cutting-edge technologies of machine translation (formerly known as "Machine Translation: Statistical Modeling and Deep Learning") method"). Its content is compiled into a book, which can be used for the study of senior undergraduates and graduate students in computer and artificial intelligence related majors, and can also be used as... -
14
Lexical Analyzer Generator Quex
Generator of lexical analyzers in C and C++. Unicode Supported.
The goal of this project is to provide a generator for lexical analyzers of maximum computational efficiency and maximum range of applications. This includes the support for Unicode (UTF8, UTF16, ...) and a large variety of other encodings directly and via nested converters such as ICU(tm) and IConv. Sophisticated buffer handling allows to operate on plain file streams, on sockets, or manually fed buffer content. 'Ready-to-build' examples explain related concepts and facilitate practical... -
15
yabasta
Yet Another BAsic Scraper and Text Analysis
YA BASTA! is a Python/R application for Lyrics Web Scraper and Text Analysis. Web scraping is developed in Python, text analysis in R as Python subprocesses. YA BASTA! is only tested on windows OS. To run YA BASTA! just type on window command prompt: python.exe yabasta.py -
16
9cc C compiler
A Small C Compiler
...It serves as a continuation of Rui Ueyama’s series of compiler projects, focusing on clarity and incremental development rather than production-level optimization. The compiler implements essential stages such as lexical analysis, parsing, semantic analysis, and code generation, providing a complete pipeline from source code to executable output. It supports a subset of the C language sufficient to understand real-world compilation concepts, making it a valuable learning resource for developers interested in low-level programming. The project emphasizes readability and simplicity, with code structured to be easily understood rather than highly optimized. ... -
17
Ghawwas_V4
An open source system for Arabic corpora processing
Ghawwas (previously known as Khawas) is an open source system for Arabic corpora processing. Ghawwas V4.0 provides the following main functions: a. Frequency list for single word and N-Grams b. Concordance c. Collocation (MI, CHI Squared, LL, T-Score, Z Score, Dice, Log Dice, Weirdness Coefficient) d. Lexical patterns search e. Two corpora frequency profile comparison based on MI, CHI, LL, T-Score, Z Score, Dice, Log Dice, Weirdness Coefficient f. Accept Windows and UTF-8 character... -
18
CSharp Identifier Scraper
Small tool that outputs C# identifier names from .cs files
This is a small test tool that reads C# source code files, performs lexical analysis and outputs the identifier names. It is built on top of the CodeConics Library. -
19
PEG.js
PEG.js is a parser generator for JavaScript
PEG.js is a simple parser generator for JavaScript that produces fast parsers with excellent error reporting. You can use it to process complex data or computer languages and build transformers, interpreters, compilers and other tools easily. PEG.js is still very much work in progress. There are no compatibility guarantees until version 1.0. Based on parsing expression grammar formalism, more powerful than traditional LL(k) and LR(k) parsers. Usable from your browser, from the command line,... -
20
SeCo Lexical Analysis Utilities
Moved to GitHub LFS
-
21
Phrasal
Statistical phrase-based machine translation system
...Distinctive features include: providing an easy to use API for implementing new decoding model features, the ability to translating using phrases that include gaps (Galley et al. 2010), and conditional extraction of phrase-tables and lexical reordering models. Developed by The Natural Language Processing Group at Stanford University, a team of faculty, postdocs, programmers and students who work together on algorithms that allow computers to process and understand human languages. Our work ranges from basic research in computational linguistics to key applications in human language technology, and covers areas such as sentence understanding, automatic question answering, machine translation, syntactic parsing and tagging, sentiment analysis. -
22
Khawarizmi - Arabic Programming Language
Basic Arabic programming language designed for students
An Arabic programming language that can help students to better understand and apply the principles and concepts of algorithmic and programming. The idea behind the project is to provide an easy language for teaching which combines the ease of Pascal programming language (usage of natural language words such as Program, Read, Write, ..) as well as the abbreviations of C language ({} || , &&,! ...). Please note that Khawarizmi is discontinued and published for educational purposes. -
23
JS/CC LALR(1) Parser Generator
A LALR(1) Parser Generator for JavaScript written in JavaScript
JS/CC is a parser generator for JavaScript/ECMAScript, which is entirely written in ECMAScript itself. It produces LALR(1) parsers with integrated lexical analyzers. JS/CC can run as a website or be invoked from a console, and runs on several platforms. -
24
SeCo Lexical Analysis Web Service
Moved to GitHub
-
25
Pecker Scanner
A static source code analyser for vulnerabilities in PHP.
A scanner named pecker, written in php,It can check dangerous functions with lexical analysis. to scans files for malicious PHP Code. Github : https://github.com/cfc4n/pecker
