a tokenizer that is general purpose, and multilanguage. it will provide consistent usage across the languages, and be tied into that language's unique common usage types.

Project Activity

See All Activity >

Categories

Algorithms

License

BSD License, GNU General Public License version 2.0 (GPLv2)

Follow fms string tokenizer

fms string tokenizer Web Site

nel_h2
Gen AI apps are built with MongoDB Atlas Icon
Gen AI apps are built with MongoDB Atlas

The database for AI-powered applications.

MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of fms string tokenizer!

Additional Project Details

Intended Audience

Developers, Other Audience

User Interface

Command-line

Programming Language

C, C#, C++, Java

Related Categories

C# Algorithms, C++ Algorithms, C Algorithms, Java Algorithms

Registered

2007-08-21