2 projects for "tokenizer" with 2 filters applied:

  • Ship Agents Faster Icon
    Ship Agents Faster

    Transform your applications and workflows into powerful agentic systems at global scale.

    Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
    Get Started Free
  • MongoDB Atlas runs apps anywhere Icon
    MongoDB Atlas runs apps anywhere

    Deploy in 115+ regions with the modern database for every enterprise.

    MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
    Start Free
  • 1
    Enhanced version of Brill's Parts-of-Speech Tagger with built-in Tokenizer and Lemmatizer.
    Leader badge
    Downloads: 51 This Week
    Last Update:
    See Project
  • 2
    LACE means "Lucene Analyzer for CJK (Chinese/Japanese/Korean) & English". It's a simple tokenizer that can handle English-CJK mixed text. Chinese words are handled using a dictionary based method.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • Next
Auth0 Logo