SentencePiece Files

Unsupervised text tokenizer for Neural Network-based text generation

This is an exact mirror of the SentencePiece project, hosted at https://github.com/google/sentencepiece. SourceForge is not affiliated with SentencePiece.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	InfoDownloads / Week
v0.2.2	2026-07-12	6
v0.2.1	2025-08-12	4
v0.2.0	2024-02-19	3
v0.1.99	2023-05-02	4
v0.1.98	2023-04-12	1
Totals: 5 Items		18

Other Useful Business Software

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Build Securely on AWS with Proven Frameworks

Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.

Download Now

$300 Free Credits for Your Google Cloud Projects

Start building on Google Cloud with $300 in free credits. No commitment, no credit card required until you're ready to scale.

Launch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.

Start Free Trial

Recommended Projects

GPT-2
Code for the paper Language Models are Unsupervised Multitask Learners
Tiktoken
tiktoken is a fast BPE tokeniser for use with OpenAI's models
Happy-LLM
Large Language Model Principles and Practice Tutorial from Scratch
LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
BioSimplify
BioSimplify is an open source tool written in Java that introduces and helps in using the bag-of-simplified-sentences (BOSS) model for automatic discourse analysis of English text. This tool is optimized for biomedical scientific literature.

SentencePiece Files

Unsupervised text tokenizer for Neural Network-based text generation

Get an email when there's a new version of SentencePiece