Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

This is an exact mirror of the Tiktoken project, hosted at https://github.com/openai/tiktoken. SourceForge is not affiliated with Tiktoken. For more information, see the SourceForge Open Source Mirror Directory.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
0.12.0 source code.tar.gz	2025-10-06	34.3 kB	0
0.12.0 source code.zip	2025-10-06	44.1 kB	0
Totals: 2 Items		78.5 kB	0

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

The database for AI-powered applications.

MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.

Start Free

Build Securely on AWS with Proven Frameworks

Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.

Download Now

Recommended Projects

Activepieces
Open Source AI Automation
Clonezilla
A partition and disk imaging/cloning program
KeePass
A lightweight and easy-to-use password manager
DeSmuME: Nintendo DS emulator
DeSmuME is a Nintendo DS emulator
7-Zip
A free file archiver for extremely high compression

Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

Get an email when there's a new version of Tiktoken