Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

This is an exact mirror of the Tiktoken project, hosted at https://github.com/openai/tiktoken. SourceForge is not affiliated with Tiktoken. For more information, see the SourceForge Open Source Mirror Directory.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
0.9.0 source code.tar.gz	2025-02-14	32.5 kB	0
0.9.0 source code.zip	2025-02-14	42.0 kB	0
Totals: 2 Items		74.4 kB	0

Other Useful Business Software

Get the most trusted enterprise browser

Advanced built-in security helps IT prevent breaches before they happen

Defend against security incidents with Chrome Enterprise. Create customizable controls, manage extensions and set proactive alerts to keep your data and employees protected without slowing down productivity.

Download Chrome

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Recommended Projects

Activepieces
Open Source AI Automation
Apache OpenOffice
The free and Open Source productivity suite
Clonezilla
A partition and disk imaging/cloning program
KeePass
A lightweight and easy-to-use password manager
DeSmuME: Nintendo DS emulator
DeSmuME is a Nintendo DS emulator

Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

Get an email when there's a new version of Tiktoken