Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

This is an exact mirror of the Tiktoken project, hosted at https://github.com/openai/tiktoken. SourceForge is not affiliated with Tiktoken.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	InfoDownloads / Week
0.14.0	2026-08-17	0
0.13.0	2026-05-15	0
0.12.0	2025-10-06	0
0.11.0	2025-08-08	1
0.9.0	2025-02-14	0
Totals: 5 Items		1

Other Useful Business Software

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.

Start Free

$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.

Start Free

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Start Free

Recommended Projects

minbpe
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm
Change File Encoding
Change encoding of text files.
q - Text as Data
Run SQL directly on CSV or TSV files
Encoder
Simple Base64 encoding software
Codepage Converter
Codepage Converter - Convert HTML/Text files to different encoding formats e.g. ANSI to UTF-8 or Unicode. Convert multiple files with 1 click. Works with all encodings. * The issue with UTF-8 has now been fixed.