Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

This is an exact mirror of the Tiktoken project, hosted at https://github.com/openai/tiktoken. SourceForge is not affiliated with Tiktoken. For more information, see the SourceForge Open Source Mirror Directory.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
0.8.0 source code.tar.gz	2024-10-03	31.6 kB	0
0.8.0 source code.zip	2024-10-03	40.5 kB	0
Totals: 2 Items		72.1 kB	0

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

Build gen AI apps with an all-in-one modern database: MongoDB Atlas

MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.

Start Free

Keep company data safe with Chrome Enterprise

Protect your business with AI policies and data loss prevention in the browser

Make AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.

Download Chrome

Recommended Projects

Activepieces
Open Source AI Automation
DeSmuME: Nintendo DS emulator
DeSmuME is a Nintendo DS emulator
Apache OpenOffice
The free and Open Source productivity suite
Clonezilla
A partition and disk imaging/cloning program
KeePass
A lightweight and easy-to-use password manager

Tiktoken Files

tiktoken is a fast BPE tokeniser for use with OpenAI's models

Get an email when there's a new version of Tiktoken