Tokenize.jl Files

Tokenization for Julia source code

This is an exact mirror of the Tokenize.jl project, hosted at https://github.com/JuliaLang/Tokenize.jl. SourceForge is not affiliated with Tokenize.jl.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
v0.5.29	2024-04-22		0
v0.5.28	2024-01-08		0
v0.5.27	2024-01-05		0
v0.5.26	2023-11-07		0
v0.5.25	2022-12-15		0
Totals: 5 Items			0

Other Useful Business Software

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

Veeam Data Platform v13.1 - Get Your Free Trial

Secure by design, portable by default. Recover clean, fast, anywhere. Start a free trial.

Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.

Try it Free

$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.

Start Free

Recommended Projects

ULS
A lexical analyzer generator that makes the class source code.
The GR module for Julia
Plotting for Julia based on GR
twitter-text
This code is used at Twitter to tokenize and parse text
Tokenizers
Fast State-of-the-Art Tokenizers optimized for Research and Production
Flask-JWT-Extended
An open source Flask extension that provides JWT support