Search Results for "token"

LLM-TLDR is a Python-based tool designed to dramatically reduce the amount of code a large language model needs to read by extracting the essential structure and context from a codebase and presenting only the most relevant parts to the model. Traditional approaches often dump entire files into a model’s context, which quickly exceeds token limits; LLM-TLDR instead indexes project structure, traces dependencies, and summarizes code in a way that preserves semantic relevance while shrinking input size by up to 95 %. This makes queries and analysis much faster and cheaper, with dramatic token savings and latency improvements for LLM-driven development workflows. The project supports multiple programming languages and includes utilities for warming an index and then generating LLM-ready contexts or summaries of specific parts of a project.

EtherApe is a graphical network monitor modeled after etherman. Featuring Ethernet, IP, TCP, FDDI, Token Ring and wireless modes, it displays network activity graphically. Hosts and links change in size with traffic. Color coded protocols display.

Tokenize is a Julia package that serves a similar purpose and API as the tokenize module in Python but for Julia. This is to take a string or buffer containing Julia code, perform lexical analysis and return a stream of tokens.