Search Results for "token system" - Page 2

tiny-llm is an educational open-source project designed to teach system engineers how large language model inference and serving systems work by building them from scratch. The project is structured as a guided course that walks developers through the process of implementing the core components required to run a modern language model, including attention mechanisms, token generation, and optimization techniques.

...The project adapts and expands upon earlier DreamBooth research by providing practical scripts, notebooks, and workflows that allow users to train personalized models on local machines, cloud environments, or platforms such as Google Colab. It focuses heavily on usability, offering detailed guides for different setups while still exposing advanced configuration options for experienced users. The system allows users to bind a unique token to a subject, which can later be used in prompts to generate consistent and recognizable outputs across different contexts. It also supports captioning, multi-subject training, and regularization techniques to improve generalization and avoid overfitting.

...The project implements techniques that allow model components to be dynamically moved between CPU memory and GPU memory during inference, significantly reducing the amount of GPU VRAM required to run the model. This approach takes advantage of the sparse activation properties of mixture-of-experts architectures, where only a subset of expert networks are used for each token during generation. By selectively loading and caching the required experts, the system avoids keeping the entire model in GPU memory at once. The repository includes notebooks and code examples that demonstrate how to run large language models on consumer hardware such as personal GPUs or cloud notebook environments.