Search Results for "digital library source code"

HiDream-I1 is an open-source image generation foundation model with 17 billion parameters. It is designed to produce high-quality images from text prompts while keeping inference practical through efficient model design. The project provides full, dev, and fast model variants with different inference step counts. It supports direct Python inference scripts, an interactive Gradio demo, and integration through the Hugging Face Diffusers library. The model uses a Llama 3.1 text encoder path and...

tiktoken is a high-performance, tokenizer library (based on byte-pair encoding, BPE) designed for use with OpenAI’s models. It handles encoding and decoding text to token IDs efficiently, with minimal overhead. Because tokenization is a fundamental step in preparing text for models, tiktoken is optimized for speed, memory, and correctness in model contexts (e.g. matching OpenAI’s internal tokenization). The repo supports multiple encodings (e.g. “cl100k_base”) and lets users switch encoding...

MUSE is a framework for learning multilingual word embeddings that live in a shared space, enabling bilingual lexicon induction, cross-lingual retrieval, and zero-shot transfer. It supports both supervised alignment with seed dictionaries and unsupervised alignment that starts without parallel data by using adversarial initialization followed by Procrustes refinement. The code can align pre-trained monolingual embeddings (such as fastText) across dozens of languages and provides standardized...