Search Results for "text decoder" - Page 3

...Its architecture combines sliding-window and global attention with a 681M-parameter vision encoder and dedicated audio encoders. A five-layer speculative decoder predicts multiple subsequent tokens to accelerate inference.

...Its architecture combines sliding-window and global attention, a 681M-parameter vision encoder, dedicated audio encoders, and a five-layer multi-token speculative decoder. It targets coding, general and visual agents, tool use, cybersecurity, long-horizon reasoning, etc.

facebook/bart-large-cnn is a large-scale sequence-to-sequence transformer model developed by Meta AI and fine-tuned specifically for abstractive text summarization. It uses the BART architecture, which combines a bidirectional encoder (like BERT) with an autoregressive decoder (like GPT). Pre-trained on corrupted text reconstruction, the model was further trained on the CNN/DailyMail dataset—a collection of news articles paired with human-written summaries. It performs particularly well in generating concise, coherent, and human-readable summaries from longer texts. ...

Inkling-Small is an open-weight general-purpose multimodal model from Thinking Machines Lab, designed for agentic systems, coding assistants, chatbots, retrieval workflows, and natural-language applications. It accepts text, images, and audio as input and produces text output, with multilingual and multi-programming-language capabilities. The model uses a sparse Mixture-of-Experts architecture with 276B total parameters and 12B active per token, enabling strong performance with lower inference cost than a fully dense model of similar scale. Its 42-layer decoder routes each token through six of 256 specialized experts plus two shared experts, while hybrid local and global attention supports efficient processing. ...