Audience
AI developers interested in a large language model
About OPT
Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these models are difficult to replicate without significant capital. For the few that are available through APIs, no access is granted to the full model weights, making them difficult to study. We present Open Pre-trained Transformers (OPT), a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers. We show that OPT-175B is comparable to GPT-3, while requiring only 1/7th the carbon footprint to develop. We are also releasing our logbook detailing the infrastructure challenges we faced, along with code for experimenting with all of the released models.
Other Popular Alternatives & Related Software
Muse Spark Open
Muse Spark Open is an open-weight, natively multimodal language model for developers and organizations building AI applications, coding systems, and agentic workflows. The released model has 816 billion total parameters, 42 billion active parameters, and a 1-million-token context window. Teams can download the model weights, run inference through supported engines such as vLLM, SGLang, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp, or Unsloth, and customize the model using PyTorch. It supports long-context reasoning, software engineering, coding, tool use, and multimodal inputs. Deployment options include cloud inference through listed API providers or self-hosting on compatible GPU systems; Meta documents a desktop setup using two Apple M5 Ultra Mac Studio computers. The model is released under the Muse Model License with limited commercial terms.
Learn more
CodeQwen
CodeQwen is the code version of Qwen, the large language model series developed by the Qwen team, Alibaba Cloud. It is a transformer-based decoder-only language model pre-trained on a large amount of data of codes. Strong code generation capabilities and competitive performance across a series of benchmarks. Supporting long context understanding and generation with the context length of 64K tokens. CodeQwen supports 92 coding languages and provides excellent performance in text-to-SQL, bug fixes, etc. You can just write several lines of code with transformers to chat with CodeQwen. Essentially, we build the tokenizer and the model from pre-trained methods, and we use the generate method to perform chatting with the help of the chat template provided by the tokenizer. We apply the ChatML template for chat models following our previous practice. The model completes the code snippets according to the given prompts, without any additional formatting.
Learn more
GLM-5.3
GLM-5.3 is Z.ai’s frontier coding model designed for complex software engineering, long-horizon agent tasks, and advanced post-training research. The model uses the same base model as GLM-5.2, with improvements coming from scaled post-training across more environments, more diverse tasks, and larger compute investment. GLM-5.3 delivers stronger coding performance, better task ownership, improved benchmark results, and greater efficiency across realistic development workflows. It is built to handle complex coding tasks, production-style engineering work, research environments, automation tasks, and agentic workflows that require multi-step execution. The model also shows emergent cyber capabilities in vulnerability discovery and exploitation-chain reasoning, with safety evaluation and hardening planned before open-weight release.
Learn more
Galactica
Information overload is a major obstacle to scientific progress. The explosive growth in scientific literature and data has made it ever harder to discover useful insights in a large mass of information. Today scientific knowledge is accessed through search engines, but they are unable to organize scientific knowledge alone.
Galactica is a large language model that can store, combine and reason about scientific knowledge. We train on a large scientific corpus of papers, reference material, knowledge bases and many other sources. We outperform existing models on a range of scientific tasks. On technical knowledge probes such as LaTeX equations, Galactica outperforms the latest GPT-3 by 68.2% versus 49.0%. Galactica also performs well on reasoning, outperforming Chinchilla on mathematical MMLU by 41.3% to 35.7%, and PaLM 540B on MATH with a score of 20.4% versus 8.8%.
Learn more
Integrations
No integrations listed.
Company Information
Meta
Founded: 2004
United States
www.meta.com
Videos and Screen Captures
Other Useful Business Software
Custom VMs From 1 to 96 vCPUs With 99.95% Uptime
Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
Product Details
Platforms Supported
Cloud
On-Premises
Training
Documentation
