Grok-1 is a 314-billion-parameter Mixture-of-Experts (MoE) large language model developed by xAI. Designed to optimize computational efficiency, it activates only 25% of its weights for each input token. In March 2024, xAI released Grok-1's model weights and architecture under the Apache 2.0 license, making them openly accessible to developers. The accompanying GitHub repository provides JAX example code for loading and running the model. Due to its substantial size, utilizing Grok-1 requires a machine with significant GPU memory. The repository's MoE layer implementation prioritizes correctness over efficiency, avoiding the need for custom kernels.

This is a full repo snapshot ZIP file of the Grok-1 code.

Features

  • 314-billion-parameter Mixture-of-Experts (MoE) architecture
  • Efficient computation, activating only 25% of parameters per token
  • Fully open-source under the Apache 2.0 license
  • JAX-based implementation with example code provided
  • Scalable and modular, designed for research and development
  • Requires high GPU memory for inference and fine-tuning
  • Correctness-focused MoE layer without custom kernels
  • Optimized for natural language understanding and generation

Project Activity

See All Activity >

License

Apache License V2.0

Follow Grok-1

Grok-1 Web Site

Other Useful Business Software
AI-powered service management for IT and enterprise teams Icon
AI-powered service management for IT and enterprise teams

Enterprise-grade ITSM, for every business

Give your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
Try it Free
Rate This Project
Login To Rate This Project

User Ratings

★★★★★
★★★★
★★★
★★
1
0
0
0
0
ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 5 / 5
features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 5 / 5
design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 5 / 5
support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 5 / 5

User Reviews

  • Really great model
Read more reviews >

Additional Project Details

Languages

English

Programming Language

Python

Related Categories

Python Large Language Models (LLM), Python AI Models

Registered

2025-02-27