ZAYA1-8B
Efficient MoE reasoning model for coding and math workloads
...It introduces architectural innovations such as Compressed Convolutional Attention, a novel MLP-based expert router, and learned residual scaling to improve routing stability and inference efficiency. The model was trained entirely on AMD infrastructure and refined through supervised fine-tuning and multi-stage reinforcement learning focused on reasoning and coding.