Search Results for "cache" - Page 3

...Its Hybrid-Attention Mixture-of-Experts architecture contains 250B total parameters while activating only 15B per token, combining three linear-attention layers with one softmax-attention layer for efficient inference. The model supports a native 1M-token context window and uses NoPE instead of rotary positional encoding, reducing long-context KV-cache requirements. Solar Open 2 includes 321 experts, with eight routed experts plus one shared expert activated per token. It was pretrained on roughly 12 trillion tokens and supports English, Korean, and Japanese. Agent capabilities include multi-step reasoning, tool calling, MCP tools, and end-to-end task execution. It also offers direct-response and high-reasoning modes and supports deployment through Transformers, vLLM, SGLang, and quantized variants.