BaseRT
BaseRT is a high-performance LLM inference runtime for Apple Silicon that lets developers pull models from Hugging Face, chat with them locally, or serve an OpenAI-compatible API from one CLI. Accelerated by hand-written Metal kernels, it is designed to deliver fast prefill and decode performance on M-series Macs, with published benchmarks showing up to 6.4× faster prefill than llama.cpp, 3.9× faster than MLX, and up to 1.33× faster decode. The basert CLI handles model downloading, conversion, interactive chat, serving, completion, benchmarking, inspection, and bundle signing. Its server supports chat, completions, embeddings, transcription, tool calls, continuous batching, paged KV caching, and prefix caching, while supported models can process text, vision, and audio. BaseRT uses its own .base model format with Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and can convert GGUF, Hugging Face, and MLX checkpoints.
Learn more
GPT-6 Sol
GPT-6 Sol is an OpenAI model designed to bring GPT-6-class intelligence to professional work, coding, computer use, and agentic workflows at a lower cost than GPT-6 Astra. The model improves on GPT-5.6 Sol in factual reliability, software engineering, business automation, and long-horizon computer tasks. Developers can adjust reasoning effort depending on task complexity, allowing Sol to allocate more computation to difficult problems while using less for simpler work. GPT-6 Sol also introduces a clearer collaboration style with less jargon, fewer unnecessary details, and more concise technical communication. Improved prompt caching helps agents reuse long contexts more efficiently while reducing latency and cached-input costs. GPT-6 Sol is available through ChatGPT Work, Codex, and the OpenAI API as gpt-6-sol.
Learn more
Amazon ElastiCache
Amazon ElastiCache allows you to seamlessly set up, run, and scale popular open-Source compatible in-memory data stores in the cloud. Build data-intensive apps or boost the performance of your existing databases by retrieving data from high throughput and low latency in-memory data stores. Amazon ElastiCache is a popular choice for real-time use cases like Caching, Session Stores, Gaming, Geospatial Services, Real-Time Analytics, and Queuing. Amazon ElastiCache offers fully managed Redis and Memcached for your most demanding applications that require sub-millisecond response times. Amazon ElastiCache works as an in-memory data store and cache to support the most demanding applications requiring sub-millisecond response times. By utilizing an end-to-end optimized stack running on customer-dedicated nodes, Amazon ElastiCache provides secure, blazing-fast performance.
Learn more
Tensormesh
Tensormesh is a caching layer built specifically for large-language-model inference workloads that enables organizations to reuse intermediate computations, drastically reduce GPU usage, and accelerate time-to-first-token and latency. It works by capturing and reusing key-value cache states that are normally thrown away after each inference, thereby cutting redundant compute and delivering “up to 10x faster inference” while substantially lowering GPU load. It supports deployments in public cloud or on-premises, with full observability and enterprise-grade control, SDKs/APIs, and dashboards for integration into existing inference pipelines, and compatibility with inference engines such as vLLM out of the box. Tensormesh emphasizes performance at scale, including sub-millisecond repeated queries, while optimizing every layer of inference from caching through computation.
Learn more
