Search Results for "edge computing"

...Because it runs in-process, developers can embed it in native apps, microservices, or edge computing scenarios where traditional server-based vector databases might be overkill.

...The runtime is capable of running language models with billions of parameters on devices with only a few hundred megabytes of memory, which is significantly lower than typical LLM infrastructure requirements. This makes PicoLM particularly suitable for edge computing, offline AI applications, and embedded AI devices that cannot rely on cloud resources.

mllm is an open-source inference engine designed to run multimodal large language models efficiently on mobile devices and edge computing environments. The framework focuses on delivering high-performance AI inference in resource-constrained systems such as smartphones, embedded hardware, and lightweight computing platforms. Implemented primarily in C and C++, it is designed to operate with minimal external dependencies while taking advantage of hardware-specific acceleration technologies such as ARM NEON and x86 AVX2 instructions. ...

ElatoAI is a real-time AI voice agent platform built around IoT hardware (ESP32) that enables continuous speech-to-speech conversations using state-of-the-art multimodal voice models with minimal latency and global performance via edge computing. The system integrates voice synthesis and recognition by connecting an ESP32 device through secure WebSockets to edge server functions written in Deno, allowing users to speak naturally with AI agents hosted through cloud APIs including OpenAI’s Realtime API, Gemini’s Live API, xAI’s Grok Voice Agent API, and others. It includes a web client (built with Next.js) for managing devices, controlling volume, and viewing conversation transcripts, while the hardware runs optimized firmware to deliver responses in near real time — even supporting >15-minute uninterrupted conversations.

...The repository includes multiple demonstration systems implemented using languages such as Python and C++, covering topics ranging from object detection to embedded vision systems. Many of the projects illustrate how computer vision algorithms can interact with hardware platforms, including robotics systems and edge computing devices. The repository provides examples that combine machine learning models with real-world applications such as robotic arms, video analysis, and automated visual measurement systems.