Search Results for "qualcomm"

...AIMET enables neural networks to run more efficiently on fixed-point AI hardware accelerators. Quantized inference is significantly faster than floating point inference. For example, models that we’ve run on the Qualcomm® Hexagon™ DSP rather than on the Qualcomm® Kryo™ CPU have resulted in a 5x to 15x speedup. Plus, an 8-bit model also has a 4x smaller memory footprint relative to a 32-bit model. However, often when quantizing a machine learning model (e.g., from 32-bit floating point to an 8-bit fixed point value), the model accuracy is sacrificed.

mllm is an open-source inference engine designed to run multimodal large language models efficiently on mobile devices and edge computing environments. The framework focuses on delivering high-performance AI inference in resource-constrained systems such as smartphones, embedded hardware, and lightweight computing platforms. Implemented primarily in C and C++, it is designed to operate with minimal external dependencies while taking advantage of hardware-specific acceleration technologies...

LiteRT is Google's next-generation on-device machine learning framework and the successor to TensorFlow Lite, designed for high-performance AI and generative AI deployment across edge devices. It provides efficient model conversion, optimization, and runtime execution while leveraging hardware acceleration from CPUs, GPUs, and NPUs. LiteRT supports a wide range of platforms, including Android, iOS, Linux, macOS, Windows, web environments, and IoT devices. The framework simplifies on-device...