Search Results for "inference engine"

MiniSearch is a minimalist web search application with a built-in AI assistant that runs largely inside the browser for privacy-focused information retrieval. The project combines metasearch capabilities with local or remote language model inference to provide conversational answers alongside traditional search results. It is designed to be lightweight, easy to deploy with Docker, and configurable for both personal and hosted use cases. The platform supports browser-level integration so users can set it as their default search engine for quick access. Its architecture emphasizes privacy by avoiding tracking and minimizing data collection while still enabling advanced AI features. ...

The challenge is to run Stable Diffusion 1.5, which includes a large transformer model with almost 1 billion parameters, on a Raspberry Pi Zero 2, which is a microcomputer with 512MB of RAM, without adding more swap space and without offloading intermediate results on disk. The recommended minimum RAM/VRAM for Stable Diffusion 1.5 is typically 8GB. Generally, major machine learning frameworks and libraries are focused on minimizing inference latency and/or maximizing throughput, all of which at the cost of RAM usage. So I decided to write a super small and hackable inference library specifically focused on minimizing memory consumption: OnnxStream. OnnxStream is based on the idea of decoupling the inference engine from the component responsible for providing the model weights, which is a class derived from WeightsProvider. ...

...If you want to use the pre-trained English model for performing speech-to-text, you can download it (along with other important inference material) from the DeepSpeech releases page.

...This part of the system would ask a question to the user and, according to the answer, respond with the next question or answer. This is done by using a inference engine that I designed along with the knowledge base and search tree to interrupt and use the input provided as an answer to the question.

Mandarax is a pure Java implementation of a rule engine. It supports mutiples types of facts and rules based on reflection, databases, EJB etc, supporting XML standards (RuleML 0.8). It provides a J2EE compliant inference engine using backward chaining.

SIES is a forward-chaining rule engine server. Clients connect to update a global knowledge-base and/or can register to be notified when specified elements are inferred. Local access for a client's own inferences is also possible.