Easy-to-use headless React Hooks to run LLMs in the browser with WebGPU. As simple as useLLM().
Features
- Supports Vicuna 7B
- Use custom system prompts and "user:"/"assistant:" role names
- Completion options like max tokens and stop sequences
- No data leaves the browser. Accelerated via WebGPU
- Hooks built to 'Bring your own UI'
- Persistent storage for conversations in browser storage. Hooks for loading and saving conversations
License
MIT LicenseFollow react-llm
Other Useful Business Software
Stop Storing Third-Party Tokens in Your Database
Rolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of react-llm!