Riffusion

Riffusion (hobby) is a Python-based open source library designed for real-time music and audio generation using stable diffusion techniques. Riffusion (hobby) works by generating and manipulating spectrogram images, which are then converted into playable audio clips, effectively bridging image-based diffusion models with sound synthesis. It implements a diffusion pipeline that supports prompt interpolation, allowing smooth transitions between different musical styles or prompts over time. Riffusion (hobby) serves as the core implementation for audio and image processing, providing essential building blocks for generating music from text prompts. It includes both developer-oriented tools and user-facing components such as a command-line interface and an interactive Streamlit application for experimentation. Additionally, it can run as a Flask server to expose model inference through an API, enabling integration with other applications or services.