Search Results for "audio generation"
Sort By:
Showing 259 open source projects for "audio generation"
View related business solutions-
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
MiMo Audio
Audio Language Models are Few-Shot Learners
...It supports audio understanding, speech intelligence, spoken dialogue, instruction-following audio generation, and text-to-speech-style tasks. The architecture combines audio tokenization, patch encoding, a language model, and patch decoding to make high-rate audio sequences more efficient to model. Overall, it is useful for researchers and developers experimenting with advanced audio LLMs, speech generation, audio reasoning, and instruction-tuned multimodal systems. -
2
MLX-Audio
A text-to-speech, speech-to-text and speech-to-speech library
...The project provides a straightforward CLI (mlx_audio.tts.generate) as well as a Python API for programmatic generation of audio, including parameters for voice choice, speed, language hints, output format, and sample rate. It includes examples such as audiobook generation to demonstrate long-form synthesis and joined audio segments. On top of that, MLX-Audio offers a modern web interface powered by FastAPI, with real-time waveform and 3D visualizations, file upload, and audio management. -
3
Step-Audio
Open-source framework for intelligent speech interaction
Step-Audio is a unified, open-source framework aimed at building intelligent speech systems that combine both comprehension and generation: it integrates large language models (LLMs) with speech input/output to handle not only semantic understanding but also rich vocal characteristics like tone, style, dialect, emotion, and prosody. The design moves beyond traditional separate-component pipelines (ASR → text model → TTS), instead offering a multimodal model that ingests speech or audio and produces speech accordingly, enabling natural dialogue, voice cloning, and expressive speech synthesis. ... -
4
Text Generation Web UI
Oobabooga - The definitive Web UI for local AI, with powerful features
A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA. Dropdown menu for switching between models. Notebook mode that resembles OpenAI's playground. Chat mode for conversation and role playing. Instruct mode compatible with Alpaca and Open Assistant formats. Nice HTML output for GPT-4chan. Markdown output for GALACTICA, including LaTeX rendering. Custom chat characters. Advanced chat features (send images, get audio responses with TTS).... -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
Step-Audio 2
Multi-modal large language model designed for audio understanding
...It integrates a latent-space audio encoder, discrete acoustic tokens, and reinforcement-learning–based training (CoT + RL) to enhance its ability to capture and reproduce voice styles, intonations, and subtle vocal cues. Moreover, Step-Audio2 supports tool-calling and retrieval-augmented generation (RAG), allowing it to access external knowledge sources or audio/text databases, thus reducing hallucinations and improving coherence in complex dialogues. -
6
LTX-2.3
Official Python inference and LoRA trainer package
LTX-2.3 is an open-source multimodal artificial intelligence foundation model developed by Lightricks for generating synchronized video and audio from prompts or other inputs. Unlike most earlier video generation systems that only produced silent clips, LTX-2 combines video and audio generation in a unified architecture capable of producing coherent audiovisual scenes. The model uses a diffusion-transformer-based architecture designed to generate high-fidelity visual frames while simultaneously producing corresponding audio elements such as speech, music, ambient sound, or effects. ... -
7
Fish Audio Python SDK
The official Python library for the Fish Audio API
Fish Audio Python is the official Python SDK for working with the Fish Audio platform. It gives developers a programmatic way to access Fish Audio features such as text-to-speech generation, audio playback, saving output files, and API-based voice workflows. The package is designed for Python applications that need speech generation without manually handling raw HTTP requests. -
8
Fun Audio Chat
Large Audio Language Model built for natural interactions
Fun Audio Chat is an interactive voice-first conversational AI platform designed to let users engage in natural spoken dialogue with large language models in real time, turning speech into context-aware responses while maintaining a smooth back-and-forth experience. It combines speech recognition, audio processing, and AI generation so users can speak simply and receive spoken replies, enabling applications such as virtual assistants, voice bots, and hands-free chat interfaces. ... -
9
JoyAI-Echo
Pushing the Frontier of Long Audio-Visual Generation
...JoyAI-Echo focuses on text-to-video and multi-shot long-video generation, while image-to-video support is not part of the current release scope. It is most useful for research and experimental video workflows that need synchronized audio, coherent characters, and editable story-level generation. -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
10
Voicebox
The open-source voice synthesis studio powered by Qwen3-TTS
...A standout capability is its multi-track timeline editor and supporting audio tools (like trimming and conversation mixing), which let creators compose multi-voice scenes instead of generating single clips in isolation. It is API-first, meaning you can use it as an app for production work or integrate its speech generation into your own software via an API layer. -
11
HeartMuLa
A Family of Open Sourced Music Foundation Models
...For text extraction from audio, it provides HeartTranscriptor, a Whisper-based model tuned specifically for lyrics transcription, which helps bridge generated or recorded audio back into structured text. It also introduces HeartCLAP, which aligns audio and text into a shared embedding space. -
12
Riffusion App
Stable diffusion for real-time music generation (web app)
Riffusion App Hobby is an open-source interactive web application that enables real-time music generation using stable diffusion models adapted for audio synthesis. Unlike traditional music generation tools, it treats audio as spectrogram images and applies diffusion techniques to generate continuous sound transitions, allowing users to create evolving musical loops and compositions. The application is built with modern web technologies including Next.js, React, and three.js, providing a responsive and visually engaging interface for experimentation. ... -
13
VoxCPM2
Tokenizer-Free TTS for Multilingual Speech Generation
...It also introduces voice design capabilities, allowing users to generate entirely new voices from natural language descriptions without requiring reference audio. -
14
LiveAvatar
Streaming Real-time Audio-Driven Avatar Generation
LiveAvatar is an open-source research and implementation project that provides a unified framework for real-time, streaming, interactive avatar video generation driven by audio and other control signals. It implements techniques from state-of-the-art diffusion-based avatar modeling to support infinite-length continuous video generation with low latency, enabling interactive AI avatars that maintain continuity and realism over extended sessions. The project co-designs algorithms and system optimizations, such as block-wise autoregressive processing and fast sampling strategies, to deliver real-time frame rates (e.g., ~45 FPS on appropriate GPU clusters) while handling non-stop generation without quality degradation. ... -
15
Miso TTS
Miso TTS is an 8 billion, highly emotive text-to-speech model
Miso TTS is an advanced 8-billion-parameter text-to-speech model developed by Miso Labs for generating highly expressive and natural-sounding conversational speech. Built on an RVQ Transformer architecture inspired by Sesame CSM, it combines a powerful Llama-based backbone with an autoregressive audio decoder to produce high-quality audio from text. The model supports both standard speech synthesis and voice-conditioned generation using optional audio prompts for voice cloning. Miso TTS generates Mimi audio codes and can leverage conversation history to create more contextually aware and realistic dialogue. Designed for local deployment, it offers watermarking by default to help promote responsible use of generated audio. ... -
16
Podcastfy.ai
Transforming Multimodal Content into Captivating Multilingual Audio
Podcastfy is an open-source Python package that transforms multi-modal content (text, images) into engaging, multi-lingual audio conversations using GenAI. Input content includes websites, PDFs, youtube videos as well as images. Unlike UI-based tools focused primarily on note-taking or research synthesis (e.g. NotebookLM), Podcastfy focuses on the programmatic and bespoke generation of engaging, conversational transcripts and audio from a multitude of multi-modal sources enabling customization and scale. -
17
WanGP
AI video generator optimized for low VRAM and older GPUs use
Wan2GP is an open source AI video generation toolkit designed to make modern generative models accessible on consumer-grade hardware with limited GPU memory. It acts as a unified interface for running multiple video, image, and audio generation models, including Wan-based models as well as other systems like Hunyuan Video, Flux, and Qwen. A key focus of the project is reducing VRAM requirements, enabling some workflows to run on as little as 6 GB while still supporting older Nvidia and certain AMD GPUs. ... -
18
AudioLM - Pytorch
Implementation of AudioLM audio generation model in Pytorch
Implementation of AudioLM, a Language Modeling Approach to Audio Generation out of Google Research, in Pytorch It also extends the work for conditioning with classifier free guidance with T5. This allows for one to do text-to-audio or TTS, not offered in the paper. Yes, this means VALL-E can be trained from this repository. It is essentially the same. This repository now also contains a MIT licensed version of SoundStream. -
19
Dia2
TTS model capable of streaming conversational audio in realtime
...It currently focuses on English generation and supports up to two minutes of generated audio. Its main value is enabling low-latency, dialogue-oriented TTS workflows where timing, turn-taking, and natural conversation matter. -
20
HunyuanVideo-Foley
Multimodal Diffusion with Representation Alignment
HunyuanVideo-Foley is a multimodal diffusion model from Tencent Hunyuan for high-fidelity Foley (sound effects) audio generation synchronized to video scenes. It is designed to generate audio that matches both visual content and textual semantic cues, for use in video production, film, advertising, games, etc. The model architecture aligns audio, video, and text representations to produce realistic synchronized soundtracks. Produces high-quality 48 kHz audio output suitable for professional use. ... -
21
LatentSync
Taming Stable Diffusion for Lip Sync
...The system leverages a U-Net diffusion backbone, with cross-attention of audio embeddings (via an audio encoder) and reference video frames to guide generation, and applies a set of loss functions (temporal, perceptual, sync-net based) to enforce lip-sync accuracy, visual fidelity, and temporal consistency. Over versions, LatentSync has improved temporal stability and lowered resource requirements — making inference more practical (e.g. 8 GB VRAM for earlier versions, somewhat higher for latest models). -
22
MOSS-TTS Family
MOSS‑TTS Family open‑source speech and sound generation model
...Its flagship model focuses on stable long speech generation, multilingual and code-switched synthesis, pronunciation control, and zero-shot voice cloning. The broader family also includes dialogue generation, prompt-based voice creation, streaming voice-agent support, and a unified audio tokenizer. It is especially useful for developers building dubbing, podcasts, audiobooks, voice assistants, character voices, and creative audio tools. -
23
HunyuanVideo
HunyuanVideo: A Systematic Framework For Large Video Generation Model
HunyuanVideo is a cutting-edge framework designed for large-scale video generation, leveraging advanced AI techniques to synthesize videos from various inputs. It is implemented in PyTorch, providing pre-trained model weights and inference code for efficient deployment. The framework aims to push the boundaries of video generation quality, incorporating multiple innovative approaches to improve the realism and coherence of the generated content. Release of FP8 model weights to reduce GPU... -
24
MusicGPT
Generate music based on natural language prompts using LLMs
MusicGPT is an open-source application designed to generate music from natural language prompts using locally executed artificial intelligence models. The software allows users to run advanced music generation systems directly on their own devices without requiring heavy dependencies such as Python or full machine learning frameworks. Instead, it provides a lightweight environment capable of executing music generation models locally on CPUs or GPUs while maintaining strong performance across operating systems including Windows, macOS, and Linux. ... -
25
Scribbletune
Create music with JavaScript
Scribbletune is a JavaScript library for creating music and sequences using a simple and intuitive syntax, allowing developers to generate MIDI files and integrate music composition into their applications.