Search Results for "audio generation"
Sort By:
Showing 282 open source projects for "audio generation"
View related business solutions-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
1
MiMo Audio
Audio Language Models are Few-Shot Learners
...It supports audio understanding, speech intelligence, spoken dialogue, instruction-following audio generation, and text-to-speech-style tasks. The architecture combines audio tokenization, patch encoding, a language model, and patch decoding to make high-rate audio sequences more efficient to model. Overall, it is useful for researchers and developers experimenting with advanced audio LLMs, speech generation, audio reasoning, and instruction-tuned multimodal systems. -
2
MLX-Audio
A text-to-speech, speech-to-text and speech-to-speech library
...The project provides a straightforward CLI (mlx_audio.tts.generate) as well as a Python API for programmatic generation of audio, including parameters for voice choice, speed, language hints, output format, and sample rate. It includes examples such as audiobook generation to demonstrate long-form synthesis and joined audio segments. On top of that, MLX-Audio offers a modern web interface powered by FastAPI, with real-time waveform and 3D visualizations, file upload, and audio management. -
3
Step-Audio
Open-source framework for intelligent speech interaction
Step-Audio is a unified, open-source framework aimed at building intelligent speech systems that combine both comprehension and generation: it integrates large language models (LLMs) with speech input/output to handle not only semantic understanding but also rich vocal characteristics like tone, style, dialect, emotion, and prosody. The design moves beyond traditional separate-component pipelines (ASR → text model → TTS), instead offering a multimodal model that ingests speech or audio and produces speech accordingly, enabling natural dialogue, voice cloning, and expressive speech synthesis. ... -
4
Text Generation Web UI
Oobabooga - The definitive Web UI for local AI, with powerful features
A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA. Dropdown menu for switching between models. Notebook mode that resembles OpenAI's playground. Chat mode for conversation and role playing. Instruct mode compatible with Alpaca and Open Assistant formats. Nice HTML output for GPT-4chan. Markdown output for GALACTICA, including LaTeX rendering. Custom chat characters. Advanced chat features (send images, get audio responses with TTS).... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Step-Audio 2
Multi-modal large language model designed for audio understanding
...It integrates a latent-space audio encoder, discrete acoustic tokens, and reinforcement-learning–based training (CoT + RL) to enhance its ability to capture and reproduce voice styles, intonations, and subtle vocal cues. Moreover, Step-Audio2 supports tool-calling and retrieval-augmented generation (RAG), allowing it to access external knowledge sources or audio/text databases, thus reducing hallucinations and improving coherence in complex dialogues. -
6
MiniMax H3
MiniMax H3 is a general-purpose, omni-modal generative system
MiniMax H3 is an omni-modal generative system for creating synchronized video and native stereo audio from complex multimodal instructions. It can understand combinations of text, images, video, and audio as generation context. Outputs range from four to fifteen seconds at 24 frames per second, with multiple aspect ratios and resolutions up to 2K. H3-Base supports text-to-video, first-frame, last-frame, and first-and-last-frame generation. -
7
LTX-2.3
Official Python inference and LoRA trainer package
LTX-2.3 is an open-source multimodal artificial intelligence foundation model developed by Lightricks for generating synchronized video and audio from prompts or other inputs. Unlike most earlier video generation systems that only produced silent clips, LTX-2 combines video and audio generation in a unified architecture capable of producing coherent audiovisual scenes. The model uses a diffusion-transformer-based architecture designed to generate high-fidelity visual frames while simultaneously producing corresponding audio elements such as speech, music, ambient sound, or effects. ... -
8
Fun Audio Chat
Large Audio Language Model built for natural interactions
Fun Audio Chat is an interactive voice-first conversational AI platform designed to let users engage in natural spoken dialogue with large language models in real time, turning speech into context-aware responses while maintaining a smooth back-and-forth experience. It combines speech recognition, audio processing, and AI generation so users can speak simply and receive spoken replies, enabling applications such as virtual assistants, voice bots, and hands-free chat interfaces. ... -
9
Fish Audio Python SDK
The official Python library for the Fish Audio API
Fish Audio Python is the official Python SDK for working with the Fish Audio platform. It gives developers a programmatic way to access Fish Audio features such as text-to-speech generation, audio playback, saving output files, and API-based voice workflows. The package is designed for Python applications that need speech generation without manually handling raw HTTP requests. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
OpenHiggsfield AI
A studio for image and video generation
OpenHiggsfield is an open-source browser studio for generating images and videos through multiple AI models from one interface. A single prompt composer handles both media types, while the selected model determines which generation workflow is used. Its catalog includes dozens of image and video models from several providers. Each model exposes only the settings it supports, such as aspect ratio, resolution, duration, audio, batch size, and prompt enhancement. Users can attach uploads or reuse media from previous generations as references, start frames, end frames, video, or audio inputs. ... -
11
Miso TTS
Miso TTS is an 8 billion, highly emotive text-to-speech model
Miso TTS is an advanced 8-billion-parameter text-to-speech model developed by Miso Labs for generating highly expressive and natural-sounding conversational speech. Built on an RVQ Transformer architecture inspired by Sesame CSM, it combines a powerful Llama-based backbone with an autoregressive audio decoder to produce high-quality audio from text. The model supports both standard speech synthesis and voice-conditioned generation using optional audio prompts for voice cloning. Miso TTS generates Mimi audio codes and can leverage conversation history to create more contextually aware and realistic dialogue. Designed for local deployment, it offers watermarking by default to help promote responsible use of generated audio. ... -
12
HeartMuLa
A Family of Open Sourced Music Foundation Models
...For text extraction from audio, it provides HeartTranscriptor, a Whisper-based model tuned specifically for lyrics transcription, which helps bridge generated or recorded audio back into structured text. It also introduces HeartCLAP, which aligns audio and text into a shared embedding space. -
13
Voyager
The open harness for creative work
...Voyager includes built-in Design, Video, and digital audio workstation (DAW) engines for creating graphics, editing videos, producing motion graphics, and composing music without requiring third-party creative software. The platform supports multiple AI models, including Claude, ChatGPT, DeepSeek, and specialized image, video, voice, and music generation models, with customizable agent skills and Model Context Protocol (MCP) integrations. -
14
WanGP
AI video generator optimized for low VRAM and older GPUs use
Wan2GP is an open source AI video generation toolkit designed to make modern generative models accessible on consumer-grade hardware with limited GPU memory. It acts as a unified interface for running multiple video, image, and audio generation models, including Wan-based models as well as other systems like Hunyuan Video, Flux, and Qwen. A key focus of the project is reducing VRAM requirements, enabling some workflows to run on as little as 6 GB while still supporting older Nvidia and certain AMD GPUs. ... -
15
JoyAI-Echo
Pushing the Frontier of Long Audio-Visual Generation
...JoyAI-Echo focuses on text-to-video and multi-shot long-video generation, while image-to-video support is not part of the current release scope. It is most useful for research and experimental video workflows that need synchronized audio, coherent characters, and editable story-level generation. -
16
Voicebox
The open-source voice synthesis studio powered by Qwen3-TTS
...A standout capability is its multi-track timeline editor and supporting audio tools (like trimming and conversation mixing), which let creators compose multi-voice scenes instead of generating single clips in isolation. It is API-first, meaning you can use it as an app for production work or integrate its speech generation into your own software via an API layer. -
17
VoxCPM2
Tokenizer-Free TTS for Multilingual Speech Generation
...It also introduces voice design capabilities, allowing users to generate entirely new voices from natural language descriptions without requiring reference audio. -
18
MOSS-TTS-Nano
MOSS-TTS-Nano is an open-source multilingual tiny speech generation
MOSS-TTS-Nano is a lightweight text-to-speech model designed for real-time voice generation in resource-constrained environments. It is part of the broader MOSS-TTS family and focuses on delivering high-quality speech synthesis with a compact architecture. The model operates efficiently on CPU-only systems, enabling deployment without specialized hardware. It supports multilingual voice cloning and produces high-fidelity audio with low latency. -
19
LiveAvatar
Streaming Real-time Audio-Driven Avatar Generation
LiveAvatar is an open-source research and implementation project that provides a unified framework for real-time, streaming, interactive avatar video generation driven by audio and other control signals. It implements techniques from state-of-the-art diffusion-based avatar modeling to support infinite-length continuous video generation with low latency, enabling interactive AI avatars that maintain continuity and realism over extended sessions. The project co-designs algorithms and system optimizations, such as block-wise autoregressive processing and fast sampling strategies, to deliver real-time frame rates (e.g., ~45 FPS on appropriate GPU clusters) while handling non-stop generation without quality degradation. ... -
20
MOSS-TTS Family
MOSS‑TTS Family open‑source speech and sound generation model
...Its flagship model focuses on stable long speech generation, multilingual and code-switched synthesis, pronunciation control, and zero-shot voice cloning. The broader family also includes dialogue generation, prompt-based voice creation, streaming voice-agent support, and a unified audio tokenizer. It is especially useful for developers building dubbing, podcasts, audiobooks, voice assistants, character voices, and creative audio tools. -
21
Podcastfy.ai
Transforming Multimodal Content into Captivating Multilingual Audio
Podcastfy is an open-source Python package that transforms multi-modal content (text, images) into engaging, multi-lingual audio conversations using GenAI. Input content includes websites, PDFs, youtube videos as well as images. Unlike UI-based tools focused primarily on note-taking or research synthesis (e.g. NotebookLM), Podcastfy focuses on the programmatic and bespoke generation of engaging, conversational transcripts and audio from a multitude of multi-modal sources enabling customization and scale. -
22
Dia2
TTS model capable of streaming conversational audio in realtime
...It currently focuses on English generation and supports up to two minutes of generated audio. Its main value is enabling low-latency, dialogue-oriented TTS workflows where timing, turn-taking, and natural conversation matter. -
23
VoiceStudio
VoiceStudio is the open-source, fully-local ElevenLabs alternative
VoiceStudio is an open-source, local-first platform for speech generation, voice cloning, transcription, dubbing, and audio production. It can clone voices from short reference recordings or design new voices from characteristics such as accent, pitch, age, and delivery style. Video dubbing combines transcription, translation, speaker preservation, speech synthesis, and final video export. It also supports multi-voice stories, audiobooks, dictation, vocal isolation, and speaker diarization. ... -
24
LongCat-Video
Foundational video generation model with 13.6B parameters
...Block Sparse Attention reduces high-resolution inference costs, while multi-reward GRPO training improves visual quality and prompt alignment. The repository includes inference scripts for single- and multi-GPU execution, model-download instructions, and interactive generation examples. It also provides audio-driven Avatar variants for expressive single- or multi-character animation. -
25
h3-metal
MiniMax H3 inference engine for Mac computers
h3-metal is a native MiniMax-H3 inference engine for Apple Silicon focused on generating video and audio locally with Metal. It supports prompt-to-video and prompt-to-audio workflows as well as first-frame, last-frame, image, video, and audio references. An interactive terminal session keeps prompt conditioning, the diffusion transformer, and the video decoder in memory for faster repeated generations. Users can trade speed, quality, and memory through denoising steps, layer counts, reuse modes, token reduction, internal render size, and SSD streaming. ...
