Audience
Musicians, artists, and anyone looking for a tool to generate audio and sounds from written text
About AudioCraft
AudioCraft is a single-stop code base for all your generative audio needs: music, sound effects, and compression after training on raw audio signals. With AudioCraft, we simplify the overall design of generative models for audio compared to prior work. Both MusicGen and AudioGen consist of a single autoregressive Language Model (LM) that operates over streams of compressed discrete music representation, i.e., tokens. We introduce a simple approach to leverage the internal structure of the parallel streams of tokens and show that, with a single model and elegant token interleaving pattern, our approach efficiently models audio sequences, simultaneously capturing the long-term dependencies in the audio and allowing us to generate high-quality audio. Our models leverage the EnCodec neural audio codec to learn the discrete audio tokens from the raw waveform. EnCodec maps the audio signal to one or several parallel streams of discrete tokens.
Other Popular Alternatives & Related Software
SFX Engine
Discover the power of our AI sound effect generator, designed specifically for audio producers, video editors, and game developers. Our AI sound effect generator empowers you to craft custom audio experiences that resonate with your audience. With endless possibilities, you can easily design the perfect sound for any project, whether it's for film, gaming, or music production. Fine-tune every sound effect with detailed text descriptions, allowing for precise customization to suit your needs. Our pricing is simple and transparent, with no hidden fees or charges. Purchase as many credits as you need, no subscription necessary. Generate any sound effect with infinite variations. Pay only for the sound effects you need. All commercial use is included by default. Every sound effect you generate is licensed for commercial use, with no additional fees or royalties. Use them in your projects without worry.
Learn more
MusicGen
Meta's MusicGen is an open source, deep-learning language model that can generate short pieces of music based on text prompts. The model was trained on 20,000 hours of music, including whole tracks and individual instrument samples. The model will generate 12 seconds of audio based on the description you provided. You can optionally provide reference audio from which a broad melody will be extracted. The model will then try to follow both the description and melody provided. All samples are generated with the melody model. You can also use your own GPU or a Google Colab by following the instructions on our repo. MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models. MusicGen can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better control over the generated output.
Learn more
OpenAI Jukebox
We’re introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artistic styles. We’re releasing the model weights and code, along with a tool to explore the generated samples. Provided with genre, artist, and lyrics as input, Jukebox outputs a new music sample produced from scratch. Jukebox produces a wide range of music and singing styles and generalizes to lyrics not seen during training. All the lyrics below have been co-written by a language model and OpenAI researchers. When conditioned on lyrics seen during training, Jukebox produces songs very different from the original songs it was trained on. We provide 12 seconds of audio to condition on and Jukebox completes the rest in a specified style. We chose to work on music because we want to continue to push the boundaries of generative models. Jukebox’s autoencoder model compresses audio to a discrete space, using a quantization-based approach called VQ-VAE.
Learn more
AudioLM
AudioLM is a pure audio language model that generates high‑fidelity, long‑term coherent speech and piano music by learning from raw audio alone, without requiring any text transcripts or symbolic representations. It represents audio hierarchically using two types of discrete tokens, semantic tokens extracted from a self‑supervised model to capture phonetic or melodic structure and global context, and acoustic tokens from a neural codec to preserve speaker characteristics and fine waveform details, and chains three Transformer stages to predict first semantic tokens for high‑level structure, then coarse and finally fine acoustic tokens for detailed synthesis. The resulting pipeline allows AudioLM to condition on a few seconds of input audio and produce seamless continuations that retain voice identity, prosody, and recording conditions in speech or melody, harmony, and rhythm in music. Human evaluations show that synthetic continuations are nearly indistinguishable from real recordings.
Learn more
Integrations
No integrations listed.
Company Information
Meta AI
Founded: 2004
United States
audiocraft.metademolab.com
Videos and Screen Captures
Other Useful Business Software
MongoDB Atlas runs apps anywhere
MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
Product Details
Platforms Supported
Cloud
Training
Documentation
Support
Online
