Audience

Audio researchers and developers needing a solution for creating realistic speech and music continuations directly from raw audio

About AudioLM

AudioLM is a pure audio language model that generates high‑fidelity, long‑term coherent speech and piano music by learning from raw audio alone, without requiring any text transcripts or symbolic representations. It represents audio hierarchically using two types of discrete tokens, semantic tokens extracted from a self‑supervised model to capture phonetic or melodic structure and global context, and acoustic tokens from a neural codec to preserve speaker characteristics and fine waveform details, and chains three Transformer stages to predict first semantic tokens for high‑level structure, then coarse and finally fine acoustic tokens for detailed synthesis. The resulting pipeline allows AudioLM to condition on a few seconds of input audio and produce seamless continuations that retain voice identity, prosody, and recording conditions in speech or melody, harmony, and rhythm in music. Human evaluations show that synthetic continuations are nearly indistinguishable from real recordings.

Other Popular Alternatives & Related Software
AudioCraft
AudioCraft
AudioCraft is a single-stop code base for all your generative audio needs: music, sound effects, and compression after training on raw audio signals. With AudioCraft, we simplify the overall design of generative models for audio compared to prior work. Both MusicGen and AudioGen consist of a single autoregressive Language Model (LM) that operates over streams of compressed discrete music representation, i.e., tokens. We introduce a simple approach to leverage the internal structure of the parallel streams of tokens and show that, with a single model and elegant token interleaving pattern, our approach efficiently models audio sequences, simultaneously capturing the long-term dependencies in the audio and allowing us to generate high-quality audio. Our models leverage the EnCodec neural audio codec to learn the discrete audio tokens from the raw waveform. EnCodec maps the audio signal to one or several parallel streams of discrete tokens.
Learn more
MuseNet
MuseNet
We’ve created MuseNet, a deep neural network that can generate 4-minute musical compositions with 10 different instruments and can combine styles from country to Mozart to the Beatles. MuseNet was not explicitly programmed with our understanding of music, but instead discovered patterns of harmony, rhythm, and style by learning to predict the next token in hundreds of thousands of MIDI files. MuseNet uses the same general-purpose unsupervised technology as GPT-2, a large-scale transformer model trained to predict the next token in a sequence, whether audio or text. Since MuseNet knows many different styles, we can blend generations in novel ways. We’re excited to see how musicians and non-musicians alike will use MuseNet to create new compositions! Choose a composer or style, an optional start of a famous piece, and start generating. This lets you explore the variety of musical styles the model can create.
Learn more
MusicGen
MusicGen
Meta's MusicGen is an open source, deep-learning language model that can generate short pieces of music based on text prompts. The model was trained on 20,000 hours of music, including whole tracks and individual instrument samples. The model will generate 12 seconds of audio based on the description you provided. You can optionally provide reference audio from which a broad melody will be extracted. The model will then try to follow both the description and melody provided. All samples are generated with the melody model. You can also use your own GPU or a Google Colab by following the instructions on our repo. MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models. MusicGen can generate high-quality samples, while being conditioned on textual description or melodic features, allowing better control over the generated output.
Learn more
Melodea
Melodea
Generate music based on a mood or tempo. Start with a chord progression and generate melodies. Customize the music to make it your own. Use the AI to generate melodies and harmonies, and then refine the melodies by recording a vocal topline. The generated music is based on hit pop songs. Export as an audio file, multitrack MIDI file, or chord notation. Private and secure; all files are saved onto your device. No signup or login is necessary. Melodea is an AI music generator, that provides melody and harmony ideas for the pro songwriter. Use the AI to generate melodies and harmonies, and then refine the melodies by recording a vocal topline. The generated music is based on hit pop songs. Start with a mood or tempo, or even your own chord progression. Customize the melodies and harmonies to make them your own. Export as an audio file, multitrack MIDI file, or chord notation. Private and secure; all files are saved onto your device.
Learn more

Integrations

See Integrations

Ratings/Reviews

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Company Information

Google
United States
research.google/blog/audiolm-a-language-modeling-approach-to-audio-generation/

Videos and Screen Captures

Play Video AudioLM Screenshot 1
Other Useful Business Software
MongoDB Atlas runs apps anywhere Icon
MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
Start Free

Product Details

Platforms Supported
Cloud
Training
Documentation
In Person
Videos
Support
Phone Support
Online

AudioLM Frequently Asked Questions

Q: What kinds of users and organization types does AudioLM work with?
Q: What languages does AudioLM support in their product?
Q: What kind of support options does AudioLM offer?
Q: What other applications or services does AudioLM integrate with?
Q: What type of training does AudioLM provide?

AudioLM Product Features