AudioCraft

AudioCraft is a comprehensive AI research framework developed by Meta AI for generative audio tasks. It provides a unified codebase for creating music, sound effects, and performing audio compression. The core of AudioCraft relies on a single autoregressive Language Model (LM) that processes compressed discrete audio representations, or tokens, generated by the EnCodec neural audio codec. This approach simplifies generative audio models by efficiently handling long-term dependencies in audio sequences and producing high-quality output. The framework includes specialized models like MusicGen for text-to-music generation and AudioGen for text-to-sound generation. MusicGen can produce diverse and long music samples, including stereo, conditioned on text descriptions or melodic features, and supports longer generations through a windowing approach. AudioGen focuses on generating environmental sounds from textual descriptions. AudioCraft is designed for researchers and developers interested in advanced generative AI for audio, offering a robust platform for experimentation and application development in this field.