Stability AI, known for its AI-powered art generator Stable Diffusion, has launched a new AI model designed to generate sounds and songs. They named the model Stable Audio Open and trained it exclusively on royalty-free recordings.
Stable Audio Open generates audio recordings up to 47 seconds long from text descriptions, such as “Rock beat played in a treated studio, session drumming on an acoustic kit.” They trained the model on approximately 486,000 samples sourced from free music libraries, including FreeSound and the Free Music Archive.
Applications and Customization
Stability AI envisions using the model to create drum beats, instrument riffs, ambient sounds, and production elements for multimedia projects like videos, films, and TV shows. Additionally, it can “edit” existing songs or apply the style of one genre to another.
“A key benefit of this open source release is that users can fine-tune the model on their own custom audio data,” Stability AI noted in a blog post. For instance, a drummer could refine the model using their own recordings to generate new beats.
Despite its versatility, Stable Audio Open has notable limitations. It is not capable of producing full songs, melodies, or high-quality vocals, as it is not optimized for these tasks. Stability AI encourages users seeking these features to use its premium Stable Audio service.
Moreover, the terms of service restrict Stable Audio Open to non-commercial use. It also shows biases due to the training data, performing inconsistently across different musical styles, cultures, and non-English descriptions. “The source of data is potentially lacking diversity and all cultures are not equally represented in the data set,” Stability AI explained.
Addressing Controversy and Copyright Issues
Stability AI has faced challenges, including a recent controversy involving the resignation of its VP of generative audio, Ed Newton-Rex. Newton-Rex left the company due to disagreements over the use of copyrighted works in training generative AI models, which Stability AI defended as “fair use.”
The launch of Stable Audio Open seems aimed at countering negative perceptions while promoting Stability AI’s premium products. As AI-generated music tools gain popularity, copyright issues have come to the forefront. In May, Sony Music sent warnings to 700 AI companies against the unauthorized use of its content, and Tennessee passed the first U.S. law addressing AI abuses in music in March.
Stability AI’s new model offers exciting possibilities for sound generation but also highlights ongoing debates about copyright and AI’s role in the creative process.
See also: Humane Advises Customers To Cease Use Of Charging Case Due To Battery Fire Risk
