AI-driven lyric-generating systems are often created by training machine learning models on massive datasets of lyrics from diverse genres and performers. Following that, these models can generate lyrics based on input prompts, topics, or styles. They have the ability to help songwriters by making new recommendations, overcoming writer’s block, and even creating whole lyrics for songs. Waterloo researchers’ technology can help musicians find inspiration and new creative possibilities for their songwriting.
Members of the University’s Natural Language Processing Lab developed LyricJam, a real-time system that employs artificial intelligence (AI) to generate lyric lines for live instrumental music. For several years, the group has been studying innovative uses of AI, lead by Olga Vechtomova, a Waterloo Engineering professor who is also cross-appointed in Computer Science.
The early work of the lab resulted in the development of a system that learns the musical expressions of artists and writes lyrics in their own manner. Vechtomova recently developed technology alongside Waterloo graduate students Gaurav Sahu and Dhruv Kumar that uses many components of music such as chord progressions, tempo, and instrumentation to synthesize lyrics that match the mood and emotions represented by live music.
One unexpected finding was that participants felt encouraged to improvise by the generated lines. For example, the lines inspired artists to structure chords differently and take their improvisation in a different direction than intended. Some musicians used the lines to see if their improvisation produced the intended emotional effect.
Olga Vechtomova
As a performer or band performs instrumental music, the system gets raw audio snippets that the neural network analyses to generate new lyric lines. The lines can then be used by the musicians to create their own song lyrics.
“The purpose of the system is not to write a song for the artist,” Vechtomova adds. “Rather, we want to assist artists in realizing their own creativity.” The technology generates poetic lines with fresh metaphors and expressions, perhaps directing the artists in new creative directions.”
The neural network designed by the researchers learns what lyrical themes, words and stylistic devices are associated with different aspects of music captured in each audio clip. For example, the researchers observed that lyrics generated for ambient music are very different than those for upbeat music.
In the field of artificial intelligence and natural language processing, there were continuous breakthroughs that may potentially enable real-time lyric production to inspire composition. The research team conducted a user study, asking musicians to use the system while playing live instruments.
“One unexpected finding was that participants felt encouraged to improvise by the generated lines,” Vechtomova added. “For example, the lines inspired artists to structure chords differently and take their improvisation in a different direction than intended. Some musicians used the lines to see if their improvisation produced the intended emotional effect.”
Another finding from the study highlighted the co-creative aspect of the experience. Participants commented that they viewed the system as an uncritical jamming partner and felt encouraged to play their musical instruments even if they were not actively trying to write lyrics.
Since LyricJam went live in June this year, over 1,500 users worldwide have tried it out.
