Proposal Title
Lulling Waters: A Poetry Reading for Real-Time Music Generation through Emotion Mapping
Loading...
Submission Type
Performance
Abstract
Through a poetic narrative, “Lulling Waters” tells the story of a whale overcoming the loss of his mother, who passed away from ingesting plastic, as he attempts to escape from the polluted oceanic world. The live performance of this poem utilizes a software system called Soundwriter, which was developed with the goal of enriching the oral storytelling experience through music. This video demonstrates how Soundwriter’s real-time hybrid system was able to analyze “Lulling Waters” through its lexical and auditory features. Emotionally salient words were given ratings based on arousal, valence, and dominance while the emotionally charged prosodic features of the speaker’s voice (pitch contour, speech rate, and intensity) influenced the final classification of the story’s emotional state. The detected emotions then guided the music to dynamically reflect the story's progress. The system also introduced an interactive scoring algorithm which translated the detected emotions into musical figures for live musicians. “Lulling Waters” was created as a proof-of-concept for our Soundwriter system in combination with the goal to ignite passion for environmental protection through poetry. Research for Soundwriter has been published via the International Computer Music Conference of 2019.
For more on Soundwriter: https://www.toshihisatsuruoka.com/soundwriter
Included in
Composition Commons, Computational Linguistics Commons, Databases and Information Systems Commons, Environmental Health and Protection Commons, Fiction Commons, Music Performance Commons, Poetry Commons, Semantics and Pragmatics Commons, Software Engineering Commons, Syntax Commons, Theory and Algorithms Commons
Lulling Waters: A Poetry Reading for Real-Time Music Generation through Emotion Mapping
Through a poetic narrative, “Lulling Waters” tells the story of a whale overcoming the loss of his mother, who passed away from ingesting plastic, as he attempts to escape from the polluted oceanic world. The live performance of this poem utilizes a software system called Soundwriter, which was developed with the goal of enriching the oral storytelling experience through music. This video demonstrates how Soundwriter’s real-time hybrid system was able to analyze “Lulling Waters” through its lexical and auditory features. Emotionally salient words were given ratings based on arousal, valence, and dominance while the emotionally charged prosodic features of the speaker’s voice (pitch contour, speech rate, and intensity) influenced the final classification of the story’s emotional state. The detected emotions then guided the music to dynamically reflect the story's progress. The system also introduced an interactive scoring algorithm which translated the detected emotions into musical figures for live musicians. “Lulling Waters” was created as a proof-of-concept for our Soundwriter system in combination with the goal to ignite passion for environmental protection through poetry. Research for Soundwriter has been published via the International Computer Music Conference of 2019.
For more on Soundwriter: https://www.toshihisatsuruoka.com/soundwriter
Bio
Ashley Muniz (b.1995) is a writer of words and sound from NYC. Drawing inspiration from electronic literature and the language poetry movement, she aims to fabricate a world both lyrically and musically, posing abstract landscapes, visceral narratives, and philosophical thought experiments. In 2019, her collaborative research paper Soundwriter: Real-Time Music Generation for Oral Storytelling through Emotion Mapping was published via the International Computer Music Conference 2019. Her poetry projects have also been accepted into the Electronic Literature Organization Conference 2020. She pursued studies in Music Business as well as Creative and Professional Writing and received a Masters in Music Theory & Composition at New York University. Her dream is to explore and expand the way in which words play a part and are experienced within a piece of music through the use of digital technology.
Toshihisa Tsuruoka (b.1994) is a sound artist-composer who specializes in electroacoustic works. His most recent collaborations include Prime Winds Ensemble, Consensus Ensemble, Bearthoven Ensemble, Quince Ensemble, Mivos Quartet, NYUO1 Orchestra and many more. As a researcher, Toshi focuses on the intersection of music and technology. In 2019, his research papers titled Soundwriter and Bookscapes Project were published via the International Computer Music Conference. Currently, he is the lead researcher on a project named Ear Talk: Sounds Worth Sharing—a system that allows people to share, shape, and form music on an online virtual space. Toshi holds a masters degree from New York University in Composition where he studied with Justin Dello Joio and Tae Hong Park.