Musical Experiments Inspire New Video Game Music Ensemble
Image by Tess McCumber
06/07/2022
By Katharine Webster
Sound recording technology (SRT) major Tess McCumber ’22 grew up playing Pokémon games on her Nintendo DS, and then also became obsessed with the massively popular game Minecraft.
So when she heard that students in the Music Department’s Contemporary Electronic Ensemble, led by Assoc. Teaching Prof. Ramon Castillo, were creating music videos using Minecraft, she joined up.
She says the practical experience of synchronizing sounds to video games built on what she had learned in classes required for her major – and gave her a better understanding of electric circuits that will help her in her new career as a sound technician at AudioLink, a company that provides voiceover recording and production services.
“We had to take electrical engineering courses about sound technology, but I didn’t really understand circuitry until we did this hands-on project in Minecraft,” McCumber says. “Seeing it visually, seeing how wiring and electronics and machines work, was the biggest takeaway.”
Image by Ramon Castillo
“Lavender Town” was among 18 UMass Lowell projects featured at the all-video, online 2022 Maker Music Festival, which drew participants from around the world. While most create their own instruments, both analog and electronic, organizer Sherry Huss was happy to accept digital performances this year because the festival was virtual, due to the COVID-19 pandemic.
Castillo encouraged students in his Contemporary Electronic Ensemble class to participate in the festival, which debuted the weekend of May 14-15. Nearly a dozen students did, and Castillo ended up organizing an entire virtual UML building. He spoke about several of the projects in a real-time session during the festival, and he will speak and perform live at A New HOPE hackers' conference in New York City in July, as well.
In addition to the Minecraft videos, the UML building at the Maker Music Festival features a video about the EcoSonic Playground Project, a pair of live performances on original or reconstructed instruments, and two video projects by students from Prof. Gena Greher’s Technology in Music Education class, as well as several of Castillo’s original compositions and collaborations with his 8-year-old daughter, Luna.
Four of the UML projects won awards at the festival, including “Lavender Town” and the EcoSonic Playground.
Image by Kyle Kashiwabara and Matt Makuc
Castillo was also inspired by his daughter, who plays the educational, multiplayer version of Minecraft, and the depth of electronic music possibilities the game offers.
“You could see the other person and interact with them in real time, and that’s exactly what we lost during remote learning,” Castillo says. “So using Minecraft in the ensemble seemed like a way we could get that back, at least to a degree. Every project the students created had to have an element of live performance, too.”
The live performances took place at the end of every semester on the social media platform Twitch.
Castillo had already begun some musical Minecraft experiments of his own, a group of works he calls “void loop ().” In “Chicken Hero, Villager Hero, Cow Hero,” chickens, villagers and cows fall from different heights onto virtual pressure plates that respond to their “weight” by triggering in-game sounds and actions, via circuits constructed of “redstone,” an electrically conductive element within the game.
Image by Tristan Fruzzetti
Vocal performance major Kyle Kashiwabara and composition for new media major Matt Makuc used stop-motion animation in “Motorbike,” another award winner, to simulate a motorcycle traveling along a road. As it travels, it activates layers of redstone-connected sound and command blocks. (Command blocks contain a line of code that can make a more complex sound or prompt another action within the game.)
Some students created both visual and musical accompaniments to live performance. In “Pillager Jam,” which also won an award, SRT major Tristan Fruzzetti created a video with an audio track and visuals from Minecraft and Logic digital recording software to accompany his solo performance on electric guitar.
The Minecraft musical projects have proved so popular that Castillo is starting a new ensemble in the fall, the Video Game Ensemble. The Music Department is funding a dedicated Minecraft server that Castillo and his students can use and modify. Castillo has already built a virtual replica of UML’s Durgin Hall inside it.
A good friend of Castillo’s, Alex Rigopulos, the co-founder and creative director of Harmonix Music Systems who helped design the original Guitar Hero and Rock Band games, has made a generous donation to fund some add-ons and other technology for the new ensemble.
Image by Ryan Katz
Ryan Katz, an SRT major and rising senior who has played in the ensemble for four semesters, stuck with creating and adapting electronic instruments and sound banks during the pandemic, rather than migrating to Minecraft.
One of his videos in the 2022 Maker Music Festival, “Interstellar Bone Collector,” features a live, improvised piece he performed with two friends, using a Yamaha keyboard he’d rewired to create new sounds, a practice known as “circuit bending.”
His other video features “The Meteorologist,” an instrument he invented to respond to the weather, using sensors that measure atmospheric and ground conditions including light, temperature and soil moisture. A human player adds more chaotic and percussive sounds by manipulating a joystick, knob and trackpad.
Although “The Meteorologist” starts by playing a simple chord progression, it never plays the same piece twice, Katz says, because it responds to the weather and human touch at that time and location.
“I wanted an intersection between what we've created and what the environment creates,” he says.
