Everything Is Music to Instrument Inventor Akito van Troyer’s Ears

Akito van Troyer's latest instrument is not a drum machine, exactly. More like a drumming machine.

Laid out on a table at the Electronic Production and Design (EPD) professor's Cambridge workspace, van Troyer's briefcase-sized prototype is surrounded by a striking mishmash of objects: containers and lids of all sizes, a box of 3D-printed plastic cups and saucers, an egg shaker, a glass ball, a styrofoam block, two decorative tin pails, a handful of playing card–sized metal plates, and a festive-looking fake pumpkin. Van Troyer has placed an object on each of the instrument's eight square pads, which have mechanical actuators that will strike these objects in a programmed sequence when he presses play on the connected MIDI controller.

He hits a button and the instrument's surface pings and rattles and shakes and thumps with the found-sound rhythm, which he can alter with further tweaks to buttons and sliders.

"The biggest thing this instrument can do is turn everyday objects into musical materials," he says. The result is a kind of dream laboratory for anybody who's ever drummed on random objects and surfaces just to hear how they sound. He calls it the Garakutakara—a play on a Japanese term that means "junk treasure." And it's the latest development in a series of sonic explorations that have fascinated van Troyer for years.

The Search for New Sounds

For as long as van Troyer has pursued music, he has sought out new sounds and new ways of making those sounds. "There are all kinds of music out there, and I wanted to experience them," van Troyer explains. "I want to create the most original sounds. Original sound counts when you're making new music."

As an undergraduate studying ethnomusicology at the University of Hawaii, van Troyer focused on Indonesian gamelan music. Gamelan is a rich and complex system filled with instruments, tones, and scales that have developed independently of Western musical traditions. At the same time, he began to explore "computer music," and discovered that the two interests could complement each other. For his senior thesis, he used a computer to analyze the sound of each instrument in a gamelan ensemble, then he created a new electronic gamelan composition with the digital versions of those instruments.

Van Troyer took this fascination with the intersection of technology and sound design through the rest of his education, from a master's degree in music technology at Georgia Tech to a second master's and Ph.D. in media arts and sciences from the Media Lab at MIT. He continued to develop new virtual sounds and instruments while also beginning to explore how to use technological tools to make music with physical objects.

Instrument Builder

The first physical instrument van Troyer built was a mechanically augmented cello that took advantage of a small but significant part of the instrument's construction. Cellos have a piece called a sound post, a wooden rod inside the body that has a critical influence on the instrument's timbre. Knowing this, van Troyer built "a robot that changes the sound post position on the fly to change the timbre of the cello as the cello player plays."

Other instruments he's built or collaborated on have included a MIDI-enabled microtonal harpsichord, a percussive geodesic dome, and a "tactiloscope"—a work-in-progress device that would translate sound into a touchable interface for hearing-impaired users.

Van Troyer also began building instruments with platforms that would strike or resonate materials the musician puts on top of them, like the Garakutakara. There's another one like this on the table in his workspace that functions differently, sending out electromagnetic pulses that vibrate metal pieces inside the materials he places on the instrument's surface. He calls it the MM-RT (which stands for "material, magnet, rhythm, and timbre"), and it won the People's Choice: Most Unusual award at the Guthman New Musical Instrument Competition in 2017.

Watch Akito van Troyer demonstrate his award-winning MM-RT for Atlanta Magazine:

Making Connections

Running through many of van Troyer's musical creations is a deep desire to create connection—between the musician and the instrument, between the performer and the audience, and between the sound and the source.

"Think about a guitar," he says. "When you play it, it's in your face: the guitar is right here, and the sound comes from there, and the [performer's] gestures directly result in the sound. So it makes sense for the audience members and it's fun to watch."

By contrast, he explains, with a conventional electronic musical instrument, the gesture—say, pressing a button on a keyboard or writing a few lines of code on a computer—is separated from the sound it produces, which comes out of a speaker somewhere else. This is the problem van Troyer's new instruments are confronting head-on.

Everyday Music

With instruments like the Garakutakara and the MM-RT, van Troyer is bringing the sound closer to the gesture. An audience member can see him place the tin pail on one of the pads, then hear the pail start to pound like a steampunk kick drum. Then, they can watch as he selects the festive fake pumpkin and slowly lowers it by the stem into the pail, and hear the transformation as each clang sounds more like a thunk. There's suspense, there's humor, and there's even a kind of narrative that connects his gestures to the sounds the audiences hear. "In a way, I'm doing storytelling, very abstractly," he says.

Ultimately, van Troyer sees this kind of performance as an opportunity to invite audiences to pay more attention to the sounds all around them every day. "If you're in the kitchen or in the office, there are different objects you interact with," he explains, saying he wanted to transform the sounds that come from those "task-oriented gestures" into musical moments.

One audience that understands van Troyer's aim instinctively: kids. He did a demonstration for a group of children recently, and they couldn't wait to try it for themselves. "They went nuts," he says. "I dressed like a scientist, and all the objects I had were lab equipment, like glass bottles and things like that. And then I would transition into a chef next. Then it's all kitchen gear. It's something you do every day, translated into a musical context."

Teaching the Tech

Of course, there's a paradox at the core of van Troyer's work: while he's designing instruments and coding programs that are meant to be intuitive and seamless for audiences and players—even children—the technology underpinning these user experiences has to be highly sophisticated. That's where his courses in the creative coding minor come in.

"A lot of students coming into my courses . . . want to make plugins. That's going to be the next big thing. But the gap to be able to make plugins is so high," he says. The creative coding minor lays the groundwork necessary for students to program the underlying technology necessary for these kinds of musical innovations.

With the advent of broadly accessible generative AI, van Troyer sees tremendous potential for students to "do more with less time at Berklee" using tools like GitHub Copilot to help them code more ambitious and complex musical programs. "AIs can take care of these kinds of minor detail programming tasks and then students can focus on more high-level stuff," he says, allowing them to "make bigger applications than they could've done in half a year."

Endless Exploration

As in all technological endeavors, though, much of van Troyer's work still depends on a series of trials and refinements. There is no AI (not yet, anyway) that will tell you what density of material to use for the instrument's percussive surface, say. Or which waveform best maps onto the way the actuators should hit the objects to get the best sound (a sawtooth wave, it turns out).

"I have to do the experiment to figure those kinds of things out," he says, and it's hard to imagine he minds the prospect of another experiment, another new sound to test out.

That sort of relentless curiosity about how sounds are shaped has particular value within electronic production and design, says van Troyer. "A lot of students are interested in doing sound design, and when you're doing sound design, it's really important to keep your ear open to what's around you. Not just for music, but generally speaking, day-to-day, or when you go into nature," he says. "What is that sound? What made that sound?"