Expanding Sound in Videogame Narratives

The Briefest History

The most basic use of music in videogames is a looped music track. It is a repeating, non-variable soundtrack accompanying our journey through the Mushroom Kingdom in Super Mario Brothers, through Dracula’s castle in Castlevania, and the jungles of Contra. A typical looped track is between 1 or 2 minutes long and continues to repeat until the player cuts off the music by completing an area, dying, or pausing the game. And since most arcade and console videogames from 1980 into early 1990’s require less than 5 minutes in a level, the short length of the music is usually sufficient. Most looped tracks are melodically and rhythmically straight forward and filled with energy (something like the music from Air Man’s level in Megaman 2).

There are very fine iterations of looped tracks in videogames. Recently, orchestras have devoted entire programs to the music of videogame composers. Rock bands like the Minibosses and The Advantage have released albums of videogame music, and of course, there is a lot of people making music with hacked consoles and 8-bit synthesizers.

This old method of composition is a limiting one. Since the music is incapable of reacting to the actions within the game, the composer must try to predict how the game will be played and determine if the music will support or fight a game’s limited actions. This, of course, is done by simple trail and error. First, by watching and playing the game. Then a writing, listening, and reworking process until the composition has engrained itself in some fashion.

For the most part, music in videogames sets a tone. It indicates how happy the Mushroom Kingdom is and how scary Bowser’s castle is. It tells us that the dudes in Contra are heroes in the vein of Rambo or Delta Force. And while the looped track can be compelling—it is hard to think of Super Mario Brothers without Koji Kondo’s playful music—there are countless instances of an annoying looped music track. For example, spending an hour in a point-and-click adventure game, and NOT solving the puzzle. The stasis of music and gameplay can turn any catchy number into a soundtrack for madness. Simply said: most looped tracks were not written for extended exposure, though there are exceptions. Games like Tetris and the recent iPhone sensation, Drop 7 combine the minimalism of a puzzle game with an equally minimal music to create hypnotic gameplay. And for most games with a looped soundtrack, the 1 or 2 minute track provides enough music for a level. Playing through a copy of New Super Mario Brothers for the Nintendo DS, I beat the first three levels of World 1 with the following times: Level 1 – 40 seconds, Level 2 – 2 minutes, 20 seconds, Level 3 – 1 minute. With so many other things to consider, I was unable to consciously listen to the music.

Regardless of the results, no matter how fine the music fits the videogame’s aesthetic, the composition is simply a non-variable loop placed atop of a game. The result is something like a finely tuned muzak station. It’s there, and hopefully you like it and think it fits the environment, but there’s no real problem if you don’t like it because you’re actually just grabbing a coin and going to the next virtual CVS/pharmacy to grab a flower or boomerang.

Thankfully, older games with loops offer straight-forward and streamlined interactions. Usual platformers had jumping, shooting and walking. Developers expect players to be on the move and performing these tasks. Your standard platformers gives you two options: 1) move through the world quickly so that you do not get bored OR 2) move through the world carefully so that you do not get killed so much that you become irritated. And somewhere between boredom and irritation is fun.

The Loudest Emotion

Let’s fast forward twenty-five years to newer models of gaming. As game making has matured, so has the amount of time a player spends in each area, level, or stage. While older game music was a key part of a game’s mood and atmosphere, a lot of newer videogames tend to engage players with sound effects rather than music composition including infamously loud gunshots, explosions, sword clashings, and of course, the screams of the dying. In violent videogames, music has taken a backseat to the immersive sounds of war. Music distracts the player in an more open (you can jump everywhere), three-dimensional environment. First-person shooters require that the player hear bad guys sneaking around and talking on super loud radios.

Games like Fallout 3, Dragon Age: Origins, and Left 4 Dead have done away with the looped soundtracks of old and replaced them with the evermore faceless sounds of film music. Now that videogames can come with every cliché of an action movie, many videogames sound like a forgettable action movie soundtrack. Loud brass plays when the hero does anything heroic, loud brass plays when the villain does anything evil, strings shimmer when the hero’s emotion isn’t obvious enough, quiet “ambient/environmental” music accompanies the player’s journey through large, ambient environments, and sometimes a burst of electronica accompanies the player’s fearless shoot out. It’s enough to make a group of intelligent women led by a priest who likes to read Shakespeare and hang out with indie-game developers bulk at everything else that a videogame has to offer. It’s not because they don’t see the potential.

Everyone knows that no other medium has the means of moving through such vast environments or of emulating discovery the way that videogames do. And with all of these promises and this beautiful feeling of exploring a post-apocalyptic wasteland and the years that one hundred people spent planning and developing the game to a perfectly fun experience, and I am still listening to film music.

An example: When a battle begins in Dragon Age: Origins, music begins to play. Here, music functions as an aural seam between two of the game’s modes: peace time and war time. But the seam is aggravatingly noticeable. A million battles later, I watch the same motion of my party taking out their weapons, the same sounds as they hit their enemies, and the same percussion and 6-note phrase. When will developer’s go beyond signifying big ideas like combat, heroism, or sadness with their music? Not just the big guys, but the little indie guys, too. If drama is to mature in videogames, then music and sound must develop with it.

The Sound of a Tectonic Plate Virtually Shifting

Very few games make sound or music a crucial part gameplay. Like the light designer in a blackbox theater, music has been delegated a technical role in videogames. Music highlights various emotions and moments, sets the tone for Mario and Luigi to jump around, and inspires that second wind in battle.

I don’t think that music should be limited to speaking for Gordon Freeman’s intense battle. I doubt the effectiveness of using music to say, “This is extremely intense,” or some other emotion like, “This is sad.” I think music in videogames should be given the chance to speak for itself, to evolve with events and evoke subtler, more pertinant ideas than “bad, sad, glad, mad.”

I do not mean that videogames have not effectively made use of prerecorded music. In Bioshock, the player walks through a wasted, underwater utopia, whereupon the player can hear Django Reinhardt playing on the radio. It doesn’t effect the gameplay, but it adds that nice bonus of being in a world that was well-developed. Another: I get in a car in Grand Theft Auto 4, the radio plays the O’Jays or Terry Riley, and that’s fine. But this is an immitation of music’s place in a real world while everything else my character sees and does is supernatural or an exaggeration. Our gaming avatars kill virtual millions, build giant cities and destroy them, and with the simple wink of an RPG speech-making skill, converse and convince multitudes. So why place music apart from the god-like powers we are given?

Before videogames, the frail intellectual introvert was spending time at a piano composing symphonies and operas or sitting with some words and writing novels. Now we are programming our computers to “[translate] seismic activity, geomagnetism, cloud cover and visibility, and the movements of the sun and moon into soundscapes and changes of light.” I’m talking about a sound and light installation by John Luther Adams entitled, “The Place Where You Go to Listen.” Kyle Gann writes in his blog, Postclassic:

The image… is a set of five glass panes altogether 20 feet wide and 9 feet tall, coated with a diffusion surface and illuminated by LED floodlights that make it a rear-projection screen for fields of pure color. The colors change with the angle and position of the sun in real time. Bell sounds are activated by movements of the aurora borealis, noises ebb and flow with seismic activity, and harmonic series’ track the phases of the moon. It’s all done by computer, with data piped in from seismological stations and whatnot, and is intended to make the viewer/listener aware of where the earth fits into its environment and what it’s doing.

Now substitute “seismic activity” with the movement and position of objects and computer controlled players, “geomagnetism” with a percentage of the virtual world that has been explored, “cloud cover and visibility” with… you can probably keep that, and “movements of the sun and moon” with time elapsed within the world, and I don’t know see what it sounds like. What? Why would videogame music suddenly mimic sound installations? Who, besides art students and their college professors, would listen to a drone modulate for the time it takes to beat a videogame? But then again, how many videogamers are going to Wagnerian operas to listen to the sentimental string symphonies that accompany our current videgoames?

A videogame with interactive sounds means a shift away from using music as the emotional sound effects. It means embracing the abstract qualities of sound and finding narratives more subtle than “good, evil, neutral” and learning to work with a world more complex than a short clips between killing could create. It means thinking about how to make sound an integral part of a psychologically engaging atmosphere. Where a “hunch” could be heard as opposed to told. Where a story or an idea or maybe even something complex enough that it does not need to be a story and isn’t trying to be, maybe this could show itself through sound rather than a swooping cut-scene camera angle. Not because cut-scenes are bad, but because we should try.

On the Other Hand

Michael Abbott, author of the gaming blog Brainy Gamer, has likened playing videogames to acting in theater.

Maybe videogames have the potential to move past the acting role that videogames assign us, but as of right now, one motif is learning to play roles. It’s a fine idea, but often it feels that we are continually put in situations that have us singing and acting in an opera where the orchestra is performing flawlessly but you’ve never heard it, and you sort of stumble through just enough to beat the level.

But on the other hand, sometimes it works. Sometimes it’s not just rethinking everything, it’s assigning a familiar role. Like being an action hero in a futuristic dystopia called Half-Life 2.

So. You’re Gordon Freeman walking down the street with a gun. Alright. You’re improvising it. You see a person shooting at you. You got them. Shooting was easy—you know this role. You meet up with Barney. He’s funny and he likes you. You don’t have to say anything. It’s cool. And of course, Half-Life 2 is six years old now. Those textures looks a little boring now, and you’ve heard this speech before, so in an act of frustration, you jump around and pick up computers, and act the royal ass waiting for Barney to finish saying, “Good grief” and “Great to see you, Gordon.”

You’re on the road again with three new weapons and a boatride behind you. Great boat, good vibes, and we’re back in it. So now you’re walking through an abandoned rail yard, when a swell of loud music indicates heroic roleplaying: you’ve walked into a goon ambush! While the music pulse pops and guns go and go, somehow you’ve starting acting in an opera. The orchestra is performing perfectly and encourages you to perform your role equally well (just don’t die). You’ve spent hours honing those hands and eyes to the crosshair, and now you are in big scene number 1. You’re a hit!