A few days ago, I had a conversation with a friend who is deeply involved in the music design business for video games. After an undergrad degree in movie music composition, he got involved in the core design team for a major game sound platform.
This is a two-part blog entry. In this, I’ll talk a little about the unfortunate state of the game music industry—as summarized to a non-expert, over a beer or two, after a long day of doing other things. Again, I'm NOT AN EXPERT: I don't even play a lot of games. Rather, a conversation has stirred thoughts, and I want to talk about them.
In the next, I’ll blue-sky for a few little bit about possible designs for more adaptive, dynamic music in game systems. Neither, I’m afraid, will be explicitly about anything being made out of people: however, this entry is about the social process involved in game design; the next entry is about the use of creative design for sound systems, based loosely on some of my various work.
My friend was discussing his work as a designer and as a consultant. He’s the program manager for the system, but spends a lot of time going to trade shows and game companies, evangelizing the facilities that his system provides, discussing the technical aspects of it, and exploring different ways that it can be used. His API essentially makes it easier to connect game events to sounds, and thus allows the composer’s job to be less programming-intensive.
This helps with some of the basic issues: it provides a vocabulary for wiring game events to sounds, so that it’s easier to code “the enemy fired a bullet, forward and to the right”. (*Update*: In case I wasn't clear before, this is pretty classic. In lots of games, audio is a crucial gameplay aspect in figuring out what's going on, is is a useful second cue. I'm more thinking about whether audio can be more sophisticated than the fairly realist take of "I step, and it goes 'thock.'")
Unfortunately, game audio is too often an afterthought. He complained about game designers who bring in music “because it’s missing something; someone has to make it fun,” who hire composers after the game is nearly finished, or who just ship out the sound design to a couple of interns.
Thus, in a world where the game developer has the resources to locate 3-D sound anywhere around the player, who can simulate curtains and echoing walls and footsteps, who can generate music on the fly, the designers often settle for a loop of music in the background and a line of recorded sounds in the foreground. This makes for a fairly dull experience, when sound might conceivably lighten the game up and add a great deal of subtle excitement.
Subtle, of course, is the critical word. It’s easy to see a good graphics system, and it’s fairly quick to observe good game AI. But what does good sound sound like? “Well, it’s CD quality” doesn’t cut it: is it exciting? Is it fun? Does it build on the game experience? Unfortunately, humans are far more sensitive to—or at least have better vocabulary for describing—video than audio, and so the music is often under-evaluated. Press kits include videos and pictures, but never game sounds without pictures. While game soundtracks are becoming popular, that’s more of a celebrity music track than it is music as part of the game. (My friend pointed out that many game reviews never bother to mention the sound at all, or limit themselves to words like “realistic” and “amusing”.)
The unused potential is nearly infinite. Think of a character walking behind a curtain ringing a bell. As the player moves around, a good ray-trace should be able to determine whether the bell sounds local or distant, muffled or loud. Unfortunately, that’s a lot of work—and rather computationally expensive. Yes, some of that information is already available to the game engine: after all, it’s doing some ray-tracing already to figure out where the curtain is, and whether the player can see it. But the graphics-oriented ray tracing system is oriented towards things that are visible, and “behind a curtain, and behind the player” isn’t quite there. (I have seen an old SIGGRAPH paper on ray tracing sound, but no matter how you see it, the system gets expensive: is that payoff to do so much work worth it when you can approximate? “In this room, play the sound quietly; in that room, play it loud”.)
Of course, the bleak picture I paint above isn’t the only thing that’s happening today.
First, music oriented games think hard about music. Amplitude dynamically mixes tracks, and adds other audio cues both to direct the game-play and to provide feedback on success, Dance Dance Revolution uses music to guide the player’s steps (to some extent—I’ll have to blog about that some other time), and uses crowd applause to reward skillful play. Space Channel 5 synchronizes the game around music.
Second, one can do moderately-more sophisticated variants of the linear play. For example, the recorded samples in a game might have slight variants: a footstep might sound randomly slightly different; a gun cartridge casing might eject with either a “plink” or a “thunk.” The music might vary level by level, or even region by region. And ambient music can be played out in different ways. For example, many games have an “outdoor ambient” track: it’s the birds chirping, the crickets humming, and the wind blowing. While the game may use up to a minute or so of this track, they often simply repeat the track over and over. Users begin to notice this, and start to get bored: “ah, the crickets have chirped. Next, we’ll hear some wind.”
In contrast, Microsoft’s “Halo” works slightly differently. The same minute has been recorded, but now it’s sliced into five-to-ten second segments. Those segments are played back in a largely random order, which creates a constant sense of novelty.
Last, one can synchronize the music with the action to create a more integrated experience. It’s this, actually, that I’m most interested in, and will be discussing more in my next entry. But to start off, there’s a couple of variants that are worth noting: one is the ways of cueing different sorts of music to the action. For example, “Splinter Cell” plays different music when enemies have spotted you and are actively hunting, as opposed to when you are unspotted. (It also plays louder footsteps for running than walking, and so gives some feedback to how audible you might be to an opponent).
My friend also discussed a snowboarding game, with DJ-mixed music in the background. When the player did a trick, the system always made sure that there was a solid downbeat at the moment the snowboard hit the snow again—which added a bit of emphasis to the jumps. (Turns out the system also played higher-pitched, quieter music during the jump, which helped ensure there was greater contrast upon landing.)
I think that’s a decent non-expert, birds-eye taxonomy of game sound. In my next entry, I’ll discuss some of my thoughts in those last categories: game music linked to game play.
July 9, 2004 04:21 PM | TrackBack | in Design