In my last entry, I talked about a variety of game sounds, and discussed some of the technologies for adaptive music systems linked to sound. In this entry, I’d like to spend a few minutes on some of the ideas that I see growing from that.
In my last discussion, I emphasized “subtle”. Here’s part of why. My friend had talked about cartoon-type sound tracks: when you walk up the stairs, for example, the game would play “stair climbing” music, perhaps with each successive note getting higher: “plunk plonk plenk plink!” When you walked down, it would play it getting lower. Users want power over their games, and many try to explore every inch of the game world, and—wow! That’s a mini-game. What happens when I walk half-way up, then turn around? And jump off? And then walk up again? The user, now actively playing the game of “learn about the sound system,” has forgotten about the underlying game of “shoot the enemy,” and is now wandering around the sound system.
Presumably, that's not realy a game goal, or a direction the designer is trying to go. And it's a distraction, rather than an enhancement, to the game.
So let’s grant that this is a cartoon world. Give me the goal of generating Carl Stalling1 music: a good solid trombone “thwaap!” when you throw a pie; a “sneaky” theme when you get on tip-toes, and so on. Carl Stalling, however, had an advantage that no game player ever will: he knew what was happening next. And as such, a lot of his music anticipates the next step: the trombone subtly sneaks out of the theme so that it can be ready with the surprising “thwaap!”; the touch of cymbal anticipates the sound of sneaking around the corner, and so readies you for the rimshot when you are discovered.
But when a character is wandering around the world, you can’t be sure that he will be compliant. How do you ensure that the pie will be thrown on beat? How can you know whether the AI will catch him (and so the sneaking / rimshot combination is correct), or whether he’ll make it past (and so a different music set might be more rewarding)? In other words, a movie composer can see into the future, and a game designer can’t.
There are certainly clever solutions--I talked yesterday about a skating game that melded into a (beat-less) riff at the beginning of a jump, and so could hit a downstroke when the board touched ground. I also got a comment on the last article, discussing a game with different zones, and a bunch of bridges between them. As the player moved between zones, the different bridges would smoothly play. Which meant that the music would shift seamlessly as the player wandered about the world.
But can we do better?
A number of years ago, I saw a dynamic adaptive music system out of Microsoft Research. I don’t recall the name offhand, but it was a research project that allowed a user to start with a basic musical theme. It would then produce simple variations on that theme based on notions like “louder” or “faster” or even “more exciting” and “funkier.” As you played the music, you could order it to get solemn, and the system would smoothly transition to playing solemnly.
So in my mind, an adaptive sound system could use that sort of idea to generate a system that pays attention to the user’s state. Like Splinter Cell, it might play different music for “seen” or “visible”—but it might do better than that. Let’s look at a few dimensions that really are available to the system. Let’s pick a simple first-person shooter with a stealth component (say, “Counterstrike” or something like that).
You can even add an aspect of judging the gameplay:
And so on. I think you can probably handle this combination of stuff with both ambient music and event sounds. And there might be more than one theme to be explored with its variations. But I think that setting up a couple of axes to the system could then allow dynamic, unpredictable music to be generated. The composer lays down the theme and works out how its variations will be played—but the game is responsible for putting the pieces together to decide when to play “sneaky” or “sneaky with spotted” and even “sneaky, and sneaking up behind the bad guy.”
My suggestion, then, is that with a system that knows something about how to score different sorts of music--on a subtler level than "louder" or "softer"--you might be able to do this with fewer than the nasty-seeming combinatoric matrix of sounds. Start with a few themes, adaptively twist them for different moods...
Just a thought.
--
I should point out that none of this is new: again, I'm a non-expert. A google search for "adaptive music" finds hundreds of hits; Gamasutra has an interesting article from 2001 about adaptive music, which touches on some of these points..
Indeed, I just noticed Adaptive Audio that engages some of these issues, including two very interesting case studies.
July 11, 2004 01:20 PM | TrackBack | in Design