Sakimoto and Adaptive Music

[Daniel DeCastro]

Would anyone know if Sakimoto or any other Japanese composer has ever discussed "Adaptive Music"? In speaking to professors about my thesis, this approach to game music composition is something I think was conceptualized here in the West. I'm not a big fan of the idea of adaptive music as I believe it takes away from the quality of the music, and begins to have the same limits as in film music, but I'd like to know if Sakimoto and others know anything about it.

[Jormungand]

What's adaptive music?

[Fatigue]

Moved posts into new thread.

[Daniel DeCastro]

Adaptive music

http://www.gamasutra.com/resource_gu...5/clark_01.htm

Sorry fatigue, thanks for moving the thread.

[Daniel DeCastro]

Here's another article that I found to be quite outrageous..

http://www.gamasutra.com/features/20...orton_01.shtml

The guy basically seems to be saying that all cut scene music is like film music, and that looping ideas is a big mistake.

I'd really like for someone like Sakimoto to refute this notion, because what I honestly feel these guys are saying here is that "We lack Sakimoto's talent therefore, it is a big mistake for us to write like he does". I'm sorry but there are a plentitude of Japanese composers and a couple of western composers who would refute this notion (Richard Jacques, Jeremy Soule, Rob King).

Maybe I am reading too far into this, but I usually just state it as I see it.

[Jormungand]

Mmm, gotta sign up for that link, and I don't really care enough to do so. I'm guessing what this is is having tracks that activate during specific scenes or events triggered in-game? Like...Rogue Squadron! Random example, but off the top of my head and applicable.

Anyway, I agree this is likely more of a North American convention. As for looping versus this, I don't really think it makes a difference in the long run...good music is good music. Or consider something like Soukaigi or Napple Tale: the music does loop in-game, but virtually all tracks were composed as standalone music.

I personally find looping music to be a chore when I compose. I mean, the actual composing part is easy, I don't generally have trouble writing a fluid transition back to a beginning point...it's the technical aspect that kills me. You know, you've got a loopable piece of music, but now you just have to get it to do that in-game without an awkward break. That's beyond my expertise, and personally I don't care to ever learn how (except if I have to!) I just write game tracks ala Soukaigi where it's a fully developed track but ends at a point where the whole track can replay again and you wouldn't know the difference. Furthermore, if you listen to the tracks alone, the endings sound proper.

[Daniel DeCastro]

Sure, Rogue Squadron would be a good example, others perhaps "Dot Hack" or "Zelda OOT".

What I find it very difficult to do is to write a loop of the same level of quality as those of Yoko Kanno, Sakimoto, Hamauzu, Sugiyama.. you get the picture. I'm just happy I'm not the only one who sees this adaptive music things as a convention, but apparently it is not limited to just North America. My biggest fear is that in the article, the author there believes that adaptive music is the future of game music. This gets my blood boiling enough to speak out about it. He's not the only one who thinks this; even my CSound professor thinks it too. Basically what this means to me is that game music will be riddled with crummy music; music that does not stand alone i.e. BGM. This is the approach they use at USC when teaching game music (USC being the absolute top film scoring school in the world) as well according to my colleague there..

Here I am working very hard trying to absorb the language of top Japanese and Western non adaptive music composers, doing fairly well at impressing my private composition instructor, and being told by colleagues that I have the chops to go out and be a game composer, but I doubt such comments given the fact that they've only been exposed to the stuff I've shown them, and that's only enough stuff for which they have time to listen to, otherwise they are busy doing whatever else it is they do. How in the world am I to write a convincing thesis with a 1 hour long presentation when such a thing would take so much longer to make a case for?

I'm changing my thesis title from "Music, it's evolution and role in Video Games" to "Effective game music scoring" at the advice of the chairman. According to him, proclaiming the worth of game music, it's validity/role/current state of quality, and position as a solid form of composition is old news at this juncture (NYU), and yet I fear that my CSound professor has made the case of game music to him but by using "Adaptive Music" as a model. He even told me that if I don't talk about adaptive music in my thesis, that it would be no different than talking about film music. I found this rather frightening, and based on what I've read, seems to be a way of "Scholastisizing" game music.

I just want to talk about the kick ass music of people like Sakimoto, Soule, Kanno and other greats, and use THAT as a model for good game scoring. What I'd really like to know is how much of a fanbase these composers have outside of the cocoebiz community.

[Daniel DeCastro]

Here is the article itself to make it easier:

Enhancing the Impact of Music
in Drama-Oriented Games

Games are not film.

Observing games over the past ten years has been fascinating. The medium has constantly been maturing - graphics have become more sophisticated, gameplay mechanics have evolved, and improved AI has enabled programmers and designers to create more realistic interactions with virtual characters in gameworlds. My favorite evolution has been the increase in quality of game music. We now have the potential to do what any film composer can do... we can tap into the deepest parts of the human emotional pool; for the most part, we're not restricted by playback or storage limits anymore.

This leads to a question. Regardless of these constant improvements, why hasn't the level of emotional intensity in a game reached the same level that film achieves? Why do we find ourselves permanently affected by watching great films like the Lord of the Rings trilogy, yet we don't glean that same heart-level impact from the games based on the same stories? There are multiple reasons. Some facets of the typical game require a bit more evolution before they can effectively suspend the disbelief of a player to any substantial level. However, the one facet that could contribute to a higher level of emotional involvement in today's games has been consistently under-utilized. That facet is music.

It's very apparent that game designers are striving and wishing for the level of emotional sophistication in their games that Hollywood achieves in film. It's apparent because we're always trying to "filmize" our games. We set up cut scenes to try and propel an emotion onto a player using camera angles, character expressions, and of course, orchestral cues. We attempt to add drama to in-game events by triggering intense battle music whenever a player gets into a fight. We add music to areas and levels to bring them an identity and to try and create an emotional backdrop. It would seem like this approach makes sense, but there's a problem. Games aren't film.

Film benefits from the luxury of being linear - having a fixed set of visuals and order of events. This enables a composer to know exactly what is coming, and to choose to speak through that fixed reality in a musical way that sets up the perfect emotional "hook." Games, however, hold levels of relativity that can't be foreseen, calculated, or controlled. Things don't always happen the same way (or on the same timeline) between two different players playing the same game. We can't expect the emotional formulas that work on a film audience to work exactly the same way on a gamer.

However... there are some concepts that we can take from film soundtracks, rework them, and apply them in games.

Before we talk about solutions, we need to dispel some myths and mistakes we currently make in how game soundtracks are implemented today.

Music Mistake #1 - "Watering down my music and making it 'subtle' will help it to fit in and work in multiple situations."

This is the "sidestepping" approach that has been used in game music for a long time. It needs to be addressed.

One of my favorite role-playing games is Elder Scrolls III: Morrowind by Bethesda Softworks. I love the music from this game, but in terms of its emotional function throughout a gameplay session, it honestly doesn't accomplish much. The music is ambient in nature and simply plays straight through for roughly thirty minutes, then starts over again. There are cues for battle encounters, but other than that, nothing. When I enter a dark dungeon that is critical to the game's storyline, the music doesn't foreshadow anything. When I have just finished a battle and am inches from death, struggling to get back to an area where I can rest, the music doesn't reflect the critical nature of the situation at all. There is not even a shift in the wrong direction - there's no shift at all. This forces the question: why is the music even playing? It's obviously not to impose emotion onto the player in a direct way. Does keeping the music playing in an ambient state add to the "ethereal experience" of being in the gameworld? Not really. It's just white noise. It would be more effective to create a soundscape of wildlife or city bustling noise; this would at least draw the player into some sort of virtual reality. Playing ambient music in the background actually does the opposite; it further detaches a player from being convinced that the gameworld is any sort of reality at all.

So why do designers and producers (and even composers) make this mistake? There are a couple of reasons. The biggest one is simply the fact that it's the norm. There has always been level music. There has always been something to hum to while you're jumping from pipe to pipe, squashing mushroom people. We generally aren't comfortable with musical silences in games. The irony is that not even a film plays music the entire time. It appears that designers and composers don't understand the significance of the "less is more" factor in music for games... we're just too attached to what's comfortable and we won't let it go.

I think we also make the mistake of watering down the music because we don't trust the player to be able to form their own emotional picture based on what they're doing and experiencing in the game. We don't realize that entering a dark forest in Lionhead Studios' Fable can be just as immersive and spooky when you're only hearing the audio backdrop, as when the music accompanies it. (Try turning off the music next time you play.) We forget, that simply by giving a player a form of input and letting them become part of the game, it is already emotionally involving them on a certain level. When we can learn to trust this concept, we can use music in specific situations to augment emotions and raise the stakes a notch. You don't have that opportunity when ambient music has already been playing for ten minutes or more - it's just not as effective.

Music Mistake #2 - "Adaptive music will solve emotional detachment issues and tie players into my game because it will follow what is actually happening."

A level of interactivity is necessary when it comes to music in games. We need to trigger things at the right times, and even having music that evolves to a degree based on changing circumstances can be effective. However, looking to highly-adaptive music methods as a solution to express more mature emotions might not be the way to go. We talked about how ambient music doesn't accomplish much emotionally because of its lack of being placed in a context. Adaptive music can suffer from a different problem: it can be too reactive.

One of the great powers a film composer has is the ability to choose from various types of music in a single scene to bring across different emotional impressions. As an example, by tying humorous music to a physically violent scene, a different type of emotion is aroused in the viewer than if there was agitating music (or even no music at all.) For this reason, it is important that the music retains its independent position of emotional influence and power, and doesn't become dependent solely upon literal events in the film. This is no different for games. If my dynamic music engine simply follows gameplay and triggers "appropriate" music based on what the player is doing or experiencing, then that music loses its ability to speak independently. It can no longer impose unique kinds of emotions by relating the music in contrasting or half-parallel ways to the situations themselves. The music loses its position of influence and becomes a slave to game events (and ultimately, a slave to player input.) If ambient music is on one end of the spectrum, then dependent "adaptive" music is on the other. Neither extreme serves very well in drama-based games; they both end up being a sort of watered-down function.

Music Mistake #3 - "Cut scenes with live orchestral music will get players more emotionally involved in my game."

[Daniel DeCastro]

Prince of Persia: The Sands of Time by Ubisoft is a gem in the animation department. Not only are the Prince's in-game movements a smooth sight to behold - the cut scenes that precede and end the game are spectacular to watch as well. It's interesting, though - the cut scenes that occur during the Prince's journey through the palace aren't rendered in "full movie splendor" like the opening and closing ones. Instead, they are either fragments of gameplay (while the Prince is receiving a vision) or they are sequences rendered with the same "realtime level" of graphics detail that appears during actual gameplay. When designing a game that focuses so heavily on story and dramatic presentation, wouldn't Ubisoft have been smarter to go all out and do all of the cut scenes "movie-style," sparing no gimmicks, including a no-holds-barred policy on the picture-synchronized musical underscore? One would think so, but it's a good thing they didn't. This approach might have actually dropped the immersion factor a notch.

We're really addressing two different issues here, but they tie into each other with the same subtly deceptive concepts. It's important to get down to the core of the issue and ask "why do game designers put cut scenes in a game?" Yes, it's true; they do expose storyline and introduce new material into the game, but then, couldn't we just do that with a simple dialogue box on the screen? We could. But that's the point . . . cut scenes are created because the designer thinks; "I want to make an emotional, dramatic impact on the player with the way I present this information." By all means, a game designer should have this goal! It makes sense then why a designer would ask for a full orchestra to accompany these cut scenes. The orchestra is legendary. It's been around for hundreds of years and is the dynamic, de facto standard for injecting emotion into a film. "So we should use it for games!" Yes we should... but by tying it to a cut scene in a game, we've quite possibly lowered its impact. How can this be?

Watching a film is a passive form of entertainment. When one sits down to watch a film, one expects to be taken on a ride of sorts and be moved in some way by simple observation. Games are different. If I've been playing a game for an hour or so, I've learned to identify with the character I'm controlling. It goes beyond identification with a film character.. it's an extension of me. When we are talking with friends about our recent Halo escapades, we don't say, "It was cool when Master Chief lobbed that grenade from the cliffs and took out a Warthog." Instead, we say, "It was cool when I lobbed that grenade." Even if Master Chief doesn't look like me or move like me, I'm still attached because I'm in control. I'm actively participating in the character's fate. Then comes the cut scene. Suddenly I'm not in control of my character anymore. And what's more, those black bars have closed in on the top and bottom of the screen, signifying to me that this is, indeed, a cut scene.



You are the Master Chief.

What does a player do next? They revert immediately to "film mode" as well. They sit back and watch things happen to the characters on screen, soaking in the great visuals and thundering orchestral music. You've just lost your player. Regardless of how attached they were to the character they were controlling, that character is no longer an extension of the player once the cut scene begins. They're "taking a break" from their role in the gameworld reality, and even if the cut scene has the best stuff in the world in terms of video and music, your player's emotional involvement will never be as high as it would have been had they remained involved in some way. In the Prince of Persia example above, at least the player maintains some connection with the Prince during an in-game sequence, because most of these sequences resemble gameplay (thus psychologically relating to user input.)

The point isn't to veer off into the philosophy of game design, but rather, to identify the reality that better sounding and more dynamic music will never compensate for the loss of agency that a player experiences during a cut scene in a game. If you were to take the same dynamic, intense orchestral music that you used during a cut scene and instead, triggered it when the player began fighting the final boss, the emotional message portrayed by that music would be ten times more effective. Why? Because the intense music is not just coloring the situation emotionally, it's also indicating to the player, "You're in the hot seat right now."

Music Mistake #4 (The Big One) - "Let's just loop the music once it reaches the end."

From the dungeon music in Final Fantasy to the Overworld theme in The Legend of Zelda, from the eighties through the new millennium - we've constantly used this technique. There are so many reasons why this is a bad approach. Looping tends to go hand-in-hand with the "watered-down, ambient music" approach; and, it makes it worse. Not only have you eliminated the emotional effectiveness of the music by generalizing it and not applying it to a context, but by looping it over and over, you've completely detached the player from even registering it altogether. And what's worse, it usually becomes annoying after a time. Now we've moved down from "why should we even have music playing here" to "why shouldn't we turn off the music altogether and listen to MP3s?" Let's be honest. Why even hire a composer in the first place if the music isn't going to play a functional part in the gaming experience?

Why do we fall into this trap? We covered some of the reasons earlier. It's familiar. It's how we did it in the eighties, the nineties, and how we still do it in most games. We can make this mistake if we have a small music budget and we "want to make the best of what we have." It might even be as simple as "I don't know what else to do besides looping, because I'm just the programmer and Mr. Producer told me to stick Music A into Level B." A programmer should not have to make these kinds of decisions on AAA titles. Ideally, they should never have to make these kinds of decisions on any title. The bottom line: If we can't move beyond mediocre methods of implementation when it comes to music, we will never progress and mature in this area.

So then... what is a good, effective way to compose and implement music as we're trying to avoid these pitfalls? How can we create situations in our games where music begins playing and literally instills fear, joy, love or anxiety in a player? Here are some basic rules we could keep in mind that would set us on the right track.

Good Music Rule #1 - Follow the dramatic arc with the game's soundtrack.

In film, the soundtrack has two purposes. One is to impose emotion on a scene, whether that is a subtle underscore during dialogue, or a full-blown cue with just visuals and music. The second purpose is to supplement a dramatic arc over the whole of the film by connecting everything together musically. This musical arc is often more important than the literal events themselves because it can infer deeper meanings... more than simple actions on the screen can. Good directors know this. Music can depict aspects like motivation, hidden emotion, or sarcasm. This second function of a musical score is something that we have not begun to address in any sophisticated manner in games. In a film there are key scenes or events that, when coupled with key musical themes or cues, bring together meaning into a whole picture. These "musical events" should occur in a game as well (beyond simply inserting a cut scene.) Composers should start thinking beyond "What does this level sound like" to "What role does this level and its characters play in the grand scheme of the game and the plot? How do I portray that with the music I write? Where do I place the music within the level to bring this across in the most effective manner?"

[Daniel DeCastro]

Baldur's Gate: Dark Alliance has more dramatic music.

In Baldur's Gate: Dark Alliance, composer Jeremy Soule comes a little closer to developing some dramatic points in his score, compared to Morrowind. Boss battles feel more intense than common battles because there is no music triggered to accompany the everyday, normal monsters. (There is only ambient background music specified by area.) When specific music kicks in for a boss battle, you not only know you are fighting a boss, but it feels more important as well. Each boss has its own identifying style and theme. And in the final battle against Eldrith, Soule brings back the main theme of the game that plays during the title screen, signifying that this battle is, indeed, the most important of all.

Create a musical climax in your game. Don't use your most intense music until you've reached critical points in the game's dramatic arc. Is the final boss battle more important than the miniboss battle? Show it in the music. A player should be able to subconsciously interpret the importance level of events based on the music that accompanies them.

Good Music Rule #2 - Never use music unless it is making a specific emotional statement to the player.

When music plays in a game, it should mean something. It should support something specific. In a film, music never plays just to play. So why do we do this in our games? Here is a good rule of thumb: "The less you use something, the more effective it is when you do use it." We shouldn't be afraid of musical silences in games. This is what ambient sound is for. Use the sounds of forests or dripping caves or crowded streets to immerse a player in a game's reality, and trigger the music when you want to bring the player up to the next level of emotional awareness. By keeping the music more sparse, it will retain its special element of influence and won't be simply "tuned out" by players when it is triggered. And more than likely, you can completely avoid looping your music by using this mentality. This approach is the best way to spend a composer's 60 minutes of music, because each minute will count.

Good Music Rule #3 - Get the composer involved early in the process!

To add to the rule above, this is absolutely necessary if you expect to have any type of emotional coherence in your game. Film composers can be given a fixed and final product (usually) and they only need to watch it to get an idea of how music should be inserted from a technical and artistic standpoint. Games are intricate pieces of software and the composer needs to understand how the system of the game works. The composer needs to know the designer's motivations from a dramatic perspective and from a story perspective, but they also need to know how that story is going to be presented, and what kind of influence the player might have on how that story progresses. The bottom line: games are much more complex than film, and require a deeper understanding from everyone involved when artistic choices are being made. This is why the current model of "hiring the composer when we're done with the game" is not a good idea and will never result in an emotionally mature experience for the player. (Especially if the composer is under extreme deadline pressure from the get-go.)

It's also important that the composer be able to do at least some (if not all) of the music implementation. The composer knows his/her music better than anyone else, and needs the ability to experiment and find what works best to match the producer/designer's vision. Whether this involves teaming up with an audio programmer, or having decent implementation tools created for inserting music, there should be a way for the composer to have a heavy influence in all the musical performance aspects of the game.

Good Music Rule #4 - The more content, the better.

Drawing off of what we just talked about in Rule #2, a piece of music can have an even greater impact if it is only played in one place in a game. There is nothing more effective than having a section of music that identifies a single critical moment or event in a dramatic storyline. The more musical content that can be created for a game, the more headroom a composer has for dedicating certain unique cues to certain places. The reality of music budget size and the cost-per-minute to hire a composer can get in the way of this; but maybe, by getting a composer involved earlier and dedicating more of a game's budget to music and sound, we can alleviate this issue. Awareness of how much influence a well-written and well-implemented musical score can have in a game, hopefully, will raise the priority of a game's soundtrack in the budget in the near future.

We are on the brink of a new reality. Games are becoming more and more a part of popular culture with every passing year; eventually, interactive entertainment will take the lead, outdistancing forms of passive entertainment like film. (I think, for me, the reality of this really sunk in when one of my friends bought Halo 2 from a local gas station.) For this transition to happen, however, we will need to focus on issues such as emotional maturity and the important role of music in our games. Music plays a large part in how a dramatic medium is perceived and processed and it's something that we need to start developing further by breaking out of old traditions and comfort zones. I hope someday soon we will all be talking about certain moments in games that moved us to tears or held special, personal meaning for us. I hope those moments come with a great soundtrack as well.