Innovation of the Week: Adaptive/Dynamic/Interactive Music

Today I want to show some appreciation for an interesting aspect of games that often goes unnoticed but plays a crucial role in enhancing immersion and emotional impact: Adaptive/Dynamic/Interactive Music. This innovation combines the fact that non-redbook CD audio hardware plays back the game music and sound effects in real-time with the interactivity and non-linearity of games, to have the music respond either to where the player moves, some action they're performing, or an event occurring near them. This can make the game world come alive in a more subtle way, help the audio part of a game's worldbuilding resonate with what is currently happening, and make players feel like their actions have consequences.

Most early instances of dynamic music is pretty clunky in execution, abruptly switching out music tracks depending on where the player is in the game, such as triggering a boss theme or end of level fanfare. However, the first example of a smooth transition within an in-game music track based on player action does go all the way back to Space Invaders (ARC, 1978), also one of the first games to even feature continuous in-game music. In this game, as the player destroys the alien ships, the descending four note bassline that makes up the music will gradually speed up while the movement of the enemies does as well. This contributes to an added sense of urgency and some would say is key to making the game an experience that people still remember to this day. The game uses discrete analog circuitry for its audio hardware.

Some Other Notable Retro Game Examples (sound chips/modules are mentioned last in parenthesis for each entry):

Dig Dug (ARC, 1982, Namco 3-channel WSG) and Dragon Slayer (PC-88, 1984, Internal Speaker) - These games introduced music interactivity in a different and playful way, by playing the in-game music only while the player is moving their avatar. Dig Dug might also be the first game to make the music suddenly play faster after a certain point in each level, following a warning jingle. On a side note, Dig Dug was probably the first chip generated video game OST composed by a woman (Yuriko Keino).

Lazy Jones (C64, 1984, SID) - This game features seamless transitions between music tracks as the player moves between rooms, made easier by the tracks all being in the same tempo and having a similar structure. The variety of tracks is quite large for the time, and it's also one of the soundtracks that introduced the signature C64 chorused basslines. In general, by pairing up two channels playing the same notes at the same time and detuning each slightly, it gives the instrument a richer sound. However the SID chip has something called a pulse width modulation effect, generally used for changing the timbre over time, which can be applied to a single channel for the same result if the settings and/or note lengths are right.

Otocky (FDS, 1987, R2A03 & Ricoh 2C33) - In this early shoot 'em up/rhythm hybrid, the music becomes partially user generated by letting the player have basic control over the main lead melody while playing. The background music is set and played by the NES sound chip, while the player controls the wavetable channel provided by the Famicom Disk System addon when they fire their weapon. 8th notes can be played and there are 5-6 notes available, in scale with the music. By picking up different power ups, the instrument can also be changed, and which one you've equipped is displayed in the GUI.

​Extase (AMI, 1990, Paula) - Tribal beats, hypnotic chants, culturally ambiguous soundscapes and general weirdness are elements that make up this soundtrack, which sometimes brings to mind bands like Art of Noise or Cabaret Voltaire. There is also an interactive aspect in-game: Notes or musical patterns are added for each player interaction, and the patterns evolve as the player completes a level. The game itself is harder to explain, being a sort of electrical engineering sim mixed with Sleeping Beauty, played like a race between two players or player vs AI to wake up a female android. Each level has you waking a different area of the android's brain (such as sight, imagination, sanity, etc.).

Monkey Island 2: LeChuck's Revenge (PC, 1991, MT-32/OPL2 (Adlib)) - This game uses a relatively advanced dynamic music system called iMUSE, an evolution of previously used systems. The in-game music changes on the fly based on your actions and the transition is more or less seamless and on beat with the previous music track. Additionally, the "outside" theme in the town of Woodtick also has 7 starting points that are chosen randomly each time it is started. As for transitions based on player actions within a room/sub area, Zaarin explains an example on their site mixnmojo.com:

"When you enter Mad Marty's boat there is a melody line playing in the right speaker. This can be called Mad Marty's melody. On the left side of the screen are the men of low moral fibre snoring away. When you wake them, the melody line in the right speaker fades down partially and a new one in the left speaker fades up. A bass and snare drum also kick in, changing the tune completely. This is the men of low moral fibre's tune. Because of iMUSE the game can change between these two dynamically whenever the player awakes the men. After they fall asleep, the music changes back to how it was initially."

The Legend of Zelda: Ocarina of Time (N64, 1998, RCP) - The game's soundtrack tends to seamlessly transition between area theme and combat theme when moving close to an enemy, creating a cohesive and dynamically changing atmosphere. The combat theme fades in on one channel first, while the prior theme fades out and is replaced fully as you move closer. In some cases, an abrupt transition with a bombastic combat music intro happens such as when facing Stalfos warriors. Of course, the game is also well known for its five note system (d, f, a, b, d; a d minor 6th chord if played together), which is used for playing simple melodies using the ocarina and which have various gameplay effects - a mechanic that seems to have first appeared in the games Loom and Wonder Boy in Monster World. No attention to rhythm and note length is needed, nor knowledge of what the notes are called.

Have you come across any retro games that implemented adaptive/dynamic music in a particularly memorable way? What are your thoughts on this innovation and its impact on your gaming experience? Share your thoughts and experiences!