In my game I want to couple the player’s position to the music, similar to what the game Braid did in one of its worlds. In Braid the player’s x-position was bound to the music in such a way that the music played at regular speed (1x) when the player is running at nominal speed, otherwise the music would slow down (down to 0 when the player doesn’t move).
I guess the easiest way to achieve this in FMOD would be to use the ratio of actual player speed to nominal speed and use that as input for a channel’s setPitch or setFrequency. However, in my game it is very important that player position and music do not get out of sync even if the music is 10mins long. That is, I want to ensure that even when the player’s interactivity leads to the music being constantly slowed down and up again, that a specific player position still corresponds to exactly that part of the music (a tolerance of 10-20ms seems to be acceptable).
What would be your suggestion for this kind of scenario? Would using setPosition be an alternative? If so, how often could one call this function? Certainly not every update cycle, right? (50 Hz in my game.) And what would you recommend in terms of buffer size for the corresponding music stream(s)?
Using setFrequency is definitely the thing to use for matching the playback speed to the player’s speed. The specificity of the correlation between x position and the audio’s playback position is where the complexity comes in, depending on your exact needs.
A simple implementation might do something like just rely on setFrequency when the player is moving, and then at specific x values/checkpoints/respawn, manually set the audio’s playback position with setPosition to an exact synchronized value. Another good time to sync like this would be when the player’s speed reaches 0, and as a result no audio is playing.
A more complex implementation might do something like calculate the expected playback position vs the current x value, and then perform some kind of drift compensation to interpolate to the target playback position. This will lead to a “more accurate” match between the x position and playback position, but will introduce audible warping, which may be undesirable. A C# example of the math required for this can be found in the Video Playback example in our Unity docs, specifically in SampleFramesAvailable().
Combining both of the above methods is also a possibility, but again, it depends on your exact needs. A couple of other things to note though - I likely wouldn’t call setPosition every update cycle, but for setFrequency it should be fine, as 50hz/20ms is the default update period used for FMOD Studio. Also, if you’re wanting to use setFrequency with negative frequency values for reversed playback when the player moves backwards. the sound you’re playing will have to be created as a sample instead of a stream.
Thank you very much for the detailed explanation and suggestions. I highly appreciate it!
I’ll check how war we can get by “hiding” synchronizations through setPosition in several game scenarios and keep the active drift compensation in mind as a fallback.
I have a related question in the following post so I’d greatly appreciate it if you could take a look at that as well: