I’ve been working on a rhythm game for a while using the built-in Unity Audio System.
In short, I have a beat tracker scripts that sends out an event when a beat (or a “half” beat) is played. A lot of things can react to these beats, for example an animation starting to play, a particle system to play etc.
The problem I am having is that I’d like to have some event instances to play exactly at the same time as the beat of the “main” event instance, which is sending out these beat events. If these events are even slightly off-beat it’s very noticeable.
With the Unity setup, I have a character playing a 3D drum sound exactly on next beat using the Unity AudioSource’s PlayScheduled method, checking the last beat’s dsp time and adding the length of the beat to that, so the result is good enough most of the time.
What would be the best approach, using FMOD, to schedule the playing of an event instance, so that it plays exactly at the time of a beat?
The easiest way to accurately schedule event instance behavior would be to handle it at design-time in the FMOD Studio application. This can be done using referenced events on a timeline alongside FMOD Studio’s quantization features to allow the Studio system to handle scheduling. Parameters can be used to control which referenced events play, their volume, etc.
It is theoretically possible to do this via the Studio API instead, but it isn’t recommended - while typically it may not be a problem, the Studio system can’t make any guarantees on synchronizing the behavior of multiple separate event instances, especially if you are making many calls via the API in a single update. This is due to the nature of the Studio system’s asynchronous processing model, where API commands you make are enqueued in a buffer, and are then submitted to the Studio system on Studio.System.update() to execute asynchronously. There are ways to make synchronized execution of event instances from the API somewhat more consistent, but setting it up in FMOD Studio is the simplest and best way of handling it.
The referenced events sound like something to look at!
Do you think it’s reasonable to use these referenced events in this scenario, where it’s possible there would be 10-20 different sound effects that I would like to play to the beat?
Let’s say one character playing an instrument and another one grunting the rhythm, and maybe a bird chirping every 7th beat.
Yes, that sounds reasonable, especially if you know which sound effects you want to play at design-time. If you wanted to control which of the sound effects played at run-time, then it’s a little more complex; this would involve setting parameters from code to control which sound effects play, for example by setting trigger conditions on the event instruments, or by automating the volume of corresponding audio tracks.
I think the challenge is that we want to consistently play 3D sound of a character playing an instrument to the rhythm, so that if you’re further away you won’t hear it but when you get closer it plays along to the “main” music.
Is this something that can be done with the design-time implementation?
What you’ll want to do is create a single parent event that plays your music, as well as all other audio you want to sync with the music, and a number of other events for all of your individual synced audio The synced audio will muted in the parent event, and instead will be sent to different events using Transceiver effects, where it can be spatialized.
Here’s a quick mockup of the parent event, playing two SFX tracks (a single looped instrument vs multiple instruments) alongside a music track:
I have placed my synced audio assets on separate audio tracks from the music, have set their faders to -∞ so they don’t produce any audio. Then, pre-fader, I’ve used a Transceiver effect set to transmit the signal from these tracks elsewhere. Each SFX track is sending to a different Transceiver channel.
Then, I’ve created a separate event with a Tranceiver set to “receive”, and placed a Spatializer on the event so the synced audio that it receives can be spatialized:
Instances of this event will all produce the same audio being played in sync by the parent event, but can all be spatialized individually.
The only other thing to note is that using a transceiver introduces a single mixer block’s worth of latency between the source and destination - if you’re finding that this makes your audio a little too unsynced for your liking, you can also use a transceiver for the music track so they’re all at the same latency.
