So, I’m working on a rhythm game in which the player moves in time with the beat.
In Unity, without using Fmod, I’ve got perfect beat tracking using audiosettings.dsptime. It starts tracking the beat when the song starts, and it even corrects itself if the game jitters or lags. It plays a sound exactly on the beat. This is all good and everything, but I want to use Fmod for the dynamic music mixing cause it’s incredibly awesome.
I created a script that waits for Beat Events from the tempo track in Fmod, and plays a sound when it detects a beat callback.This however, is not perfect and frequently becomes off beat until the song loops again, especially when running at a lower frame rate.
So, what is the best way to do perfect beat tracking with Fmod? Ideally, it would involve the tempo markers in Fmod so that it can change the BPM mid-song, but that may not be the best solution.
Indeed, if you use fmod callbacks to trigger some calculations in Unity then send back a event start in fmod, from what I know you’ll not have an exact sync, since the tick in Unity will usually be the standard video framerate (which is insufficient for perfect audio sync). I could be wrong, I’m more a UE user, but I guess this problem is the same in Unity.
The way to have a perfect synchronisation would be to trigger the sound, quantized, directly from the main fmod event, with a boolean parameter condition that you’d set in the game code. I don’t know if this could apply to your setup, though.
I should’ve been more specific, the end result will have all kinds of sounds. It wouldn’t really be feasible to put them directly in the Fmod track and trigger them from code.
This here is some of the project I’m working on. As you can see, there are many different possible sounds, these are being played with one shots in Unity.
I would even be ok with an Fmod equivalent to AudioSettings.dspTime to calculate the beats myself from code.
So I’ve got a script working that uses : FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out masterChannelGroup); and : masterChannelGroup.getDSPClock(out dspClock, out parentDSP); to track the time between beats using the “dspClock”, but this slowly falls apart when using Transition Regions in FMOD. Without transitions, it can run for quite a while and still stay in sync. With transitions though, it slowly becomes off beat every time it transitions, which sounds like it’s on time but apparently is slightly off.
I’m stumped. Is there no way to have perfect precision with FMOD? I’d really like to use it but I just can not figure how to convert my project to it. My previous solution for beat tracking worked perfectly in unity, so how can I do the same with FMOD?
Ok, I found a solution. I’m not gonna post the full thing unless someone else actually comes looking for it, but the main thing to note is that in the marker callback, you can get the marker position in milliseconds. The position is the same regardless of when it receives the call back.
So, we can calculate the latency by getting the current timeline position (of the music event) and subtracting the marker position from it, then dividing by 1000 to convert it to seconds.
It took way too long to find this solution but it works amazingly well!
It works with transitions and tempo changes as far as I can tell.
VERY IMPORTANT: In FMOD Studio, you HAVE to set up your tempo markers properly for it to work, it’s entirely dependent on the tempo markers and regular markers. Any time the song reaches a tempo or regular marker, it will check/correct the timing of the beat. So if those are not on beat, the rest of the game won’t be either.
Hey. Thank you so much for sharing your code. It’s exactly what I need for my university project!
Is there any chance you’d be able to talk it through with me? I’m extremely new to FMOD and although I know this is what I need it’s quite hard to digest.
How far have you gotten in regard to beat tracking? Is there anything in particular that isn’t making sense? Have you had a read through our FMOD Studio and Unity integration documentation?
Hey, I’ve slowly gotten a lot working! As far as I can tell it won’t stray from the beat when running for a long period of time.
Things I got working:
-Basic beat tracking, with events called on the up and down beats.
-Tempo changes without skipping a beat.
-Discrepancy correction IF the beat ends up being off for whatever reason.
Things that don’t currently work:
-Tempo ramping.
-Swinging the up-beats.
Something that might help are tempo markers (FMOD Studio | Glossary) which allow you to control the beats and time of an event. More information can be found in our docs.
Could you elaborate on what you are wanting to do here?
I’m aware of the tempo markers, I use them for regular tempo changes in the track. When I say “tempo ramping” I mean interpolating between two different tempos, a smooth transition.
As for “swinging the up-beats”, I mean offsetting the point between the beats by a certain amount, a longer amount of time between the “start” of the beat and the “middle” than the amount of time between the “middle” of the beat and the “end” of the beat.
Since FMOD doesn’t have a “swing” feature I just used a workaround that is sufficient for my needs.
Oh hey! Sorry I didn’t see your comment sooner. I’d be more than happy to help you out with understanding the code. I should have some time tomorrow to assist with whatever you need!
I’ve also used Unity audio for my beat-tracking project right now, but we’re interested in FMOD for various reasons.
I noticed that if I wanted to play another sound (event instance) at exactly at the same time as the next beat (for example a grunt or a jump sound), how would this be achieved? With unity I’ve just used the PlayScheduled with the next beat’s time. Did you have anything similar, and if so how did you approach it?
In mine at least, I’m subscribing a method to the “OnFixedBeat” delegate in the beat tracker script I provided, like so: “BeatTracker.OnFixedBeat += (the method you want to happen on the beat)”. You can create a beat counter that ticks up on the beat, and play the sound when the number of beats has elapsed.
Oh, if I remember right the beat tracker won’t start unless you press the spacebar during playmode, you can change this obviously but I forgot to remove that.
I tried subscribing to the fixed beat event, but it felt like everything was slightly off-beat. Obviously I’m still using a mixture of FMOD and Unity Audio to just feel things out, so there could be some delay in the case of the event firing and then calling an audio clip to play.
Based on other forum posts, there’s probably some delay too if another event instance is requested to start on this beat. Did you get single sound effects to play nicely on top of the rhythm so that it didn’t seem like they’re “out of sync”?
And yeah I noticed that spacebar thing, I made some small tweaks for it to work “within” my current beat tracking.
The track could be misconfigured on FMOD side also, but I don’t have the access to that right now, so I have to get back to you once I can confirm that it’s all good.
@bloo_regard_q_kazoo I tried your method, but clicking space causes a null reference, NullReferenceException: Object reference not set to an instance of an object BeatTracker.SetTrackTempo (),
do you have any clues to what causes this?
So sorry I’m late, I don’t have notifications on for fmod forums. My two guesses for the Null Reference are one, what @Connor_FMOD said, or two, the track tempo is not set in FMOD, make sure you have at least one track tempo marker at the very beginning of the track.
