Perfect Beat Tracking in Unity?

So, I’m working on a rhythm game in which the player moves in time with the beat.

In Unity, without using Fmod, I’ve got perfect beat tracking using audiosettings.dsptime. It starts tracking the beat when the song starts, and it even corrects itself if the game jitters or lags. It plays a sound exactly on the beat. This is all good and everything, but I want to use Fmod for the dynamic music mixing cause it’s incredibly awesome.

I created a script that waits for Beat Events from the tempo track in Fmod, and plays a sound when it detects a beat callback.This however, is not perfect and frequently becomes off beat until the song loops again, especially when running at a lower frame rate.

So, what is the best way to do perfect beat tracking with Fmod? Ideally, it would involve the tempo markers in Fmod so that it can change the BPM mid-song, but that may not be the best solution.

Any ideas?

Indeed, if you use fmod callbacks to trigger some calculations in Unity then send back a event start in fmod, from what I know you’ll not have an exact sync, since the tick in Unity will usually be the standard video framerate (which is insufficient for perfect audio sync). I could be wrong, I’m more a UE user, but I guess this problem is the same in Unity.
The way to have a perfect synchronisation would be to trigger the sound, quantized, directly from the main fmod event, with a boolean parameter condition that you’d set in the game code. I don’t know if this could apply to your setup, though.

I should’ve been more specific, the end result will have all kinds of sounds. It wouldn’t really be feasible to put them directly in the Fmod track and trigger them from code.

This here is some of the project I’m working on. As you can see, there are many different possible sounds, these are being played with one shots in Unity.

I would even be ok with an Fmod equivalent to AudioSettings.dspTime to calculate the beats myself from code.

So I’ve got a script working that uses : FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out masterChannelGroup); and : masterChannelGroup.getDSPClock(out dspClock, out parentDSP); to track the time between beats using the “dspClock”, but this slowly falls apart when using Transition Regions in FMOD. Without transitions, it can run for quite a while and still stay in sync. With transitions though, it slowly becomes off beat every time it transitions, which sounds like it’s on time but apparently is slightly off.

I’m stumped. Is there no way to have perfect precision with FMOD? I’d really like to use it but I just can not figure how to convert my project to it. My previous solution for beat tracking worked perfectly in unity, so how can I do the same with FMOD?

Any help is greatly appreciated!

Ok, I found a solution. I’m not gonna post the full thing unless someone else actually comes looking for it, but the main thing to note is that in the marker callback, you can get the marker position in milliseconds. The position is the same regardless of when it receives the call back.

So, we can calculate the latency by getting the current timeline position (of the music event) and subtracting the marker position from it, then dividing by 1000 to convert it to seconds.

It took way too long to find this solution but it works amazingly well!

1 Like

I think this is exactly what we are looking for if you wouldn’t mind sharing your implementation @bloo_regard_q_kazoo

Disclaimer: I tried to comment the code as best I can but I also have no idea what I’m doing, so I apologize for any messy or weird code.

Anyways, here is the beat tracker script!

It works with transitions and tempo changes as far as I can tell.

VERY IMPORTANT: In FMOD Studio, you HAVE to set up your tempo markers properly for it to work, it’s entirely dependent on the tempo markers and regular markers. Any time the song reaches a tempo or regular marker, it will check/correct the timing of the beat. So if those are not on beat, the rest of the game won’t be either.

If you have any questions, feel free to ask!