Can't Make the pre-Beat Input Work

So I’m trying creating the system for a rhythm game.
The game should be something like rhythm heaven, based on the beat so I first took the beat from FMOD and now I’m trying to set up the input based on the beat.
The idea is that there is a time window where the input is valid and from the beat onwards everything is fine, the problem is the pre-beat.
I tried predicting the next beat and using that to get the pre-beat working in many ways but it still doesn’t.
I’m pretty subborn and I don’t want to create dozens of markers in fmod to make it work, I want something more automatic.
I’ll leave there the two scripts how they are atm, there are still leftovers from previous iterations but now the prediction for the next beat use a fixed time (I also tried taking every time the difference of the last two beats but can’t see difference in practice):

Things that I tried: getting every time the difference of the last two beats, using Time.time, using getTimelinePosition, now I’m using DSPClock

MusicManager:

using UnityEngine;
using FMODUnity;
using FMOD.Studio;
using System.Runtime.InteropServices;
using FMOD;
using System;
using TMPro;

public class MusicManager : MonoBehaviour
{
    public static MusicManager instance;

    public TimelineInfo timelineInfo = null;

    [Header("Volume")]
    [Range(0, 1)]
    [SerializeField] float masterVolume = 1;
    [Range(0, 1)]
    [SerializeField] float musicVolume = 1;
    [Range(0, 1)]
    [SerializeField] float soundVolume = 1;
    private Bus masterBus, musicBus, soundBus;

    [Header("Other")]
    [SerializeField] EventReference music;
    [SerializeField] TextMeshProUGUI text;
    [SerializeField] Pulse pulse;

    EventInstance musicInstance;
    GCHandle timelineHandle;
    EVENT_CALLBACK beatCallback;

    public static int lastBeat = 0;
    public static string lastMarkerString = new FMOD.StringWrapper();

    public static ulong lastBeatTime = 0;
    public static ulong secondLastBeatTime = 0;

    [StructLayout(LayoutKind.Sequential)]
    public class TimelineInfo
    {
        public int currentBeat = 0;
        public StringWrapper lastMarker = new StringWrapper();
    }

    void Awake()
    {
        instance = this;
        if (!music.IsNull)
        {
            musicInstance = RuntimeManager.CreateInstance(music);
        }
        masterBus = RuntimeManager.GetBus("bus:/");
        musicBus = RuntimeManager.GetBus("bus:/Music");
        soundBus = RuntimeManager.GetBus("bus:/Effects");
    }

    void Start()
    {
        if (!music.IsNull)
        {
            timelineInfo = new TimelineInfo();
            beatCallback = new EVENT_CALLBACK(BeatEventCallback);
            timelineHandle = GCHandle.Alloc(timelineInfo, GCHandleType.Pinned);
            musicInstance.setUserData(GCHandle.ToIntPtr(timelineHandle));
            musicInstance.setCallback(beatCallback, EVENT_CALLBACK_TYPE.TIMELINE_BEAT | EVENT_CALLBACK_TYPE.TIMELINE_MARKER);
        }
        Invoke("MusicStart", 2f);
    }

    void MusicStart()
    {
        musicInstance.start();
    }

    public EventInstance GetMusicInstance()
    {
        return musicInstance;
    }

    void Update()
    {

        masterBus.setVolume(masterVolume);
        musicBus.setVolume(musicVolume);
        soundBus.setVolume(soundVolume);
    }

    void OnDestroy()
    {
        musicInstance.setUserData(IntPtr.Zero);
        musicInstance.stop(FMOD.Studio.STOP_MODE.ALLOWFADEOUT);
        musicInstance.release();
        timelineHandle.Free();
    }

    [AOT.MonoPInvokeCallback(typeof(FMOD.Studio.EVENT_CALLBACK))]
    static FMOD.RESULT BeatEventCallback(EVENT_CALLBACK_TYPE type, IntPtr instancePtr, IntPtr parameterPtr)
    {
        EventInstance instance = new EventInstance(instancePtr);
        IntPtr timelineInfoPtr;
        FMOD.RESULT result = instance.getUserData(out timelineInfoPtr);

        if (result != FMOD.RESULT.OK)
            UnityEngine.Debug.LogError("Timeline CallBack Error: " + result);
        else if (timelineInfoPtr != IntPtr.Zero)
        {
            GCHandle timelineHandle = GCHandle.FromIntPtr(timelineInfoPtr);
            TimelineInfo timelineInfo = (TimelineInfo)timelineHandle.Target;

            switch (type)
            {
                case EVENT_CALLBACK_TYPE.TIMELINE_BEAT:
                    {
                        var parameter = (TIMELINE_BEAT_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(TIMELINE_BEAT_PROPERTIES));
                        timelineInfo.currentBeat = parameter.beat;
                        MusicManager.instance.BeatUpdate();
                    }
                    break;
                case EVENT_CALLBACK_TYPE.TIMELINE_MARKER:
                    {
                        var parameter = (TIMELINE_MARKER_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(TIMELINE_MARKER_PROPERTIES));
                        timelineInfo.lastMarker = parameter.name;
                    }
                    break;

            }
        }
        return FMOD.RESULT.OK;
    }

    public void MarkerUpdate()
    {

    }

    public void BeatUpdate()
    {
        text.text = "Beat: " + lastBeat;
        
        pulse.Puls();
        if (lastBeatTime == 0)
        {
            musicInstance.getChannelGroup(out ChannelGroup group);
            group.getDSPClock(out ulong dpsClock, out _);
            lastBeatTime = dpsClock;
        }
        else
        {
            musicInstance.getChannelGroup(out ChannelGroup group);
            group.getDSPClock(out ulong dpsClock, out _);
            secondLastBeatTime = lastBeatTime;
            lastBeatTime = dpsClock;
        }
    }
}

PlayerInput:

using FMOD;
using FMOD.Studio;
using FMODUnity;
using UnityEngine;

public class PlayerInput : MonoBehaviour
{
    [SerializeField] EventReference correctSound;
    float timeWindow = 0.05f;
    EventInstance musicInstance;
    ulong beatInterval;

    void Start()
    {
        musicInstance = MusicManager.instance.GetMusicInstance();
        beatInterval = (ulong)((60 / 120)* AudioSettings.outputSampleRate);
    }

    void Update()
    {
        if(Input.GetMouseButton(0))
        {
            ulong nextBeatTime = MusicManager.lastBeatTime + beatInterval;

            musicInstance.getChannelGroup(out ChannelGroup group);
            group.getDSPClock(out ulong dpsClock, out _);
            ulong currentTime = dpsClock;
            float timeToLastBeat = Mathf.Abs(currentTime - MusicManager.lastBeatTime);
            float timeToNextBeat = Mathf.Abs(nextBeatTime - currentTime);
            
            ulong toleranceDSP = (ulong)(AudioSettings.outputSampleRate * timeWindow);

            if (timeToLastBeat <= toleranceDSP || timeToNextBeat <= toleranceDSP)
            {
                UnityEngine.Debug.Log("✅ Correct");
                RuntimeManager.PlayOneShot(correctSound);
            }
            else
                UnityEngine.Debug.Log("❌ Miss");
        }
    }
}

Hi,

Unfortunately, a super in-depth debug of your code falls outside of the scope of forum support.
Logging the values you’re using to calculate the tolerance period at each step to ensure that they’re what you expect is probably a good first step here; at a glance, there’s a couple of issues with the code your posted that may be contributing to it not working as you expect:

  • For DSP clocks, you likely want the parent clock instead, i.e. group.getDSPClock(out _, out ulong dpsClock);
  • Consider casting any ulong variables to another type, or adjusting the mathematical operation, if you plan on doing subtraction that could end up in the negatives - since ulong are unsigned, you’re likely getting a bad value from Mathf.Abs(nextBeatTime - currentTime);

I’d also recommend avoiding calling methods directly from within callback code (i.e. MusicManager.instance.BeatUpdate();), and free timelineHandle in case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED: as is done in our timeline callbacks example to avoid any hangs/crashes. If you need to call a method based on callback behavior, it’s generally better to pass a delegate into the callback via TimelineInfo, or set a bool in TimelineInfo which is checked from Update() in order to call the method you want.

Additionally, another way to approach this that doesn’t involve estimating the next beat time based on the previous ones would be to simply have a set tolerance time, and when a user performs an input, continue to check whether a beat callback has fired for the duration of that tolerance period.