I’m developing a music game in UE4 and am currently using the engine’s built-in audio to play a music file.
The issue I’m having is there is always a delay between playing the sound and when it actually starts playing. The length of the delay is dependent on audio quality, hardware, etc and shifts out all of the other timing in the game.
I need to know exactly when the sound actually starts playing, and ideally it’s precise playback position (accounting for hardware playback delays) so I can sync in-game events with that.
Is Studio::EventInstance::getTimelinePosition precise enough for this? Can I access it in the UE4 Studio plugin from C++ or BP?
Querying the timeline will return the position of the cursor so that might be enough. We haven’t exposed via Blueprint but it is accessable via C++. I will add that to the list of functions to expose (should just be a few lines of code).
We have had many requests for callbacks when the cursor hits named markers, and also on the beat/bar, which is going to be done as part of our 1.6 release (sometime between now and 2 months time).
When we add the API for marker/beat callbacks we’ll update the UE4 integration to expose that to blueprint.
Another flip side of the question is for FMOD to have minimal latency. That is something we want to do a pass over for our UE4 integration. We have some initialization options in 1.06 to help with this, but it hasn’t been put into the UE4 integration yet.
I implemented a test for this yesterday to get an idea of what’s possible.
I hooked up a callback to an event instance in code for started, stopped, and restarted. It appears at least the started callback is executed when expected… I’m seeing about a 30-80ms delay (just measured with accumulative delta times in actor tick) between playing the sound and the callback being called. In the callback the timeline pos is 0. This solves my main problem, which was skewed timing due to different latency accross devices. UE4 world audio time keeps counting during latency and doesn’t provide a callback for when the sound actually starts playing, so FMOD is a life saver there!
Timeline pos didn’t seem as helpful… I tried printing its value every frame and got the same value every 3 frames (was at either 60 or 120 fps, I’ll have to check) which isn’t precise enough for syncing. I’m guessing that tracks which chunk has been sent to the audio buffer? Could still be useful for syncing up paused tracks–I have yet to look into that.
Would LOOOVE an API for marker/beat callbacks. I noticed there was something like that for synch markers in an older FMOD API, and I noticed there are sync points in the current API, but no callback. Can these be queried at the beginning of a stream and then tracked in tick? Even if that’s possible I’m sensing it wouldn’t be nearly as accurate as a callback though! (But I still might be able to use it for something anyway…)
FMOD has several threads at work. All Studio event scheduling is on its own thread - the event callbacks occur there. The 30-80ms is due to the play sound being created and then executing asynchronously (there are some improvements we can do to that coming up). Querying timeline on the main UE4 thread will just return whatever the last async update reported so it won’t be as accurate. Best of all will be marker/beat callbacks, which is high up on our list! Probably the next medium sized feature we’re going to work on in fact.