I am using fmod core and have an issue with Channel::setPosition for a game project:
I have background music playing in a stream (system->createStream)
I want to be able to accurately use Channel::setPosition without delay.
Limitations:
The file format is mp3 and is stored remotely so cannot be changed.
I cannot load the mp3 into memory instead of streaming because I need to be able to switch between different files quickly. Loading multiple mp3s into memory does not seem like a good solution. (Channel::setPosition is instant when I have the song loaded into memory.)
Problem:
Calling Channel::setPosition can make the game lag for a few frames.
FMOD_NONBLOCKING removes the lag but causes the song to be off sync. From what I can tell there is no way to pre-calculate the delay of setPosition before calling it.
Calling Channel::setPosition using FMOD_TIMEUNIT_RAWBYTES seems to not cause any lag.
Attempt:
I know all millisecond target values beforehand, so I tried calling Channel::setPosition with FMOD_TIMEUNIT_MS for each value and then saving the Channel::getPosition FMOD_TIMEUNIT_RAWBYTES value.
Then during gameplay I call Channel::setPosition using FMOD_TIMEUNIT_RAWBYTES.
The playback does not seem to be synced correctly though, and every time I call Channel::setPosition using FMOD_TIMEUNIT_RAWBYTES there is a strange glitch sound almost like scratching a record at the start of playback.
MP3 (and many other formats) require internal priming during a seek, this means decoding multiple frames before the target position to ensure the decoder has enough internal state to play your desired position without artifacts (the glitch you hear). When you use FMOD_TIMEUNIT_RAWBYTES you are forcing the codec to that exact position and skipping any priming, which causes the audible glitch.
When seeking a stream there is a non-zero amount of time required to flush the file position, basically instant for in-memory, slower for on-disk, and very slow for a net stream. Added to that time is the priming and initial decode getting the stream ready for playback. If you use FMOD_NONBLOCKING, these operations occur asynchronously on a different thread, otherwise, they occur synchronously in the setPosition call. There is no way to know ahead of time how long this processing will take.
Sometimes you need to seek a stream, but you know so in advance, for the implementation of FMOD Studio’s timeline we have this case with looping. So we prepare a second stream pre-flushed to the correct position and play them back to back. For the cases where you don’t know ahead of time, you might need to delay actioning the seek until the target is ready, again using a second stream and playing the first till it’s ready.
For predictable loops, you can setLoopPoints ahead of time and the stream system will take care of it for you without delay.
I used the solution suggested with preparing a second stream and then switching streams. This works very well with zero delay.
What I’m wondering is if there is any performance cost in having a prepared stream idle in the background? Currently I’m preparing the second stream, and then switching when triggered. The second stream can sit in the prepared state for a long time not being used.
I’m thinking of adding the ability to crossfade between two streams that are sync sensitive. In that case I could have up to 2 streams active, and 2 preparation pre-flush streams (4 total). I understand the active streams require CPU, but what is the cost of the idle preparation streams once they have been pre-flushed but are not in use? Thanks!
Apologies for the delayed response- there is no ongoing CPU overhead for a stream that has been loaded and is not playing. It will just be taking up memory, and only when the time comes for it to play will it use CPU resources. You can confirm this by initializing FMOD with the FMOD_INIT_PROFILE_ENABLE flag, attaching the Core API Profiler and ticking the CPU Usage checkbox. Here is a screenshot of the Core API Profiler attached to the gapless_playback example. You can see that the prebuffered sounds only use CPU when they are actually playing:
