I am trying to capture the output from a device and play it in real time through a FMOD Sound (FMOD_OPENUSER | FMOD_LOOP_NORMAL | FMOD_CREATESTREAM). Minimizing the delay between capture and playback here is important.
The issues arise when FMOD requests more data than I can fetch from the device. Should I keep looping or is there a return value to the pcmreadcallback that will tell fmod to request this packet later? Is there a way to partially fill the buffer with what we can and let FMOD know the number of bytes written ?
As I could only find little documentation on the behavior of the read callback, especially when it comes to valid return values, nor could I find equivalent examples.
My current approach does the following:
decodebuffersize has been set to 2048 as this was the minimum value I observed to be playing without stuttering
Read callback keeps looping until the requested amount of data is written (could stall the thread longer than necessary for this even though it has partially filled the data)
If more data is fetched from the device than needed, saves it in a temporary buffer for the next frame
Is this how FMOD deals with devices internally? I need to play the captured sound in real time and without distortion or stuttering, this is very important. I would love to see how FMOD deals with recording the microphone input, for instance
FMOD requests more data than I can fetch from the device
The only thing you can do there is stall inside the read callback until the data is ready. This is just the same if there was a network stall or packet loss, or the media is slow (like a cdrom).
If the decode buffer size is big enough then it won’t stutter. You wouldn’t be making it smaller , you’d be making it bigger, to hid any glitches or stalls.
It would only stutter if the length of the delay getting the data was longer than the time it takes to play a stream decode buffer.
Considering i am playing the sound in real time, if the playback has, say 50 ms delay, but fmod requests ~400ms (default value per documentation) every time, then the playback will be glitching every single time, as it will take at least 400 ms of real time to gather 400ms of buffer.
streams are not for ‘real time’, their purpose is to allow large amounts of data to be buffered up so you can hide the slowness of the media, netstreams and cdroms for example.
If you want low latency access directly to the mix stream, add a dsp callback instead. you can either use addDSP to add it to a channelgroup, or even have it play on a channel (and get the features of a channel like being able to set the frequency of it) using System::playDSP
if FMOD wants 5ms of data in this case you just buffer your own reads somewhere else and feed from that buffer in the dsp callback.