CreateStream has too big latency ( ~ 1 second )

Unity
,
1

FMOD Version: 2.02.19
Unity Version: 2022.3.55f1
Platform: Editor/Desktop

Goal: Is to use Dissonance
with fmod programmer instrument instead of bus.

Additional information: Dissonance has paid asset to implement fmod playback with bus, and it works perfectly without latency. My implementation works, but has latency.

Issue: using coreSystem.createStream creates latency ~ 1 second, which exists even if you listen to yourself offline. Note even i remove programmer event, latency still exists.

Notes: I wont post code for AudioGenerator class as its a part of paid asset, but if needed i can upload it privately or the whole project.

        private AudioGenerator _generator;
        private int _sampleRate;
        private GCHandle _handle;
        private Sound _sound;
        private FMODSoundService _audioService;
        
        public Sound Sound => _sound;
        
        public override float Amplitude => _generator?.Amplitude ?? 0;

        protected override void OnEnable()
        {
            UnityEngine.Debug.Log("OnEnable");
            base.OnEnable();
            
            //Create Audio Generator
            _generator = new AudioGenerator(this);
            _handle = GCHandle.Alloc(_generator);
            
            var coreSystem = RuntimeManager.CoreSystem;
            coreSystem.getSoftwareFormat(out _sampleRate, out _, out _);
        
            var exitInfo = new CREATESOUNDEXINFO
            {
                numchannels = 1,
                format = SOUND_FORMAT.PCMFLOAT,
                defaultfrequency = _sampleRate,
                pcmreadcallback = SoundReadCallback,
                userdata = (IntPtr)_handle,
                cbsize = Marshal.SizeOf(typeof(CREATESOUNDEXINFO)),
                length = (uint)(_sampleRate * sizeof(float))
            };

            
            var res = coreSystem.createStream(string.Empty,  MODE.OPENUSER | MODE.LOOP_NORMAL, ref exitInfo, out _sound);
            _sound.setUserData((IntPtr)_handle);
            UnityEngine.Debug.Log($"after userdata set sound: {_sound.handle.ToString("X")}");
            _sound.getUserData(out var testData);
            UnityEngine.Debug.Log($"after userdata set data: {testData.ToString("X")}");
            
            if(Check(res, "Failed to create FMOD sound") == false)
                return;
            
            //Play Sound
            coreSystem.getMasterChannelGroup(out var masterChannel);
            res = coreSystem.playSound(_sound, masterChannel, false, out var channel);
            UnityEngine.Debug.Log($"played sound {res.ToString()}");
            
            if(Check(res, "Failed to play FMOD sound") == false)
                return;
        }
  
        [AOT.MonoPInvokeCallback(typeof(SOUND_PCMREAD_CALLBACK))]
        private static RESULT SoundReadCallback(IntPtr sound, IntPtr data, uint datalen)
        {
            var soundDummy = new Sound(sound);
            soundDummy.getUserData(out var userdata);
            
            var length = datalen / sizeof(float);

            if (userdata == IntPtr.Zero)
            {
                FloatMemClear(data, length);
                return RESULT.OK;
            }

            // Extract a data source from the GC handle and copy the data into the DSP output buffer
            var ud = GCHandle.FromIntPtr(userdata);
            var audioGenerator = (AudioGenerator)ud.Target;

            // If the target is null clear the buffer to silence and early exit.
            if (audioGenerator != null)
            {
                return audioGenerator.GetAudio(data, length, 1);
            }

            FloatMemClear(data, length);
            return RESULT.OK;
        }

        private static bool Check(RESULT result, string message, bool err = false)
        {
            if (result == RESULT.OK)
                return true;

            const string fmt = "{0}. FMOD Result: {1}";
            if (err)
                Log.Error(fmt, message, result);
            else
                Log.Warn(fmt, message, result);

            return false;

        }

        private static void FloatMemClear(IntPtr buffer, uint length)
        {
            unsafe
            {
                var ptr = (float*)buffer.ToPointer();
                FloatMemClear(ptr, length);
            }
        }

        private static unsafe void FloatMemClear(float* buffer, uint length)
        {
            for (var i = 0; i < length; i++)
                buffer[i] = 0;
        }
2

bump

3

Apologies for the delayed response.

After testing your code, edited to load a large float PCM file upfront and then stream it using PCMREAD_CALLBACK, I unfortunately haven’t been able to reproduce the latency you’re describing, which makes me think that the issue has to do with how AudioGenerator is handling its sample data.

That said, I’d recommend trying a few things on the FMOD side of things:

  • Check that you haven’t set defaultDecodeBufferSize in the Core System’s Advanced settings to a value above the default, which could increase latency
  • See whether creating a regular sound instead of a stream makes any difference - since you’re using OPENUSER and PCMREAD_CALLBACK, the functional difference between a stream and OPENUSER is somewhat negligible
  • Try using PCMREAD_CALLBACK to stream a file from a buffer or from disk instead of AudioGenerator to see whether you can still reproduce the issue

Otherwise, could I get you to upload a stripped-down version of your project in which you can still reproduce the issue to your FMOD User Profile for me to take a closer look at?

4

@Leah_FMOD

Thanks for response!)

I have uploaded DissonanceFMODSoundIssue project to my profile.
You should be able to reproduce issue by playing Offline scene.
FMODEventPlaybackPrefab is implementation with sound which has ~1sec delay.
FMODPlaybackPrefab is implementation with bus and has nearly no latency.
Scripts are also in same folder.

I also have not changed any default settings for fmod, thanks!

image
image2096Ă—1141 275 KB

1 Like
5

Thanks for uploading your project!

The reason for the difference in latency is that Dissonance’s implementation uses a custom DSP, while yours uses a custom read callback. The DSP’s buffer size is the determined by the core system’s DSP buffer size, which by default is 1024 samples. On the other hand, a custom read callback’s buffer size is determined by CREATESOUNDEXINFO.decodebuffersize, which by default matches the default decode buffer size set for the system. The system’s default decode buffer size is 400ms - at a sample rate of 48kHz this is 19200 samples, meaning it incurs 18.75x more latency than the DSP implementation.

The easiest fix is to specify a lower decodebuffersize in the sound’s CREATESOUNDEXINFO. If you specify a size matching the system’s DSP buffer size, you should observe latency roughly equivalent to the DSP implementation. You can grab the DSP buffer size with RuntimeManager.CoreSystem.getDSPBufferSize().

1 Like
6

Lol it works like a charm

7

Happy to hear!