FMOD hangs in unity periodically

I’m not sure how to reproduce this, but it happens every so often.
We use programmer sounds for dialogue in our game. Sometimes, when we try to play audio, the program hangs waiting for FMOD to play the audio.

For some reason, starting a new program outside of unity (Activity Monitor in this instance) seems to unblock FMOD.

That does sound strange. Could you tell us exactly how you’re handling the programmer sound callbacks and calls?

Certainly! Here’s the class I use to control FMOD. You’ll want to look at “HandleCreateDialogueRoutineAsync”

using System;
using System.Collections;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using FMOD.Studio;
using FMODUnity;
using PixelCrushers.DialogueSystem;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.SceneManagement;

namespace Audio
{
    public class FMODSound : Sound
    {
        // Constants
        private const float GarbageCollectorIntervalSecs = 1f;
        private static readonly WaitForSeconds GarbageCollectorIntervalWait = new WaitForSeconds(GarbageCollectorIntervalSecs);
        private static readonly FMOD.Sound NullSoundStruct; // zeroed out
        private static readonly FMOD.Studio.SOUND_INFO NullSoundInfoStruct; // zeroed out
        private static readonly FMOD.Studio.EventInstance NullEventInstance; // zeroed out
        private static readonly SoundInstance NullSoundInstance = new FMODSoundInstanceContainer().GetReference();

        // References
        [FMODUnity.BankRef]
        public List<string> GlobalBanks;
        [FMODUnity.BankRef]
        public List<string> LocalizedGlobalBanks;
        [FMODUnity.EventRef]
        public string DialogueEvent;

        // Properties
        private List<FMODSoundInstanceContainer> ActiveSoundInstances;
        private Dictionary<string, string> LoadedBanksToOwner;
        private SoundInstance CurrentMusic;
        private Coroutine GarbageCollectorCoroutine;

        void OnValidate()
        {
            for (int i = 0; i < LocalizedGlobalBanks.Count; i++)
            {
                if (LocalizedGlobalBanks[i].Contains("_")) LocalizedGlobalBanks[i] = LocalizedGlobalBanks[i].Split('_')[0];
            }
        }

        protected override void OnEnable()
        {
            // Base
            base.OnEnable();

            // Create Loaded Banks Map
            LoadedBanksToOwner = new Dictionary<string, string>();

            // Create Sound Instance List
            ActiveSoundInstances = new List<FMODSoundInstanceContainer>();

            // Load Global Banks 
            foreach (string bank in GlobalBanks) LoadBank(bank);

            // Load Localized Global Banks 
            foreach (string bank in LocalizedGlobalBanks) LoadBankLocalized(bank);

            // Start Garbage Collector
            GarbageCollectorCoroutine = StartCoroutine(GarbageCollectorRoutine());
        }

        protected override void OnDisable()
        {
            // Stop Garbage Collector
            StopCoroutine(GarbageCollectorCoroutine);

            // Force Garbage Collect All
            RunGarbageCollectorSweep(true);

            // Unload All Banks
            UnloadAllBanks();
        }

        /**
         * Bank Loading
         */
        protected override void HandleLoadBankLocalized(string bank) => HandleLoadBank(bank + "_" + Localization.Language);
        protected override void HandleLoadBank(string bank)
        {
            if (!LoadedBanksToOwner.ContainsKey(bank))
            {
                try
                {
                    FMODUnity.RuntimeManager.LoadBank(bank);
                }
                catch (BankLoadException e)
                {
                    Debug.LogError("Failed to load bank: " + e);
                }
            }
            if (SceneManager.GetActiveScene() == null)
            {
                UnityAction<Scene, Scene> handler = null;
                SceneManager.activeSceneChanged += handler = (Scene from, Scene to) =>
                {
                    SceneManager.activeSceneChanged -= handler;
                    LoadedBanksToOwner[bank] = SceneManager.GetActiveScene().name;
                };
            }
            else LoadedBanksToOwner[bank] = SceneManager.GetActiveScene().name;
        }

        protected override void HandleUnloadBank(string bank)
        {
            if (string.IsNullOrEmpty(bank) || !LoadedBanksToOwner.ContainsKey(bank)) return;
            LoadedBanksToOwner.Remove(bank);
            FMODUnity.RuntimeManager.UnloadBank(bank);
        }

        private void UnloadAllBanks()
        {
            string[] banks = new string[LoadedBanksToOwner.Count];
            LoadedBanksToOwner.Keys.CopyTo(banks, 0);
            foreach (string bank in banks) UnloadBank(bank);
        }

        /**
         * Sound Playing
         */
        protected override SoundInstance HandleCreateSound(string soundEventKey) => CreateSoundInstance(soundEventKey).GetReference();

        protected override SoundInstance HandleCreateMusic(string musicEventKey)
        {
            // Stop Old Music
            if (CurrentMusic != null) CurrentMusic.Stop();

            // Return Sound Instance
            CurrentMusic = CreateSoundInstance(musicEventKey).GetReference();
            CurrentMusic.Start();
            return CurrentMusic;
        }

        protected override void HandlePlayOneShot(string soundEventKey) => FMODUnity.RuntimeManager.PlayOneShot(soundEventKey, Vector3.zero);

        protected override void HandleCreateDialogueRoutineAsync(string dialogueTableKey, Action<SoundInstance> soundReadyCallback)
        {
            if (soundReadyCallback == null) return;
            StartCoroutine(HandlePlayDialogueRoutineAsyncRoutine(dialogueTableKey, soundReadyCallback));
        }
        private IEnumerator HandlePlayDialogueRoutineAsyncRoutine(string dialogueTableKey, Action<SoundInstance> soundReadyCallback)
        {
            // Load Sound Path
            FMOD.Studio.SOUND_INFO dialogueSoundInfo;
            FMOD.RESULT keyResult = FMODUnity.RuntimeManager
                .StudioSystem
                .getSoundInfo(dialogueTableKey, out dialogueSoundInfo);
            if (keyResult != FMOD.RESULT.OK)
            {
                Debug.LogError("Couldn't find dialogue with key: " + dialogueTableKey);
                soundReadyCallback?.Invoke(NullSoundInstance);
                yield break;
            }

            // Load Sound
            FMOD.Sound dialogueSound;
            FMOD.MODE soundMode = FMOD.MODE.LOOP_NORMAL
                | FMOD.MODE.CREATECOMPRESSEDSAMPLE
                | FMOD.MODE.NONBLOCKING;
            FMOD.RESULT soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(
                dialogueSoundInfo.name_or_data, soundMode | dialogueSoundInfo.mode,
                ref dialogueSoundInfo.exinfo, out dialogueSound);
            if (soundResult != FMOD.RESULT.OK)
            {
                Debug.LogError("Couldn't load sound: " + dialogueTableKey);
                soundReadyCallback?.Invoke(NullSoundInstance);
                yield break;
            }

            // Wait to Load
            int maxFrameWait = 120;
            FMOD.OPENSTATE openstate;
            uint percentbuffered;
            bool starving;
            bool diskbusy;
            dialogueSound.getOpenState(out openstate, out percentbuffered, out starving, out diskbusy);
            while (openstate != FMOD.OPENSTATE.READY)
            {
                yield return null;
                dialogueSound.getOpenState(out openstate, out percentbuffered, out starving, out diskbusy);
                if (--maxFrameWait <= 0)
                {
                    dialogueSound.release();
                    Debug.LogError("Failed to load dialogue sound " + dialogueTableKey);
                    soundReadyCallback?.Invoke(NullSoundInstance);
                    yield break;
                }
            }

            // Create Instance 
            soundReadyCallback(CreateSoundInstance(DialogueEvent, dialogueSound, dialogueSoundInfo).GetReference());
        }

        protected override SoundInstance HandleGetCurrentMusic() => CurrentMusic;

        /**
         * Timescale Pause Toggle
         */
        protected override void HandleTimescalePauseToggle(bool pause)
        {
            foreach (FMODSoundInstanceContainer instanceContainer in ActiveSoundInstances)
            {
                instanceContainer.TimeScalePause(pause);
            }
        }

        private FMODSoundInstanceContainer CreateSoundInstance(string eventName) => CreateSoundInstance(eventName, NullSoundStruct, NullSoundInfoStruct);
        private FMODSoundInstanceContainer CreateSoundInstance(string eventName, FMOD.Sound sound, FMOD.Studio.SOUND_INFO soundInfo)
        {
            FMOD.Studio.EventInstance instance = FMODUnity.RuntimeManager.CreateInstance(eventName);
            FMODSoundInstanceContainer instanceContainer = new FMODSoundInstanceContainer(instance, eventName, sound, soundInfo);
            ActiveSoundInstances.Add(instanceContainer);
            return instanceContainer;
        }

        private IEnumerator GarbageCollectorRoutine()
        {
            while (true)
            {
                yield return GarbageCollectorIntervalWait;
                RunGarbageCollectorSweep();
            }
        }

        private void RunGarbageCollectorSweep(bool destroyAll = false)
        {
            List<FMODSoundInstanceContainer> retainList = new List<FMODSoundInstanceContainer>();
            FMOD.Studio.PLAYBACK_STATE state;
            foreach (FMODSoundInstanceContainer container in ActiveSoundInstances)
            {
                // Destroy Everything
                if (destroyAll)
                {
                    container.DestroyInstance();
                    continue;
                }

                // Conditional Destruction 
                // Must be Stopped/Paused and Dead
                if (!container.IsAlive())
                {
                    bool isPaused;
                    container.EventInstance.getPaused(out isPaused);
                    container.EventInstance.getPlaybackState(out state);
                    if (state == FMOD.Studio.PLAYBACK_STATE.STOPPED || isPaused)
                    {
                        container.DestroyInstance();
                    }
                }
                else retainList.Add(container);
            }
            ActiveSoundInstances = retainList;
        }

        /**
         * Sound Instance Classes
         */
        private class FMODSoundInstanceContainer
        {
            // Parent Class Public
            public FMOD.Studio.EventInstance EventInstance;
            public string EventKey;
            public List<CallbackAndMask> Callbacks;
            // Properties
            private FMOD.Studio.EVENT_CALLBACK FMODCallback;
            private WeakReference<FMODSoundInstance> Reference;
            private FMODCallbackWrapper CallbackWrapper;
            private GCHandle CallbackWrapperHandle;
            private bool timeScalePaused;

            public FMODSoundInstanceContainer() : this(NullEventInstance, string.Empty) { } // Used for failure to load
            public FMODSoundInstanceContainer(FMOD.Studio.EventInstance eventInstance, string eventKey) : this(eventInstance, eventKey, NullSoundStruct, NullSoundInfoStruct) { }
            public FMODSoundInstanceContainer(FMOD.Studio.EventInstance eventInstance, string eventKey, FMOD.Sound sound, FMOD.Studio.SOUND_INFO soundInfo)
            {
                this.EventInstance = eventInstance;
                this.EventKey = eventKey;
                this.Reference = new WeakReference<FMODSoundInstance>(new FMODSoundInstance(this));
                this.Callbacks = new List<CallbackAndMask>(128);
                // Setup Callback Handler
                this.FMODCallback = new FMOD.Studio.EVENT_CALLBACK(FMODUnmanagedCallback);
                this.CallbackWrapper = new FMODCallbackWrapper(FMODCallbackHandler, sound, soundInfo);
                this.CallbackWrapperHandle = GCHandle.Alloc(CallbackWrapper, GCHandleType.Pinned);
                this.EventInstance.setUserData(GCHandle.ToIntPtr(CallbackWrapperHandle));
                this.EventInstance.setCallback(this.FMODCallback,
                FMOD.Studio.EVENT_CALLBACK_TYPE.STOPPED  // Stop is the only user supported event 
                | FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND
                | FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROY_PROGRAMMER_SOUND
                | FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED
                | FMOD.Studio.EVENT_CALLBACK_TYPE.STOPPED
                );
            }

            public SoundInstance GetReference()
            {
                FMODSoundInstance instance;
                Reference.TryGetTarget(out instance);
                return instance;
            }

            public bool IsAlive()
            {
                FMODSoundInstance instance;
                return Reference.TryGetTarget(out instance);
            }

            public void DestroyInstance()
            {
                if (IsAlive()) Debug.LogError("Destroying SoundInstance that still has an owner");
                EventInstance.release();
                EventInstance.stop(FMOD.Studio.STOP_MODE.IMMEDIATE);
            }

            public void TimeScalePause(bool pause)
            {
                if (pause)
                {
                    if (timeScalePaused) return;
                    PLAYBACK_STATE playbackState;
                    EventInstance.getPlaybackState(out playbackState);
                    if (playbackState == PLAYBACK_STATE.PLAYING)
                    {
                        timeScalePaused = true;
                        EventInstance.setPaused(true);
                    }
                }
                else if (timeScalePaused)
                {
                    timeScalePaused = false;
                    EventInstance.setPaused(false);
                }
            }

            private void FMODCallbackHandler(FMOD.Studio.EVENT_CALLBACK_TYPE eventType)
            {
                foreach (CallbackAndMask callback in this.Callbacks)
                {
                    if (callback.IsCallbackTypeListener(eventType))
                    {
                        callback.Callback?.Invoke(callback.TranslateType(eventType));
                    }
                }
            }

            [AOT.MonoPInvokeCallback(typeof(FMOD.Studio.EVENT_CALLBACK))]
            private static FMOD.RESULT FMODUnmanagedCallback(FMOD.Studio.EVENT_CALLBACK_TYPE type, IntPtr instancePtr, IntPtr parameterPtr)
            {
                // Get Instance
                FMOD.Studio.EventInstance instance = new FMOD.Studio.EventInstance(instancePtr);
                // Retrieve the user data
                IntPtr callbackWrapperPtr;
                FMOD.RESULT result = instance.getUserData(out callbackWrapperPtr);
                if (result != FMOD.RESULT.OK)
                {
                    Debug.LogError("Failed to fetch user data for audio callback: " + result);
                }
                else if (callbackWrapperPtr != IntPtr.Zero)
                {
                    // Grab Parameters
                    GCHandle callbackWrapperHandle = GCHandle.FromIntPtr(callbackWrapperPtr);
                    FMODCallbackWrapper callbackWrapper = (FMODCallbackWrapper)callbackWrapperHandle.Target;

                    // Handle Default Actions
                    switch (type)
                    {
                        case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
                            {
                                FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES programmerSoundProperties =
                                    (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(
                                        parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
                                programmerSoundProperties.sound = callbackWrapper.Sound.handle;
                                programmerSoundProperties.subsoundIndex = callbackWrapper.SoundInfo.subsoundindex;
                                Marshal.StructureToPtr(programmerSoundProperties, parameterPtr, false);
                                break;
                            }
                        case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROY_PROGRAMMER_SOUND:
                            {
                                FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES parameter =
                                    (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(
                                        parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
                                FMOD.Sound sound = new FMOD.Sound(parameter.sound);
                                sound.release();
                                break;
                            }
                        case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED:
                            {
                                callbackWrapperHandle.Free();
                                break;
                            }
                    }

                    // Call User Callbacks
                    callbackWrapper.UserCallbackHandler?.Invoke(type);
                }
                return FMOD.RESULT.OK;
            }

            private class FMODSoundInstance : SoundInstance
            {
                private FMODSoundInstanceContainer Container;
                public FMODSoundInstance(FMODSoundInstanceContainer container) => this.Container = container;

                public override string GetEventName() => Container.EventKey;

                public override void OnStateChange(Action<SoundInstanceState> callback, SoundInstanceState stateMask)
                {
                    // Sanity Checks
                    if (callback == null) return;
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't set callback on destroyed sound instance");
                        callback(SoundInstanceState.Error);
                        return;
                    }
                    if (stateMask != SoundInstanceState.Stopped)
                    {
                        Debug.LogError("Only Stopped and Error state are supported for callback");
                        return;
                    }

                    // Store Callback
                    Container.Callbacks.Add(new CallbackAndMask(callback, stateMask));
                }

                public override void SetParameter(string parameter, float value, bool skipSeek)
                {
                    // Sanity
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't set parameter on destroyed sound instance");
                        return;
                    }
                    FMOD.RESULT result = Container.EventInstance.setParameterByName(parameter, value, skipSeek);
                    if (result != FMOD.RESULT.OK)
                    {
                        Debug.LogError("Failed to set variable " + parameter + " to " + value + " with result " + result);
                    }
                }

                public override void Start()
                {
                    // Sanity
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't start a destroyed sound instance");
                        return;
                    }
                    Container.EventInstance.start();
                }

                public override void Stop(bool immediate = false)
                {
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't attach a destroyed sound instance");
                        return;
                    }
                    Container.EventInstance.stop(immediate ? FMOD.Studio.STOP_MODE.IMMEDIATE : FMOD.Studio.STOP_MODE.ALLOWFADEOUT);
                }

                public override void Pause(bool pause)
                {
                    // Sanity
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't pause a destroyed sound instance");
                        return;
                    }
                    Container.EventInstance.setPaused(pause);
                }

                public override void AttachToGameObject(GameObject gameObject)
                {
                    // Sanity
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't attach a destroyed sound instance");
                        return;
                    }
                    FMODUnity.RuntimeManager
                        .AttachInstanceToGameObject(
                            Container.EventInstance, gameObject.transform, gameObject.GetComponent<Rigidbody2D>());
                }

                public override void DetachFromGameObject()
                {
                    // Sanity
                    if (!Container.EventInstance.isValid())
                    {
                        Debug.LogError("Can't detach a destroyed sound instance");
                        return;
                    }
                    FMODUnity.RuntimeManager.DetachInstanceFromGameObject(Container.EventInstance);
                }
            }

            public class CallbackAndMask
            {
                public SoundInstance.SoundInstanceState Mask;
                public Action<SoundInstance.SoundInstanceState> Callback;

                public CallbackAndMask(Action<SoundInstance.SoundInstanceState> callback, SoundInstance.SoundInstanceState mask)
                {
                    this.Mask = mask;
                    this.Callback = callback;
                }

                public bool IsCallbackTypeListener(FMOD.Studio.EVENT_CALLBACK_TYPE eventType)
                {
                    return (TranslateType(eventType) & Mask) != 0;
                }

                public SoundInstance.SoundInstanceState TranslateType(FMOD.Studio.EVENT_CALLBACK_TYPE eventType)
                {
                    switch (eventType)
                    {
                        case FMOD.Studio.EVENT_CALLBACK_TYPE.STOPPED:
                            return SoundInstance.SoundInstanceState.Stopped;
                    }
                    return 0;
                }
            }
        }
    }

    [StructLayout(LayoutKind.Sequential)]
    class FMODCallbackWrapper
    {
        // User Callback Handler 
        public Action<FMOD.Studio.EVENT_CALLBACK_TYPE> UserCallbackHandler;
        // Dialogue Parameters
        public FMOD.Sound Sound;
        public FMOD.Studio.SOUND_INFO SoundInfo;

        public FMODCallbackWrapper(Action<FMOD.Studio.EVENT_CALLBACK_TYPE> userCallbackHandler)
        {
            this.UserCallbackHandler = userCallbackHandler;
        }

        public FMODCallbackWrapper(
            Action<FMOD.Studio.EVENT_CALLBACK_TYPE> userCallbackHandler,
            FMOD.Sound sound,
            FMOD.Studio.SOUND_INFO soundInfo) : this(userCallbackHandler)
        {
            this.Sound = sound;
            this.SoundInfo = soundInfo;
        }
    }
}

Thanks! While we look over that, would you mind telling us which version of Unity and which version of FMOD Studio you’re using?

Oh, and are there any warnings or errors appearing in Unity’s logs when this issue occurs?

There are no warnings or logs when this happens. I’m using FMOD 2.01.07 and Unity 2019.4.25f1.

It’s hard to reproduce. I haven’t been working on audio as much and so it hasn’t been happening as often which makes me thing that it has something to do with building in FMOD while Unity is running. I’m not sure though. I haven’t had to rebuild in a little while since we just wrapped initial dialogue recording.

Oh no, I’m wrong. It just happened again. I’d been editing a random prefab, and then when I started the game again, the voice was locked until I opened a different program.

I wonder if this has to do with (FMOD: Cleaning up duplicate platform warning - #24 by VoodooDetective).

It’s unlikely to have any connection, but I’ll investigate.

We haven’t yet been able to reproduce this issue, so we haven’t been able to determine the cause or find a solution. However, as we have not received reports of this issue from other Unity Integration users, it seems likely that the cause is something unique to your project. Are you aware of any code or content in your Unity project that might result in it hanging until another process is started?

Also, do the hangs still occur if you comment out the lines of code that play programmer sounds?

Hmm, OK, I better try to get this to reproduce consistently. I’ll let you know if I find anything illuminating.

Just to be clear, Unity doesn’t hang, I have a part of the game that waits until the dialogue plays before continuing. That part of the game hangs silently until I open another process at which point the sound loads and plays immediately.