Unity thread error and freeze when playing Programmer Sound

I’ve modified the Programmer Sound example to try and play a WAV file generated by a TTS library. The sound file is generated fine and the path appears correct, but FMOD errors out and freezes while trying to play the sound.

I’m using Unity 2022.2.9f1 and FMOD 2.02

using System;
using System.Collections.Generic;
using UnityEngine;
using Crosstales.RTVoice;
using System.Runtime.InteropServices;
using FMODUnity;


public class TextToSpeech : MonoBehaviour
{
    private static string soundFolder = "TTS";
    private static string soundName = "tts_file";
    public EventReference eventName;

    FMOD.Studio.EVENT_CALLBACK ttsCallback;

    private static string path;

    private void Start()
    {
        ttsCallback = new FMOD.Studio.EVENT_CALLBACK(TTSEventCallback);
        path = Application.persistentDataPath + "/" + soundFolder + "/";
    }
    
    public void SayText(string text)
    {
        Speaker.Instance.Generate(text, path + soundName, Speaker.Instance.VoiceForName("Reed"));
        Debug.Log(path);
        
        var ttsInstance = FMODUnity.RuntimeManager.CreateInstance(eventName);
        
        // Pin the key string in memory and pass a pointer through the user data
        GCHandle stringHandle = GCHandle.Alloc(soundName + ".wav");
        ttsInstance.setUserData(GCHandle.ToIntPtr(stringHandle));

        ttsInstance.setCallback(ttsCallback);
        ttsInstance.start();
        ttsInstance.release();
    }
    
    [AOT.MonoPInvokeCallback(typeof(FMOD.Studio.EVENT_CALLBACK))]
    static FMOD.RESULT TTSEventCallback(FMOD.Studio.EVENT_CALLBACK_TYPE type, IntPtr instancePtr, IntPtr parameterPtr)
    {
        FMOD.Studio.EventInstance instance = new FMOD.Studio.EventInstance(instancePtr);

        // Retrieve the user data
        IntPtr stringPtr;
        instance.getUserData(out stringPtr);

        // Get the string object
        GCHandle stringHandle = GCHandle.FromIntPtr(stringPtr);
        String key = stringHandle.Target as String;

        switch (type)
        {
            case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
            {
                FMOD.MODE soundMode = FMOD.MODE.LOOP_NORMAL | FMOD.MODE.CREATECOMPRESSEDSAMPLE | FMOD.MODE.NONBLOCKING;
                var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));

                if (key.Contains("."))
                {
                    FMOD.Sound dialogueSound;
                    var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(path + key, soundMode, out dialogueSound);
                    if (soundResult == FMOD.RESULT.OK)
                    {
                        parameter.sound = dialogueSound.handle;
                        parameter.subsoundIndex = -1;
                        Marshal.StructureToPtr(parameter, parameterPtr, false);
                    }
                }
                else
                {
                    FMOD.Studio.SOUND_INFO dialogueSoundInfo;
                    var keyResult = FMODUnity.RuntimeManager.StudioSystem.getSoundInfo(key, out dialogueSoundInfo);
                    if (keyResult != FMOD.RESULT.OK)
                    {
                        break;
                    }
                    FMOD.Sound dialogueSound;
                    var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(dialogueSoundInfo.name_or_data, soundMode | dialogueSoundInfo.mode, ref dialogueSoundInfo.exinfo, out dialogueSound);
                    if (soundResult == FMOD.RESULT.OK)
                    {
                        parameter.sound = dialogueSound.handle;
                        parameter.subsoundIndex = dialogueSoundInfo.subsoundindex;
                        Marshal.StructureToPtr(parameter, parameterPtr, false);
                    }
                }
                break;
            }
            case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROY_PROGRAMMER_SOUND:
            {
                var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
                var sound = new FMOD.Sound(parameter.sound);
                sound.release();

                break;
            }
            case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED:
            {
                // Now the event has been destroyed, unpin the string memory so it can be garbage collected
                stringHandle.Free();

                break;
            }
        }
        return FMOD.RESULT.OK;
    }
}

It appears to be crashing on this line.

var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(path + key, soundMode, out dialogueSound);

The error:

UnityException: get_isPlaying can only be called from the main thread.
Constructors and field initializers will be executed from the loading thread when loading a scene.
Don't use this function in the constructor or field initializers, instead move initialization code to the Awake or Start function.
  at (wrapper managed-to-native) UnityEngine.Application.get_isPlaying()
  at FMODUnity.RuntimeManager.get_Instance () [0x00014] in /Assets/Plugins/FMOD/src/RuntimeManager.cs:122 
  at FMODUnity.RuntimeManager.get_CoreSystem () [0x00001] in /Assets/Plugins/FMOD/src/RuntimeManager.cs:199 
  at TextToSpeech.TTSEventCallback (FMOD.Studio.EVENT_CALLBACK_TYPE type, System.IntPtr instancePtr, System.IntPtr parameterPtr) [0x00085] in /Assets/Scripts/Accessibility/TextToSpeech.cs:64 
  at (wrapper native-to-managed) TextToSpeech.TTSEventCallback(FMOD.Studio.EVENT_CALLBACK_TYPE,intptr,intptr)
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityEngine.<>c:<RegisterUECatcher>b__0_0(Object, UnhandledExceptionEventArgs) (at /Users/bokken/build/output/unity/unity/Runtime/Export/Scripting/UnhandledExceptionHandler.bindings.cs:46)
UnityEngine.Application:get_isPlaying()
FMODUnity.RuntimeManager:get_Instance() (at Assets/Plugins/FMOD/src/RuntimeManager.cs:122)
FMODUnity.RuntimeManager:get_CoreSystem() (at Assets/Plugins/FMOD/src/RuntimeManager.cs:199)
TextToSpeech:TTSEventCallback(EVENT_CALLBACK_TYPE, IntPtr, IntPtr) (at Assets/Scripts/Accessibility/TextToSpeech.cs:64)

(Filename: Assets/Plugins/FMOD/src/RuntimeManager.cs Line: 122)

Hi,

Unfortunately, I was not able to reproduce the issue loading a .wav file I had on disk. Could get the directory or full path of a .wav file you are trying to load? It may be an issue with the location of the file. Is it ever able to run the createSound() function or is it always breaking on that line?

Could you please enable API Error logging and Memory Tracking and let me know if you get any other errors?

I’ve enabled those settings and there are no additional errors. The file exists and the path is correct. I get the same error when using the static file path from a previously generated file. The error happens when trying to retrieve the “CoreSystem”. It never makes it to the createSound call.

It seems to have trouble retrieving the FMOD instance from the static callback function.

UnityEngine.Application:get_isPlaying()
FMODUnity.RuntimeManager:get_Instance() (at Assets/Plugins/FMOD/src/RuntimeManager.cs:122)
FMODUnity.RuntimeManager:get_CoreSystem() (at Assets/Plugins/FMOD/src/RuntimeManager.cs:199)

I’m running this on a Mac OS version 13.6.1.

Hi,

I tried reproducing on Mac and I still had no luck.

Would it be possible to get a copy of your project or a stripped-down version displaying the issue uploaded to your profile? Please note you will have to register a project with us before you can upload files.

I’ve been able to make it work by caching the system instance outside the static call back function. Is this ok to do or will it cause problems?

I’ve replaced this line in the static callback function.

var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(path + key, soundMode, out dialogueSound);

With a static member variable at the top of the class.

private static FMOD.System system;

Then caching it in Start().

system = FMODUnity.RuntimeManager.CoreSystem;

And then calling the cached instance from the static call back function.

var soundResult = system.createSound(path + key, soundMode, out dialogueSound);

Are there any issues using FMOD like this?

Also, is there something that needs to be set in the project for it to work from the static callback function? FMOD seems to be working everywhere else for sound in the project. I’ve only had a problem using the callback with programmer sounds.

Hi,

Apologies for the delayed response.

Just to confirm, is this happening while the editor is running or outside of the game?

It freezes in the editor and just crashes in a build when trying to get an instance of the core system.

Hi,

Would it be possible to get a copy of your project or a stripped-down version displaying the issue uploaded to your profile? Please note you will have to register a project with us before you can upload files.

It is confusing why caching the core system is not causing the issue while accessing the system directly is. If I could test it on my side I may be able to resolve the issue.