Playing back audio table programmer sounds on Unity Timeline at edit time (and runtime)

[kkukshtel]

Hey there,

I’m working on a project right now where we are leveraging Audio Tables and I’m trying to build out some tooling to allow designers to layout specific audio table events on a given Unity timeline.

I know this is already a weird thing at the edges of FMOD functionality, but I’ve gotten pretty close but am running into some blocking issues.

Specifically - I’ve wrapped the FMOD playable in my own custom playable and am initing everything correctly and setting the callbacks properly for programmer sound creation, but as you might expect I’m unable to leverage the Runtime API to actually create sounds at edit time such that they can be played and scrubbed back at edit time.

I did get OK FMOD.Result commenting out the lines that block this in the Runtime file but I’m not hearing audio playback anyways. So I’m wondering if there is special weirdness around creating programmer sounds at edit time with the Runtime API beyond leaking runtime state to the editor.

I’m wondering what could be done here or if it’s a dead end and would love some guidance.

Thanks!

[Leah_FMOD]

HI,

It’s difficult for me to really suggest any specific fix without more context - can I get you to post your modified FMOD Playable code for me to take a look at? I should be able to diagnose the issue and hopefully point you in the right direction.

[kkukshtel]

The playable code is just boilerplate, the issue stems from the callback and need to create sounds with the Runtime API (meaning, there is no ability to create sounds at edit time without the runtime API).

Here’s the relevant block in the callback code (which is mostly just the example code) -

case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
{
    FMOD.MODE soundMode = FMOD.MODE.LOOP_NORMAL | FMOD.MODE.CREATECOMPRESSEDSAMPLE | FMOD.MODE.NONBLOCKING;
    var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
    FMOD.Studio.SOUND_INFO dialogueSoundInfo;
    var keyResult = FMODUnity.RuntimeManager.StudioSystem.getSoundInfo(key, out dialogueSoundInfo);
    // FMODUnity.EditorUtils.System.getSoundInfo(key, out dialogueSoundInfo); edit mode - not supported
    if (keyResult != FMOD.RESULT.OK)
    {
        UnityEngine.Debug.LogError($"failed to load programmer sound {key} with error {keyResult}");
        break;
    }
    FMOD.Sound dialogueSound;
    var soundResult = FMODUnity.RuntimeManager.CoreSystem.createSound(dialogueSoundInfo.name_or_data, soundMode | dialogueSoundInfo.mode, ref dialogueSoundInfo.exinfo, out dialogueSound);
    if (soundResult != FMOD.RESULT.OK)
    {
        UnityEngine.Debug.LogError($"failed to create programmer sound {key} with error {soundResult}");
        break;
    }

    parameter.sound = dialogueSound.handle;
    parameter.subsoundIndex = dialogueSoundInfo.subsoundindex;
    Marshal.StructureToPtr(parameter, parameterPtr, false);
    break;
}

I looked around on the forums a lot and you’ve answered a similar issue before:

@Leah_FMOD

While it is technically possible to call into RuntimeManager to create a Studio System, it is only designed to be used at runtime. This is because RuntimeManager destroys the created System in OnDestroy(), which only happens when a scene or game ends. so you’ll leak the Studio System in the current editor instance when calling from outside of runtime. FMOD can support up to eight Systems at once - once you call into RuntimeManager eight times from the editor, FMOD will no longer be able to create a new system until you restart the editor.

As from the quoted post - is the way to do this to create a new system at edit time that the timeline uses?

What would be nice is if audiotable audio worked in the timeline by default, but given the complexity around what that audio may be maybe it’s not possible?

Let me know whats best here.

[Leah_FMOD]

@kkukshtel

The playable code is just boilerplate, the issue stems from the callback and need to create sounds with the Runtime API (meaning, there is no ability to create sounds at edit time without the runtime API).

[…]

As from the quoted post - is the way to do this to create a new system at edit?

There’s two ways I would recommend using to tackle this - creating a new system that loads your banks at edit time, and then is released when it’s no longer needed (i.e. finished auditioning the timeline, play state change, etc.) would be one way.

The other way would be to make changes to EditorUtils (./Plugins/FMOD/src/Editor/EditorUtils.cs), which is the class in the FMOD integration that handles auditioning events in editor, so that you can use it in editor instead of using the runtime system.

EditorUtils should already handle a lot of the busywork for you, like bank loading, handling state changes, releasing the system etc., but you would likely need to modify the accessibility level of some members, as well as add some additional code of your own, so depending on the complexity of what you need it may be simpler to create your own system instead.