Hello!
A project we’re working on has recently switched to FMOD instead of the built-in Unity audio system. We use a lot of timelines and our project is localized. I found this thread that helped me, but I am still battling some issues.
We can use normal FMOD events just fine in a timeline with FMOD event tracks, both in play mode and in editor preview, but things are a bit more complicated when it comes to using localized audio tables.
I made a custom track whose clips take an FMOD event (like the built-in FMOD event track) as well as the key for our sound.
At runtime, I can simply use the callback given in this programmer sounds tutorial, so no issues there. However, when it comes to editor preview, I obviously cannot use the RuntimeManager.
I saw that for normal events, the EditorUtils class has callbacks that take care of the sound preview, so I made a “LocalizedEditorUtils” that holds similar code but made for my custom track.
In order to play the programmer instrument in editor preview, I set a callback similar to the aforementioned tutorial, with the added code that gets or creates a FMOD.System from a static class I made.
This method mostly works, with some problems:
I didn’t find how to have the sound start later than its beginning, meaning if i set my timeline head in the middle of the clip, it will still play as if it was a the beginning.
If a clip plays while another one is playing in another track, the clips stops. I do use different event instances so I’m not sure why it’s doing that.
It seems the sound stops earlier than it should sometimes. That might have to do with the sound taking a little time to load and then the clip being over before the whole sound played.
I hope my explanation makes sense, thanks for you input!
Could I please get some more info from you? Specifically:
Your FMOD for Unity plugin version number
Your Unity version number
To achieve playback starting from a specific point within the audio, you would need to manually set the playback position of the FMOD event instance, which involves using the Studio::EventInstance::setTimelinePosition.
I can’t seem to reproduce the issue with multiple FMOD event tracks playing different clips at the same time. Could you please elaborate more on that? Would it be possible to share some code snippets on how you set up the custom track and play the programmer sound? Additionally, some screen shots of how you set up the event in your FMOD Studio would be helpful.
That does sound like it is related to how FMOD handles loading of the samples for programmer instruments. Could you please try using the Studio::EventDescription::loadSampleData to preload data for events associated with your timeline clips before they are needed?
Hope this helps, let me know if you have any questions.
Thanks for your answer! Here is the info you asked for: Unity version : 2022.3.11f1 FMOD version : 2.02.24 FMOD for Unity version : 2.2.24
I did try to use this method, but even when using 1000 or 5000 as a parameter the sound still played from the very beginning. Here is a snippet of it (this one is the code used at runtime) :
...
#Above is identical to FMODEventPlayable
foreach (var param in Parameters)
{
eventInstance.setParameterByID(param.ID, param.Value);
}
GCHandle stringHandle = GCHandle.Alloc(Key);
eventInstance.setUserData(GCHandle.ToIntPtr(stringHandle));
eventInstance.setCallback(new FMOD.Studio.EVENT_CALLBACK(DialogueEventCallback));
eventInstance.setVolume(CurrentVolume);
eventInstance.setTimelinePosition((int)(ClipStartTime * 1000.0f)); // Doesn't seem to change anything, even with hard values like 5000
eventInstance.start();
eventInstance.release();
}
You’ll find the code is largely identical to PlayEvent() from FMODEventPlayable, with the addition of the key used to find the localized audio and the callback to play the programmer sound.
As for the content of the DialogueEventCallback callback, here is the CREATE_PROGRAMMER_SOUND part of it:
case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
{
MODE soundMode = MODE.CREATESTREAM;
var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
if (key.Contains("."))
{
Sound dialogueSound;
var soundResult = RuntimeManager.CoreSystem.createSound(Application.streamingAssetsPath + "/" + key, soundMode, out dialogueSound);
if (soundResult == RESULT.OK)
{
parameter.sound = dialogueSound.handle;
parameter.subsoundIndex = -1;
Marshal.StructureToPtr(parameter, parameterPtr, false);
}
}
else
{
FMOD.Studio.SOUND_INFO dialogueSoundInfo;
var keyResult = RuntimeManager.StudioSystem.getSoundInfo(key, out dialogueSoundInfo);
if (keyResult != RESULT.OK)
{
break;
}
Sound dialogueSound;
var soundResult = RuntimeManager.CoreSystem.createSound(dialogueSoundInfo.name_or_data, soundMode | dialogueSoundInfo.mode, ref dialogueSoundInfo.exinfo, out dialogueSound);
if (soundResult == RESULT.OK)
{
parameter.sound = dialogueSound.handle;
parameter.subsoundIndex = dialogueSoundInfo.subsoundindex;
Marshal.StructureToPtr(parameter, parameterPtr, false);
}
}
break;
}
Once again, it should be largely identical to the code provided in the programmer instrument tutorial.
You’ll see that there is a placeholder .wav filled in the event, but once in Unity we do load the audio associated with the given key and not that placeholder .wav. Also I noticed that the instrument was Async, but even when disabling it and reexporting the banks SetTimelinePosition didn’t work.
And finally, here is the editor code for previewing events, once again, similar to the code of PreviewEvent from EditorUtils:
public static FMOD.Studio.EventInstance PreviewLocalizedEvent(EditorEventRef eventRef, Dictionary<string, float> previewParamValues, float volume = 1, float startTime = 0.0f, string key = "")
{
FMOD.Studio.EventDescription eventDescription;
FMOD.Studio.EventInstance eventInstance;
EditorUtils.CheckResult(EditorUtils.System.getEventByID(eventRef.Guid, out eventDescription));
EditorUtils.CheckResult(eventDescription.createInstance(out eventInstance));
foreach (EditorParamRef param in eventRef.Parameters)
{
FMOD.Studio.PARAMETER_DESCRIPTION paramDesc;
if (param.IsGlobal)
{
EditorUtils.CheckResult(EditorUtils.System.getParameterDescriptionByName(param.Name, out paramDesc));
}
else
{
EditorUtils.CheckResult(eventDescription.getParameterDescriptionByName(param.Name, out paramDesc));
}
float value = previewParamValues.ContainsKey(param.Name) ? previewParamValues[param.Name] : param.Default;
param.ID = paramDesc.id;
if (param.IsGlobal)
{
EditorUtils.CheckResult(EditorUtils.System.setParameterByID(param.ID, value));
}
else
{
EditorUtils.CheckResult(eventInstance.setParameterByID(param.ID, value));
}
}
// Pin the key string in memory and pass a pointer through the user data
GCHandle stringHandle = GCHandle.Alloc(key);
eventInstance.setUserData(GCHandle.ToIntPtr(stringHandle));
//eventInstance.set3DAttributes(FMODUnity.RuntimeUtils.To3DAttributes(transform));
eventInstance.setCallback(new FMOD.Studio.EVENT_CALLBACK(DialogueEventCallback));
EditorUtils.CheckResult(eventInstance.setVolume(volume));
EditorUtils.CheckResult(eventInstance.start());
//EditorUtils.CheckResult(eventInstance.release());
previewEventInstances.Add(eventInstance);
return eventInstance;
}
As for DialogueEventCallback, it’s the same as the one used previously.
Thank you, I will try that.
I understand that I’m throwing a lot of text at you. I hope my explanations are clear enough. Thanks for any help you may provide!
Thank you for sharing such detailed setup and code.
Unfortunately, I wasn’t able to reproduce the issue from my end with the information provided. Given the complexity of your project, would it be possible to upload your project (or a stripped down version that still demonstrates the issue) to your FMOD user profile? This would allow me to take a closer look and better understand what might be going wrong.
Good news! While making the repro project got most of my issues sorted.
My issue with the inability to play the localized sounds simultaneously in the timeline was due to me not handling banks properly: I unloaded and loaded the localized banks every time I started a new clip.
The problem with the sounds not playing at the correct position was apparently that the instruments were not Async in FMOD. I was sure that I did try to set them not Async before but alas I supposed I must have made a mistake then.
