Hi there 
I’m currently trying to convert a speech recognition solution work with FMOD audio, rather than the built-in Unity Audio. I’m stumped at a point in the script where the Unity method AudioClip.GetData is used to receive some data. I’m wondering if there is a way to extract the same data from FMOD sounds?
I’m aware of a method called ReadData in FMOD, but I was, unfortunately, unable to apply it to get the desired values.
I’m a bit new to working this deeply with FMOD, so any help is appreciated
Hi,
There is a way to get the sample data from an audio source but there is a bit of setup to do, we will need to create a Sound to read the data from. Here is an example of how:
GetSampleData(string filePath)
/// <summary>
/// Return sample data in a byte array from an audio source using its file path
/// </summary>
/// <param name="filePath"></param>
/// <returns></returns>
private byte[] GetSampleData(string filePath)
{
// Very useful tool for debugging FMOD function calls
FMOD.RESULT result;
// Sound variable to retrieve the sample data from
FMOD.Sound sound;
// Creating the sound using the file path of the audio source
// Make sure to create the sound using the MODE.CREATESAMEPLE | MDOE.OPENONLY so the sample data can be retrieved
result = FMODUnity.RuntimeManager.CoreSystem.createSound(filePath, FMOD.MODE.CREATESAMPLE | FMOD.MODE.OPENONLY, out sound);
// Debug the results of the FMOD function call to make sure it got called properly
if (result != FMOD.RESULT.OK)
{
Debug.Log("Failed to create sound with the result of: " + result);
return null;
}
// Retrieving the length of the sound in milliseconds to size the arrays correctly
result = sound.getLength(out uint length, FMOD.TIMEUNIT.MS);
if (result != FMOD.RESULT.OK)
{
Debug.Log("Failed to retrieve the length of the sound with result: " + result);
return null;
}
// Buffer which the sample data will be copied too
System.IntPtr buffer;
// Creating a pointer to newly allocated memory
// This pointer must be released at the end of the function!
buffer = Marshal.AllocHGlobal((int)length * sizeof(byte));
// Creating the return array which will have the sample data is a readable variable type
// Using the length of the sound to create it to the right size
byte[] byteArray = new byte[(int)length];
// Retrieving the sample data to the pointer using the full length of the sound
result = sound.readData(buffer, length, out uint read);
if (result != FMOD.RESULT.OK)
{
Debug.Log("Failed to retrieve data from sound: " + result);
return null;
}
// Make sure the pointer has been populated
if (_buffer != System.IntPtr.Zero)
{
// Coping the data from our pointer to our usable array
Marshal.Copy(buffer, byteArray, 0, (int)length);
}
//Releasing the pointer
Marshal.FreeHGlobal(buffer);
//Returning the array populated with the sample data to be used
return byteArray;
}
Hopefully, calling Sound::readData this way will get you the values you are looking for!
Hope this helps!
Hello,
I have the exact same issue as the operator. I want to make use of the audioclip data in Unity, but with FMOD.
I want to use the audioclip’s data for the correct mouth synchronization in a dialogue system.
I have tried using your approach, which seems like the right approach
First of all, from which location do I get the file path? I’ve tried several file paths based on my folder structure already, but with no luck. What’s the correct path structure? Can you give me an example?
Also, I don’t seem to understand why your example requires a file path. Why not just use the event path instead to retrieve the audioclip?
I’ve also had a look at callbacks, and it seems to give me what I’m looking for, but it somehow only uses the master channel group. In my case I have a voice channel I wanna use. https://www.fmod.com/docs/2.02/unity/examples-dsp-capture.html
I think your documentation about how to implement this logic is kinda vague. It’s not really helping me.
Well I got your points here. Thanks for sharing
1 Like
Hi,
The file path I am using is: "C:\Users\XXX\Music\Epic SciFi Music.wav", but this was purely for the example. There are different ways to create a sound which can be found under FMOD API | Core API Reference.
In the example, as the DSP is being added to the MasterChannelGroup it is the only one that is needed but there are a number of ways to retrieve a channel. When referencing your voice channel could you elaborate on what it is? Could you screen shot it in your FMOD Studio Project?
Hi,
Thanks for your quick apply. Much appreciated.
Oh, I had no idea that I needed the full path including the filename. Just wondering. Doesn’t this mess up the code if the game is being built?
And also, I can’t really see the benefits in this logic, because then I would have to somewhere store hundreds of paths of dialogue, which is basically already are stored in the FMOD project.
About DSP. Maybe I did it wrong. Still trying to get my head around it. I want to rule out all audio but keep the Voice for the facial animations, because I don’t want other background noise to interfere.
Maybe I need a different approach?
Hi,
Again, this was just an example. Depending on where you are getting the audio source this will be different. E.g. Using an event.
Here is a reliable way to get the Voice Group from an Event Emitter. Make sure this is called after the event has started:
emitter = GetComponent<FMODUnity.StudioEventEmitter>();
if (emitter == null)
return;
emitter.EventInstance.getChannelGroup(out FMOD.ChannelGroup masterChannelGroup);
masterChannelGroup.getGroup(0, out voiceGroup);
I am then attaching the DSP from the example to the voiceGroup.
Hope this helps!
Thanks for your reply, but it didn’t really help me.
Everything works perfectly on the masterchannel group, but I only want the “Voice” channel.
I’ve tried to get the channelgroup via the bus, based on the path to the voice channel. The Voice bus exists. I know that for a fact because I debugged it.
However, it wont allow me to add a DSP onto that channelgroup for some reason. The addDSP requires an index, but I’m kinda confused about which index it requires and what it’s for. I’ve tried several indexes like these [-3,-2,-1,0,1,2,30], but with the same outcome.
Apart from the Start function, I’ve also tried to add the DSP in runtime and while the NPC is talking, but it prints the same error.
So the issue with this method is that the channelGroup may not be ready when you are trying to assign the DSP to it. To resolve this we need to use a couple of commands:
bus.lockChannelGroup() will ensure that the channelGroup is created and will stay in memory, it will have to be unlocked in the future. (FMOD API | Studio API Reference)
FMODUnity.RuntimeManager.StudioSystem.flushCommands(); forces all commands to be called before continuing to ensure that the commands we are calling are being processed by the Studio System. (FMOD API | Studio API Reference).
FMODUnity.RuntimeManager.StudioSystem.flushSampleLoading() forces all sample loading and unloading to be completed so our calls can be run. (FMOD API | Studio API Reference)
if (FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out currentMAster) == FMOD.RESULT.OK)
{
FMODUnity.RuntimeManager.StudioSystem.getBus("bus:/In-World/Voice", out bus);
bus.lockChannelGroup();
FMODUnity.RuntimeManager.StudioSystem.flushCommands();
FMODUnity.RuntimeManager.StudioSystem.flushSampleLoading();
bus.getChannelGroup(out ChannelGroup group);
if (FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out mCaptureDSP) == FMOD.RESULT.OK)
{
if (!currentMAster.hasHandle())
{
Debug.Log("No master");
return;
}
FMOD.RESULT result = currentMAster.addDSP(0, mCaptureDSP);
Debug.Log(result.ToString());
if (result != FMOD.RESULT.OK)
{
Debug.LogWarningFormat("FMOD: Unable to add mCaptureDSP to the master channel group");
}
else
success = true;
}
else
{
Debug.LogWarningFormat("FMOD: Unable to create a DSP: mCaptureDSP");
}
}
else
{
Debug.LogWarningFormat("FMOD: Unable to create a GCHandle: mObjHandle");
}
Try updating your code with this, hope it helps!
Thanks for your reply! Unfortunately It’s still not working. I’m still always getting the master channel’s audio just like before and with no errors.
I’m wondering why you are assigning “group”, if it isn’t going to be used. I did try to replace “currentMaster” with “group” but then the debug warning error shows up again. (Recent screenshot)
Apologies for that, I have included the whole script without reference issues.
EventDataAndBus.txt (5.1 KB)
That works perfectly!
Thank you so much for your help!
1 Like
