Getting Current Event Waveform information/volume

Hey,

In previous Unity projects, I’ve sampled an audio source’s current waveform volume (the amplitude of the waveform at the time its currently playing essentially, post effects/ect) and piped it into a simple ‘talker’ script that basically passes that value (multiplied by a constant) to an animation to get the actor to ‘talk’ by opening its mouth when the volume is loud and closing it when its quiet.

Is there a way to access this data easily in Fmod? I dont see anything obvious in the API so i figured I’d ask here.

				sampleTime = Mathf.Min (audioSource.timeSamples + sampleOffset, audioSource.clip.samples - sampleDataLength);
                currentUpdateTime = 0f;
				audioSource.clip.GetData(clipSampleData, sampleTime); //I read 1024 samples, which is about 80 ms on a 44khz stereo clip, beginning at the current sample position of the clip.
                clipLoudness = 0f;
                i = 0;
				for (i = 0; i < clipSampleData.Length; i++)
                {
                    clipLoudness += Mathf.Abs(clipSampleData[i]);
                }

		clipLoudness /= sampleDataLength; //clipLoudness is what you are looking for

is roughly the code we use for unity audio sources if that helps clarify what type of data i’m trying to access.

1 Like

You can access peak (loudest sample in a given audio buffer) and rms (root mean squared, average sample value) volume data by enabling metering on any of the channel’s DSPs with DSP::setMeteringEnabled, and retrieve its FMOD_DSP_METERING_INFO using DSP::getMeteringInfo.
Here is a simple example of how to retrieve the left and right volumes of a playing event instance.

DisplayEventSound.cs
using System.Collections;
using UnityEngine;

public class DisplayEventSound : MonoBehaviour
{
    public FMODUnity.EventReference eventReference;

    public float leftVolume;
    public float rightVolume;

    FMOD.DSP_METERING_INFO inputInfo, outputInfo;

    FMOD.Studio.PLAYBACK_STATE playbackState;
    FMOD.DSP dsp;

    void Start()
    {
        StartCoroutine(PlayEventAsync());
    }

    IEnumerator PlayEventAsync()
    {
        FMODUnity.RuntimeManager.StudioSystem.getEvent(eventReference.Path, out FMOD.Studio.EventDescription eventDescription);
        eventDescription.createInstance(out FMOD.Studio.EventInstance eventInstance);
        eventInstance.set3DAttributes(FMODUnity.RuntimeUtils.To3DAttributes(gameObject));
        eventInstance.start();

        // Need to wait for event instance to play before getting channel group
        do
        {
            eventInstance.getPlaybackState(out playbackState);
            yield return null;
        }
        while (playbackState != FMOD.Studio.PLAYBACK_STATE.PLAYING);

        FMOD.ChannelGroup channelGroup;
        eventInstance.getChannelGroup(out channelGroup);

        channelGroup.getDSP(FMOD.CHANNELCONTROL_DSP_INDEX.FADER, out dsp);
        dsp.setMeteringEnabled(true, true);
    }

    // Update is called once per frame
    void Update()
    {
        if(playbackState == FMOD.Studio.PLAYBACK_STATE.PLAYING)
        {
            dsp.getMeteringInfo(out inputInfo, out outputInfo);
            leftVolume = outputInfo.rmslevel[0];
            rightVolume = outputInfo.rmslevel[1];
        }
    }
}

@jeff_fmod what if I trigger a whole series of dialogue by doing a PlayOneShot?

I won’t have a event instance because the way I did this is by using Command instruments that will command other events to fire off.

I have the player say one line of dialogue, then a command instrument at the end of that dialogue is triggered and another person speaks.

So I am simply starting the whole chain of dialogue by playing a OneShot.

How does one get volume info in this kind of situation?

If you need to do sophisticated things with your EventInstances, such as checking their volume in realtime, then PlayOneShot is probably not the best way to be playing event. Perhaps you could try creating your events with RuntimeManager.CreateInstance, and start / release them yourself.

If you are using Command Instruments then you can set an Event Callback on FMOD_STUDIO_EVENT_CALLBACK_START_EVENT_COMMAND, which will give you access to the newly created EventInstance in the parameters argument. You can then access the ChannelGroup and get the DSP metering as suggested above. For an example of how to assign a callback to an EventDescription you can refer to the Programmer Sounds scripting example.

I figured out a work around. I basically just did what you said and created instances, and had a list of events that chain off one another within my game. Ie, the dialogue is not chained in FMOD anymore.

This allowed me to get the volume from the event instances and animate character mouth movements.

Thanks!

1 Like

Is there a way to do this for Unreal?

Unfortunately, there’s no way to do this using Blueprints in Unreal. However, you can still accomplish this by interacting directly with the FMOD API using C++ code. In this case, you’ll want to access your event instance’s underlying channelgroup with Studio::EventInstance::getChannelGroup, retrieve a DSP from the channelgroup with ChannelControl::getDSP, and then follow the steps that Jeff outlined to get the metering info from the DSP. Jeff’s example is in C#, but it should still serve as a decent example of how you might go about doing this.

Please see our Unreal docs on Programming Support for more information on directly interacting with the FMOD C++ API.

@graycloud just following up on this - was my response and Jeff’s C# example sufficient to point you in the right direction? If not, was there anything you were unclear about?