Fmod FFT for lip sync - can't get channel group

OK… I’m a little (or maybe a lot) out of my depth here. I can get the master bus easy and it works fine but not a separate return. I’ve also tried to route the event into a new group on the mix page but with the same result.



Screenshot 2023-08-09 at 18.18.55

Hi,

While I cannot tell from the small code snippet that you’ve posted, the issue here is likely that the Bus’ underlying ChannelGroup doesn’t exist until another sound (i.e. an event routed to the bus) is playing into it.

What complicates this further is that creating and starting your event instance will not actually cause the ChannelGroup to be created until the Studio system has asynchronously processed the event instance creation and start commands. As a result, if you are doing this at game start, you may need to move some of your code into Update() to ensure the ChannelGroup has been created.

The following code is a simple example of creating and starting an event instance that is routed into the Return Bus “bus:/VoiceDX”, getting the Bus it is routed to and waiting for the Bus’ ChannelGroup to be created by the Studio system:

using FMODUnity;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class GetBusChannelGroup : MonoBehaviour
{

    private FMOD.Studio.Bus bus;
    public EventReference eventRef;
    private FMOD.Studio.EventInstance eventInstance;
    private FMOD.ChannelGroup channelGroup;

    // Start is called before the first frame update
    void Start()
    {

        eventInstance = FMODUnity.RuntimeManager.CreateInstance(eventRef);
        eventInstance.start();
        eventInstance.release();

        bus = FMODUnity.RuntimeManager.GetBus("bus:/VoiceDX");
    }

    // Update is called once per frame
    void Update()
    {
        if (!channelGroup.hasHandle())
        {
            Debug.Log(bus.getChannelGroup(out channelGroup));
        }
        
    }
}

If the above code snippet doesn’t help with resolving your issue, could I get your FMOD for Unity version, and a log where the issue occurs with your Logging Level to “Log”?

Hi Louis

Thank you for your reply. The script is the FFT template I’m just pulling the line renderer data as a separate var. Your explanation makes sense to me and I just need to implement it. I’m still a bit fuzzy as to why the master bus “exists” at the start but a separate bus doesn’t. I realise that this is most likely due to me not understanding the overall implementation and from your suggestion probably not something I really need to get. I’m going to try this and also maybe just try and implement this using a coroutine rather than an in the Update but heyho… baby elephant steps… Thanks again for your feedback.

I assume that you mean that you don’t understand why the master channel group exists at the start, but the channel group of a separate bus does not?

All buses should exist (have a handle) at runtime, including both the master bus and “VoiceDX” return bus, as long as your master bank is loaded, and they should be retrievable by string as long as the master strings bank is loaded. It’s just that the underlying ChannelGroups do not exist until there is actually a sound playing into them. The master bus is an exception to this, and the underlying master ChannelGroup should exist at all times.

If you’re planning to interact directly with the Core API, then I’d say an understanding of some of this wouldn’t go amiss, but it depends on what/how much you’re doing with the Core API. That said, if you are interested, I would recommend connecting to your Unity game with the Core API Profiler, which will allow you to visualize the DSP graph and what changes are made to it when you interact with various API functions. The Core API Profiler can be located in your FMOD Studio API install directory in the ./bin folder.

No problem! If you have any further questions, feel free to ask.

1 Like

Hi @Leah_FMOD

Still a hack and slash job but got the bus allocation to work properly. I’m not able to restart the process on trigger exit without destroying objects and such but that’s for a later day. Posting the code here in case it’s useful for someone. Thanks for the help.

https://youtu.be/pB0uceAhpmg

using System;
using UnityEngine;
using FMOD.Studio;
using FMODUnity;
using FMOD;
using System.Runtime.InteropServices;
using Debug = UnityEngine.Debug;

public class GetFMODAmplitude : MonoBehaviour
{
    private ChannelGroup channelGroup;
    public Bus bus;
    private bool isCheckingHandle = false;
    [Header("Add the relevant bus here")]
    [SerializeField] string fmodBus;
    IntPtr unmanagedData;
    uint length;
    int i;


    //fmod script https://www.fmod.com/docs/2.01/unity/examples-spectrum-analysis.html
    const int WindowSize = 1024;
    private DSP mFFT;
    private float[] mFFTSpectrum;
    public float amplitude = 0;
    DSP_PARAMETER_FFT fftData;


    private void Start()
    {
        isCheckingHandle = true;  
    }

    public void StartDX()
    {
        bus = RuntimeManager.GetBus(fmodBus);
    }

    public void StopDX()
    {
       
    }

    private void FFT()
    {
        if (RuntimeManager.CoreSystem.createDSPByType(DSP_TYPE.FFT, out mFFT) == RESULT.OK)
        {
            mFFT.setParameterInt((int)DSP_FFT.WINDOWTYPE, (int)DSP_FFT_WINDOW.HANNING);
            mFFT.setParameterInt((int)DSP_FFT.WINDOWSIZE, WindowSize * 2);
            RuntimeManager.StudioSystem.flushCommands();

            if (bus.hasHandle())
            {
                // Get the channel group
               if (bus.getChannelGroup(out channelGroup) == RESULT.OK)
                {
                    channelGroup.addDSP(CHANNELCONTROL_DSP_INDEX.HEAD, mFFT);
                }
            }
        }
    }

    void OnDestroy()
    {
        if (bus.hasHandle())
        {
            if (bus.getChannelGroup(out channelGroup) == RESULT.OK)
            {
                if (mFFT.hasHandle())
                {
                    channelGroup.removeDSP(mFFT);
                }
            }
        }
    }

    void Update()
    {
        
        if (!channelGroup.hasHandle())
        {  
            bus.getChannelGroup(out channelGroup);
        }
        else
        {
            if (isCheckingHandle)
            {
                FFT();
                isCheckingHandle = false;
            }
        }

        if (mFFT.hasHandle())
        {
            if (mFFT.getParameterData((int)DSP_FFT.SPECTRUMDATA, out unmanagedData, out length) == RESULT.OK)
            {
                fftData = (DSP_PARAMETER_FFT)Marshal.PtrToStructure(unmanagedData, typeof(DSP_PARAMETER_FFT));
                if (fftData.numchannels > 0)
                {
                    if (mFFTSpectrum == null)
                    {
                        // Allocate the fft spectrum buffer once
                        for (int i = 0; i < fftData.numchannels; ++i)
                        {
                            mFFTSpectrum = new float[fftData.length];
                        }
                    }
                    fftData.getSpectrum(0, ref mFFTSpectrum);

                    /// Calculate amplitude (average magnitude)
                    /// use this to affect other params eg animation for lipsync

                    for (i = 0; i < WindowSize; ++i)
                    {
                        amplitude += Mathf.Abs(mFFTSpectrum[i]);
                    }
                    amplitude /= WindowSize * 10000f;
                }
            }
        }
    }
}

1 Like