How to get FFT Data for Both Outputs of a Stereo Bus

Hi there, I have been looking into utilizing spectral analysis in a project I am working on and have been following and using the documented code example linked below: https://fmod.com/docs/2.02/unity/examples-spectrum-analysis.html

I am just wondering what changes would need to be made to this code to allow for it to display the summed left and right signal from the bus instead of just the left? I am using this to analyse 3D audio but don’t want the spacialiser to effect the FFT output.

Also how would I change the code to make it display the right channel output spectrum instead of the left channel?

Thank you,
Aidan

The channels of the FFT’s spectrum data are simply arranged in order. If you want to get the FFT of the right channel instead of the left, instead of:

fftData.getSpectrum(0, ref mFFTSpectrum);

You can do:

fftData.getSpectrum(1, ref mFFTSpectrum);

Alternatively, if you want to retrieve the spectrum data for all channels, you can use the other getSpectrum overload which passes the data for all channels out by reference:

// create and initialize buffer for all channels
float[][] allFFTData = new float[fftData.numchannels][];
for (int i = 0; i < fftData.numchannels; ++i)
{
    allFFTData[i] = new float[fftData.length];
}
// pass buffer to function by reference to get spectrum data for all channels
fftData.getSpectrum(ref allFFTData);

Note that summing the FFT data from all channels isn’t the same as an FFT of the sum of all channels, as the former will only give you the total signal power, which may be all you need for your purposes.

If not, the simplest way to sum the channels before the FFT would be to use DSP::setChannelFormat to set the DSP to mono, which will force a downmix before the FFT occurs. This will affect the rest of the signal chain though, so you may wish to create a split in the signal and place your FFT DSP in parallel with a ChannelGroup instead of inserting it into the ChannelGroup. To do this, instead of adding the DSP to the ChannelGroup, you’ll need to manually connect it to another DSP like so:

// add FFT DSP parallel to signal 
// get head DSP of channel group
FMOD.DSP cgHead;
channelGroup.getDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, out cgHead);
// creates a loop in signal - potentially bad, but we mute the FFT DSP's output so it's fine
mFFT.addInput(cgHead);
cgHead.addInput(mFFT);
mFFT.setWetDryMix(1, 0, 0);
// set pre-processing downmix to mono for FFT DSP
mFFT.setChannelFormat(FMOD.CHANNELMASK.MONO, 1, FMOD.SPEAKERMODE.MAX);
// set FFT DSP as active
mFFT.setActive(true);

Additionally, if you wanted to avoid any spatialization affecting your FFT, you could place your FFT DSP (either in sequence or in parallel) before the spatializer in your DSP chain.

Ahhh I knew I would’ve missed something silly with left and right! Thank you for that!

I am trying to apply the code changes for summing and using allFFTData instead of using mFFTSpectrum but am having some trouble with -
float level = lin2dB(allFFTData[i]);

giving error: Argument 1: cannot convert from ‘float’ to ‘float’

Would you be able to have a look over this to spot anywhere I seem to have made a mistake when editing the original script?

//--------------------------------------------------------------------
//
// This is a Unity behaviour script that demonstrates how to access
// the Core API channel group of a bus, and how to create and
// add an FFT DSP. 
//
// This document assumes familiarity with Unity scripting. See
// https://unity3d.com/learn/tutorials/topics/scripting for resources
// on learning Unity scripting. 
//
//--------------------------------------------------------------------

using System;
using UnityEngine;
using System.Runtime.InteropServices;

class ScriptUsageFFT : MonoBehaviour
{
    private FMOD.DSP mFFT;
    private LineRenderer mLineRenderer;
    private float[] mFFTSpectrum;
    float[][] allFFTData;
    const int WindowSize = 8;
    private FMOD.Studio.Bus bus;

    void Start()
    {
        mLineRenderer = gameObject.AddComponent<LineRenderer>();
        bus = FMODUnity.RuntimeManager.GetBus("bus:/FFT");
        mLineRenderer.positionCount = WindowSize;
        mLineRenderer.startWidth = mLineRenderer.endWidth = 0.1f;

        // Create a DSP of DSP_TYPE.FFT
        if (FMODUnity.RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.FFT, out mFFT) == FMOD.RESULT.OK)
        {
            mFFT.setParameterInt((int)FMOD.DSP_FFT.WINDOWTYPE, (int)FMOD.DSP_FFT_WINDOW.HANNING);
            mFFT.setParameterInt((int)FMOD.DSP_FFT.WINDOWSIZE, WindowSize * 2);
            FMODUnity.RuntimeManager.StudioSystem.flushCommands();

            // Get the master bus (or any other bus for that matter)
            FMOD.Studio.Bus selectedBus = FMODUnity.RuntimeManager.GetBus("bus:/FFT");
            if (selectedBus.hasHandle())
            {
                // Get the channel group
                FMOD.ChannelGroup channelGroup;
                if (selectedBus.getChannelGroup(out channelGroup) == FMOD.RESULT.OK)
                {
                    // Add fft to the channel group
                    if (channelGroup.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, mFFT) != FMOD.RESULT.OK)
                    {
                        Debug.LogWarningFormat("FMOD: Unable to add mFFT to the master channel group");
                    }
                }
                else
                {
                    Debug.LogWarningFormat("FMOD: Unable to get Channel Group from the selected bus");
                }
            }
            else
            {
                Debug.LogWarningFormat("FMOD: Unable to get the selected bus");
            }
        }
        else
        {
            Debug.LogWarningFormat("FMOD: Unable to create FMOD.DSP_TYPE.FFT");
        }
    }

    void OnDestroy()
    {
        FMOD.Studio.Bus selectedBus = FMODUnity.RuntimeManager.GetBus("bus:/FFT");
        if (selectedBus.hasHandle())
        {
            FMOD.ChannelGroup channelGroup;
            if (selectedBus.getChannelGroup(out channelGroup) == FMOD.RESULT.OK)
            {
                if(mFFT.hasHandle())
                {
                    channelGroup.removeDSP(mFFT);
                }
            }
        }
    }

    const float WIDTH = 10.0f;
    const float HEIGHT = 0.1f;

    void Update()
    {
        if (mFFT.hasHandle())
        {
            IntPtr unmanagedData;
            uint length;
            if (mFFT.getParameterData((int)FMOD.DSP_FFT.SPECTRUMDATA, out unmanagedData, out length) == FMOD.RESULT.OK)
            {
                FMOD.DSP_PARAMETER_FFT fftData = (FMOD.DSP_PARAMETER_FFT)Marshal.PtrToStructure(unmanagedData, typeof(FMOD.DSP_PARAMETER_FFT));
                float[][] allFFTData = new float[fftData.numchannels][];
                if (fftData.numchannels > 0)
                {
                    if (allFFTData == null)
                    {
                        // Allocate the fft spectrum buffer once
                        for (int i = 0; i < fftData.numchannels; ++i)
                        {
                            //mFFTSpectrum = new float[fftData.length];
                            allFFTData[i] = new float[fftData.length];
                        }
                    }
                    //fftData.getSpectrum(0, ref mFFTSpectrum);
                    // pass buffer to function by reference to get spectrum data for all channels
                    fftData.getSpectrum(ref allFFTData);

                    var pos = Vector3.zero;
                    pos.Set(15, 1, 34);
                    pos.x = WIDTH * -0.5f;

                    for (int i = 0; i < WindowSize; ++i)
                    {
                        pos.x += (WIDTH / WindowSize);

                        //float level = lin2dB(mFFTSpectrum[i]);
                        float level = lin2dB(allFFTData[i]);
                        pos.y = (80 + level) * HEIGHT;

                        mLineRenderer.SetPosition(i, pos);
                        
                    }
                }
            }
        }
    }

    private float lin2dB(float linear)
    {
        return Mathf.Clamp(Mathf.Log10(linear) * 20.0f, -80.0f, 0.0f);
    }
}

I really appreciate the help!

allFFTData is a two dimensional float array - the first dimension is the channels, and the second dimension is the FFT data for each channel. You’re getting the error because you’re only accessing the channels with allFFTData[i] which returns a float array, and are passing it to lin2dB(), which is expecting a float.

What you presumably want to do is iterate over the channels, and for each channel iterate over the data points i.e. for ints i and j, do float level = lin2dB(allFFTData[i][j]);