Getting spectrum of master channel in Unity...

Unity
1

Hey all! I’m new to FMOD and I’m using it to create an audio visualizer in Unity (so I’m using c#). I have the visualizer working using the getSpectrum function from Unity’s native audio engine, but I’d like to switch to FMOD. Unfortunately, I’m unsure of how exactly to do this.

From what I understand, I need to create a custom DSP, add it to my master channel, and then use the getParameterData function on it in update in order to get the frequency spectrum at every frame.

I suppose my question is…where exactly in my code am I supposed to add the DSP? From looking through the code, it seems the system created by FMOD is only accessible in some editor scripts (from which I cannot call functions), and in a script called “RuntimeManager.cs.” Is RuntimeManager.cs where I’m supposed to add the DSP? Thanks in advanced!

2

Here’s an example I wrote but never included in the docs due to lack of polish

using System;
using UnityEngine;
using System.Runtime.InteropServices;

class ScriptUsageLowLevel : MonoBehaviour
{
    FMOD.Studio.EventInstance musicInstance;
    FMOD.DSP fft;

    LineRenderer lineRenderer;

    const int WindowSize = 1024;

    void Start()
    {        
        lineRenderer = gameObject.AddComponent<LineRenderer>();
        lineRenderer.SetVertexCount(WindowSize);
        lineRenderer.SetWidth(.1f, .1f);

        musicInstance = FMODUnity.RuntimeManager.CreateInstance("event:/Music/Basic/Random Layered");


        FMODUnity.RuntimeManager.LowlevelSystem.createDSPByType(FMOD.DSP_TYPE.FFT, out fft);
        fft.setParameterInt((int)FMOD.DSP_FFT.WINDOWTYPE, (int)FMOD.DSP_FFT_WINDOW.HANNING);
        fft.setParameterInt((int)FMOD.DSP_FFT.WINDOWSIZE, WindowSize * 2);

        FMOD.ChannelGroup channelGroup;
        FMODUnity.RuntimeManager.LowlevelSystem.getMasterChannelGroup(out channelGroup);
        channelGroup.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, fft);

        musicInstance.start();
    }

    const float WIDTH = 10.0f;
    const float HEIGHT = 0.1f;

    void Update()
    {
        IntPtr unmanagedData;
        uint length;
        fft.getParameterData((int)FMOD.DSP_FFT.SPECTRUMDATA, out unmanagedData, out length);
        FMOD.DSP_PARAMETER_FFT fftData = (FMOD.DSP_PARAMETER_FFT)Marshal.PtrToStructure(unmanagedData, typeof(FMOD.DSP_PARAMETER_FFT));
        var spectrum = fftData.spectrum;

        if (fftData.numchannels > 0)
        {
            var pos = Vector3.zero;
            pos.x = WIDTH * -0.5f;

            for (int i = 0; i < WindowSize; ++i)
            {
                pos.x += (WIDTH / WindowSize);

                float level = lin2dB(spectrum[0][i]);
                pos.y = (80 + level) * HEIGHT;

                lineRenderer.SetPosition(i, pos);
            }
        }
    }

    float lin2dB(float linear)
    {
        return Mathf.Clamp(Mathf.Log10(linear) * 20.0f, -80.0f, 0.0f);
    }
}
1 Like
3

You are a lifesaver! Thanks a bunch :slight_smile:

4

Hey @NicholasWilcox I keep getting 0 channels listed for my fft spectrum, any idea why that may be?

–edit : nvm, I think it was because I was creating a song instead of an instance

5

Check the return of the FMOD functions, most will return a FMOD_RESULT.
Specifically check addDSP().
Also, it may sound obvious but make sure the sound is playing before trying to read the data.

6

Hey Cameron thanks yeah, it was playing, but I was also just using a stand alone song which didn’t seem to read anything. As soon as I created an instance it worked perfectly! :smiley:

7

FYI

FMODUnity.RuntimeManager.LowlevelSystem

has been replaced with

FMODUnity.RuntimeManager.CoreSystem

Unity 2018.3.5f1
FMOD 2.00.03

1 Like
8

The script works fine but I am having trouble assigning the DSP just to my EventInstance and getting that information fed back into the spectrum visualization.

All I did was this:

FMOD.ChannelGroup channelGroup;
 musicInstance.start();

  yield return new WaitForSeconds(1.0f);
  //FMODUnity.RuntimeManager.LowlevelSystem.getMasterChannelGroup(out channelGroup);
  musicInstance.getChannelGroup(out channelGroup);
  channelGroup.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, fft);

While profiling with the lowlevel FMOD Profiler app, the FFT module correctly appears in the chain.

Not sure what else to do right now. The information here https://fmod.com/resources/documentation-api?page=content/generated/FMOD_Studio_EventInstance_GetChannelGroup.html#/ hasn’t helped I am afraid.

9

Nevermind, I found the problem and it had nothing to do with the code :slight_smile:
The EventInstance.getChannelGroup() method works just fine.

10

I am back with a question:
Why does the spectrum buffer always returns one that is only half full?
Ex: if I set a window-size of 512, only 256 samples are useful, the rest is a bunch of zeros.
If I set a window-size of 1024, only 512 are usable and so forth.

Why creating buffers that are half “garbage”? Isn’t that a waste of memory?
My understanding of why this is happening is cause of the nyquist frequency. However, while that makes sense in theory, why putting the filtering in the hands of the end user?

11

Hello!

I have the exact same problem but cant find a solution

As you I want do add the FFT-DSP to the channelgroup of the musicInstance but something doesnt work.
Could you describe what the error in your case was? Maybe I have the same :thinking:

12

Hi,
I also have the problem that when I add the fft dsp effect to a channelgroup of an event instance i don’t get any data whereas when the effect is added to the master bus all is fine.
How would I solve this?

My code is similar to the one above:

instance.getChannelGroup(out channelGroup);

//FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out channelGroup); //this works

channelGroup.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, fft); //this displays nothing

the rest of the code is essentially the same as the one posted by ZeroSuitSenpai.

Many thanks for your help
Constantin

13

We now have an example on our website for accessing the Spectrum Data, you will want to get the channelgroup from a Studio::Bus and not from the CoreSystem.

1 Like
14

hi cameron,

many thanks for the link and tip. i didn’t realise the difference between the studio::bus and coresystem, yet. I’ll read into this. :slight_smile:

:slight_smile:
constantin