# Fmod lipsync question fmod volume affect spectrum extracting

**URL:** <https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274>\
**Category:** Unity\
**Created:** [January 26, 2022, 4:16am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274 "2022-01-26T04:16:38Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [January 26, 2022, 4:16am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/1 "2022-01-26T04:16:38Z")

</div>

when i extract specturm from fmod the code is:  
RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP\_TYPE.FFT, out m\_FFTDsp);  
RuntimeManager.CoreSystem.getMasterChannelGroup(out master);  
var m\_Result = master.addDSP(FMOD.CHANNELCONTROL\_DSP\_INDEX.HEAD, m\_FFTDsp);  
m\_FFTDsp.setMeteringEnabled(true, true);

the lip sync is right.  
but when i change the volume, the lip action disappear.  
does volume affect spectrum.

extract spectrum code like this:  
public string RecognizeByAudioSource(FMOD.DSP m\_FFTDsp, int rate)  
{  
string result = null;  
IntPtr unmanagedData;  
uint length;  
m\_FFTDsp.getParameterData((int)FMOD.DSP\_FFT.SPECTRUMDATA, out unmanagedData, out length);  
FMOD.DSP\_PARAMETER\_FFT fftData = (FMOD.DSP\_PARAMETER\_FFT)Marshal.PtrToStructure(unmanagedData, typeof(FMOD.DSP\_PARAMETER\_FFT));  
if (fftData.spectrum != null && fftData.spectrum.Length \> 0)  
{  
playingAudioSpectrum = fftData.spectrum[0];  
Recognize(ref result, rate);  
}  
return result;  
}

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [January 31, 2022, 10:03pm UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/2 "2022-01-31T22:03:55Z")

</div>

I might need a little more context for your project to understand the problem here, what do you mean by lip sync?

> [@simingtu](#):
>
> does volume affect spectrum.

Yes, if you turn the volume to 0 there will be no spectrum data, and if you turn the volume up too high you will get clipping, which will reflect in the spectrum data.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 10, 2022, 2:48am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/3 "2022-02-10T02:48:43Z")

</div>

i want to use spectrum data to control person’s lip action.  
i extract vowel word to control the bone of lip.  
the bone weight of a, e, i, o, u extracted from the analysis of spectrum.  
so, if i turn the volume to zero, the specturm will be equal to zero, then i can not extract vowel word.

> **[bandicam 2020-04-28 20-58-07-386\_哔哩哔哩\_bilibili](https://www.bilibili.com/video/BV1BK411578P?from=search&seid=13134832780952677459&spm_id_from=333.337.0.0)**
>
> 语音口型同步演示blog: https://penghuailiang.gitee.io/blog/2020/mouth/github: https://github.com/huailiang/LipSync

when the player turn the bmg volum to zero, the role’s dialog lip action will disappear. it will affect the game experience.  
do u have other workaround?

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [February 10, 2022, 11:07pm UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/4 "2022-02-10T23:07:57Z")

</div>

Thank you for the additional information, I can see what you mean now. I think you will need to add a [Fader DSP](https://fmod.com/resources/documentation-api?version=2.02&page=core-api-common-dsp-effects.html#fmod_dsp_fader) after your [FFT DSP](https://fmod.com/resources/documentation-api?version=2.02&page=core-api-common-dsp-effects.html#fmod_dsp_type_fft), and then control the volume on the fader using [`setParameterFloat`](https://fmod.com/resources/documentation-api?version=2.02&page=core-api-dsp.html#dsp_setparameterfloat) with the `FMOD_DSP_FADER_GAIN` parameter as the index argument. Then you can hook that up to the player’s volume controls instead of a group/VCA and that should still allow the FFT to get data regardless of what the fader volume is set to.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 14, 2022, 3:49am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/5 "2022-02-14T03:49:11Z")

</div>

> [@jeff\_fmod](#):
>
> en co

i create a dsp and then add this dsp, but when i set volume to the fader dsp, it can not control the volume.  
code follows:

1.var result1 = RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP\_TYPE.FADER, out m\_faderDsp);  
2.RuntimeManager.CoreSystem.getMasterChannelGroup(out master);  
var result2 = master.addDSP(-1, FaderDsp);  
3.var result3 = FaderDsp.setParameterFloat((int)FMOD.DSP\_FADER.GAIN, volume);

can u help me solve this problem.

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [February 15, 2022, 4:44am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/6 "2022-02-15T04:44:53Z")

</div>

Can you please check that m\_faderDSP is not bypassed? e.g

```cpp
m_faderDSP.setBypass(false);

```

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 15, 2022, 8:28am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/7 "2022-02-15T08:28:43Z")

</div>

yes, m\_faderDsp is not bypass.

can u give me a more effective communication way?

```
private bool m_addedFaderDSP = false;
private FMOD.DSP m_faderDsp;
private FMOD.ChannelGroup master;

public void InitFaderDSP()
{
    if (m_addedFaderDSP) return;
    m_addedFaderDSP = true;
    var result = RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.FADER, out m_faderDsp);
    result = RuntimeManager.CoreSystem.getMasterChannelGroup(out master);
}

public void SetMainVolume(float volume)
{
    if (FMODUnity.RuntimeManager.Instance != null)
    {
        InitFaderDSP();

        //if (!mainVCA.isValid())
        // FMODUnity.RuntimeManager.StudioSystem.getVCA("vca:/Master", out mainVCA);
        //if (mainVCA.isValid()) mainVCA.setVolume(volume);
        //var res = master.setVolume(volume);
        var res = m_faderDsp.setParameterFloat((int)FMOD.DSP_FADER.GAIN, volume);
        Debug.LogError(res + " volume=" + volume);
    }
}
```

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [February 16, 2022, 1:40am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/8 "2022-02-16T01:40:45Z")

</div>

In this code snippet you are not calling `ChannelGroup.addDSP` on `master`, changing it to this is working for me:

```auto
    public void InitFaderDSP()
    {
        if (m_addedFaderDSP) return;
        m_addedFaderDSP = true;
        var result = RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.FADER, out m_faderDsp);
        result = RuntimeManager.CoreSystem.getMasterChannelGroup(out master);
        result = master.addDSP(-1, m_faderDsp); // Add DSP to master group
    }

```

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 16, 2022, 2:15am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/9 "2022-02-16T02:15:55Z")

</div>

sorry, i forgot to addDSP to master group.  
however, after i added the m\_faderDsp to master group,  
m\_faderDsp.setParameterFloat((int)FMOD.DSP\_FADER.GAIN, volume);  
still can not control volume.  
will the index -1 affect the fader dsp.  
master.addDSP(-1, m\_faderDsp);  
i do not clearly understand the index, i can not find the rule of index when i try add several dsp to mater.  
so trouble u again! 🤣

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 16, 2022, 2:28am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/10 "2022-02-16T02:28:12Z")

</div>

the full code is:  
using FMODUnity;  
using System.Collections;  
using System.Collections.Generic;  
using UnityEngine;  
using UnityEngine.UI;

public class FmodDsp : MonoBehaviour  
{  
public Slider m\_slider;  
private bool m\_addedFaderDSP = false;  
private FMOD.DSP m\_faderDsp;  
private FMOD.ChannelGroup master;

```
public void OnValueChange(float val)
{
    var res = m_faderDsp.setParameterFloat((int)FMOD.DSP_FADER.GAIN, m_slider.value);
    Debug.LogError(res + " volume=" + m_slider.value);
}

public void InitFaderDSP()
{
    if (m_addedFaderDSP) return;
    m_addedFaderDSP = true;
    var result = RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.FADER, out m_faderDsp);
    result = RuntimeManager.CoreSystem.getMasterChannelGroup(out master);
    result = master.addDSP(-1, m_faderDsp);
}

public void AddDSP()
{
    InitFaderDSP();
}

```

}

1.first init dsp  
2.and then dray the silder to control the volume. the rOnValueChange method is the callback of silder.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 16, 2022, 2:31am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/11 "2022-02-16T02:31:56Z")

</div>

![image](https://canada1.discourse-cdn.com/flex036/uploads/fmod/original/2X/3/3554d5e07f83230d67ec9121a57ee497196a24f0.png)

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [February 16, 2022, 2:59am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/12 "2022-02-16T02:59:50Z")

</div>

What are the min and max values on your slider? The [`FMOD.DSP_FADER.GAIN`](https://fmod.com/resources/documentation-api?version=2.02&page=core-api-common-dsp-effects.html#fmod_dsp_fader) parameter value needs to be in the range of -80 to 10.  
As for the -1 index, that coincides with placing the DSP at the head of the signal chain. You can read more about special locations here: [`FMOD_CHANNELCONTROL_DSP_INDEX`](https://fmod.com/resources/documentation-api?version=2.02&page=core-api-channelcontrol.html#fmod_channelcontrol_dsp_index). I believe you will need to place it at `FMOD_CHANNELCONTROL_DSP_INDEX.TAIL` (-3) for your use case.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 16, 2022, 4:24am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/13 "2022-02-16T04:24:51Z")

</div>

![image](https://canada1.discourse-cdn.com/flex036/uploads/fmod/original/2X/e/ec6170fdba3a8927db53ebcf4433756989cecb00.png)

thank u very much.  
i change the min and max, it can control volume.  
but my lip action still be affected by the volume.  
i put the fader dsp to -3 index, and fft dsp to -2 dsp as the picture above.  
so, what is the problem will be about?

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 16, 2022, 6:48am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/14 "2022-02-16T06:48:49Z")

</div>

i change the index of fft dsp to -2, and fader dsp index to -1.  
then the volume will not affect the specturm data.  
but i have a question, the index of dsp can be -3, -2, -1  
-3 and -1, which one is the head.  
i think fader dsp should be after the fft dsp.  
but from the result, we i set fft dsp index to -2, and fader dsp to -1, the reuslt is right.  
this seems -1 the tail.  
so can u explain the reason for me.

---

<div class="post-metadata">

**Author:** ![jeff\_fmod](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/jeff_fmod/32/1766_2.png) [@jeff\_fmod](https://qa.fmod.com/u/jeff_fmod)\
**Post date:** [February 16, 2022, 10:54pm UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/16 "2022-02-16T22:54:57Z")

</div>

> [@simingtu](#):
>
> but from the result, we i set fft dsp index to -2, and fader dsp to -1, the reuslt is right.  
> this seems -1 the tail.

I recommend you read the [DSP White Paper](https://fmod.com/resources/documentation-api?version=2.02&page=white-papers-dsp-architecture.html) which will explain everything in detail.

> [@simingtu](#):
>
> how to control tone volme and meanwhile that will not affect spectrum.

The problem is that the VCA is sitting before the FFT in your signal chain, so cutting off any audio from the VCA cuts off any audio that gets to your FFT. You will need to attach the FFT before the VO group. It is a little tricky attaching a dsp to a group, the easiest way would be to play an event that is on the VO group, then get it’s parent group, which should be the VO group. Then you can attach the FFT there. Here is a basic example just to illustrate what I’m talking about:

```auto
    void Start()
    {
        StartCoroutine(AddDSPAsync());
    }

    IEnumerator AddDSPAsync()
    {
        FMOD.RESULT result;
        result = RuntimeManager.StudioSystem.getEvent("event:/VO/Welcome", out FMOD.Studio.EventDescription ed);
        result = ed.createInstance(out FMOD.Studio.EventInstance instance);
        result = instance.start();
        
        FMOD.ChannelGroup cg;
        while(instance.getChannelGroup(out cg) != FMOD.RESULT.OK)
        {
            yield return null;
        }

        result = cg.getParentGroup(out FMOD.ChannelGroup vo);

        result = RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.FFT, out FMOD.DSP fft);
        result = vo.addDSP(-1, fft);

        result = instance.stop(0);
        result = instance.release();
    }

```

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 17, 2022, 3:49am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/17 "2022-02-17T03:49:28Z")

</div>

for example, i have two events A and B, there are both on tone vca.  
i play two events.  
i get event A channel group A\_C, and event b channel group B\_C.  
the parent group of event A is A\_PC, the parent group of event B is B\_PC,  
but the hashcode of A\_PC and B\_PC is not the same.

so, if i to add a fft dsp to A\_PC, it can not affect B\_PC.  
so why?

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 17, 2022, 7:02am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/18 "2022-02-17T07:02:26Z")

</div>

my simple demo is: [LipSync: 口型fmod](https://gitee.com/yichichunshui/lip-sync.git)

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/fmod/original/2X/1/14adbdce5ccc2ff51cb63bd90f5622543ff3a62a.jpeg)

can u help me to control mainvolume and tonevolume without affect lip action.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 17, 2022, 7:03am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/19 "2022-02-17T07:03:53Z")

</div>

![image](https://canada1.discourse-cdn.com/flex036/uploads/fmod/original/2X/e/e24b7a62f3904203dbbc53bcadb451f49b0f6833.png)  
double click FmodScene, and run unity.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 17, 2022, 9:52am UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/20 "2022-02-17T09:52:35Z")

</div>

can u tell me, why fader dsp’s value is at range -80 to 10 from your early response.  
and i found the volume decreasion is not linear. the volume disappear at about -40.  
so what is the decreasion curve of the fader dsp.  
i want to remap the value [-80,10] to the linear control volume, just like vca set volume. vca control volume is linear.

---

<div class="post-metadata">

**Author:** ![simingtu](https://avatars.discourse-cdn.com/v4/letter/s/9e8a1a/32.png) [@simingtu](https://qa.fmod.com/u/simingtu)\
**Post date:** [February 17, 2022, 1:02pm UTC](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274/21 "2022-02-17T13:02:02Z")

</div>

i have a group of events, how to get the root parent of the these events.  
u mean i just play one of it, and get the parent of this event?  
but how many times getParentGroup will be called?

var result = emiters[0].EventInstance.getChannelGroup(out FMOD.ChannelGroup g1);  
result = g1.getParentGroup(out FMOD.ChannelGroup parentGroup);  
result = parentGroup.getParentGroup(out FMOD.ChannelGroup parentGroup2);  
result = parentGroup2.getParentGroup(out FMOD.ChannelGroup parentGroup3);  
toneGroup = parentGroup3;  
result = toneGroup.addDSP(-1, m\_faderDsp);

in the above code , i find three times. is there a easier way to get the root parent of tone group.

[Next page](https://qa.fmod.com/t/fmod-lipsync-question-fmod-volume-affect-spectrum-extracting/18274.md?page=2)