Lipsync when instance volume was set to 0 on the VCA

aleveque · October 21, 2025, 5:54pm

Hello,

I tried to use SALSA to do lipsync.
For this, I rewired things and it usually work pretty good. Usually.

..Because when the user wants to set the volume to 0, then the system doesn’t get data (or a flat line).

The event is wired to a VCA, whose volume has been set to 0
The instance is retrieved by the lipsync system which creates a DSP to its channel group.

Here’s the code:

private void OnVoicePlaying(EventInstance eventInstance)
{
	if (m_playingInstances == null)
		return;

	if (!ShouldHaveSalsaUponInit)
		this.m_salsa = this.GetSalsa();

	if (!this.m_salsa)
		return;

	/*if (stopRoutine != null)
		StopCoroutine(stopRoutine);
	stopRoutine = null;*/

	// Define a basic DSP that receives a callback each mix to capture audio
	FMOD.DSP_DESCRIPTION desc = new FMOD.DSP_DESCRIPTION();
	desc.numinputbuffers = 1;
	desc.numoutputbuffers = 1;
	desc.read = m_readCallback;
	desc.userdata = GCHandle.ToIntPtr(m_objHandle);

	// Create an instance of the capture DSP and attach it to the master channel group to capture all audio
	FMOD.ChannelGroup eventCG;
	FMOD.DSP captureDSP;

	FMOD.RESULT result;

	result = eventInstance.getChannelGroup(out eventCG);
	if (VerboseFmodResults)
		Debug.Log("getChannelGroup => " + result);

	result = FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out captureDSP);
	if (VerboseFmodResults)
		Debug.Log("createDSP => " + result);

	result = captureDSP.setChannelFormat(FMOD.CHANNELMASK.MONO, 1, FMOD.SPEAKERMODE.MONO);
	if (VerboseFmodResults)
		Debug.Log("setChannelFormat => " + result);

	result = eventCG.addDSP(0, captureDSP);
	if (VerboseFmodResults)
		Debug.Log("addDSP => " + result);

	m_isPlaying = true;

	m_salsa.getExternalAnalysis = null;
	m_salsa.getTriggerIndex = null;
	m_salsa.InitializeDelegates(); // ensures those two delegates to be at their init values.

	m_salsa_getTriggerIndex = m_salsa.getTriggerIndex;  // gets the previous delegate in our own ref.
	m_salsa.getTriggerIndex = GetTriggerIndexForSalsa;	// ..So that it would be called within our delegate override

	m_salsa.getExternalAnalysis = GetAnalysisForSalsa;
	
	m_playingInstances.Add(eventInstance, captureDSP);
}

How can I wire the DSP so it can get the data before the VCA fades the volume to 0?

Is there a tool that we could use to visualise the ChannelGroup ?
On this page: https://www.fmod.com/docs/2.02/api/using-dsp-effects-in-the-core-api.html
It feels like a tree

aleveque · October 21, 2025, 6:55pm

I just created this:


#if UNITY_EDITOR
private void Editor_VisualiseChannelGroup(StringBuilder str, int tabs, FMOD.ChannelGroup eventCG)
{
	FMOD.RESULT retval;

	string tabsStr = new string('\t', tabs);

	string name;
	retval = eventCG.getName(out name, 255);
	if (retval != FMOD.RESULT.OK)
	{
		str.Append(tabsStr).AppendLine("<color=red>ERROR getName</color>");
	}

	str.Append(tabsStr).Append(name).Append(" (").Append(eventCG.handle).AppendLine(")");

	int numDSP;
	retval = eventCG.getNumDSPs(out numDSP);
	if(retval != FMOD.RESULT.OK)
	{
		str.Append(tabsStr).Append("\t<color=red>ERROR getNumDSPs</color>");
	}
	else
	{
		FMOD.DSP dsp;
		FMOD.DSP subDsp;
		FMOD.DSPConnection dspCo;
		uint version;
		int channels, w, h;
		int numIn, numOut;

		for (int i = 0; i < numDSP; i++)
		{
			retval = eventCG.getDSP(i, out dsp);
			if (retval != FMOD.RESULT.OK)
			{
				str.Append(tabsStr).Append("\t<color=red>ERROR getDSP(").Append(i).AppendLine("</color>");
			}
			else
			{
				retval = dsp.getInfo(out name, out version, out channels, out w, out h);
				if (retval != FMOD.RESULT.OK)
				{
					str.Append(tabsStr).Append("\t<color=red>ERROR DSP::getInfo(").Append(i).AppendLine("</color>");
				}
				else
				{
					str.Append(tabsStr).Append("\t- DSP: ").Append(name).Append(" (").Append(dsp.handle).AppendLine(")");

					retval = dsp.getNumInputs(out numIn);
					if (retval != FMOD.RESULT.OK)
					{
						str.Append(tabsStr).Append("\t\t<color=red>ERROR DSP::getNumInputs(").Append(i).AppendLine("</color>");
					}
					else
					{
						for (int d = 0; d < numIn; d++)
						{
							retval = dsp.getInput(d, out subDsp, out dspCo);
							if (retval != FMOD.RESULT.OK)
							{
								str.Append(tabsStr).Append("\t\t<color=red>ERROR DSP::getInput(").Append(i).AppendLine("</color>");
							}
							else
							{
								retval = subDsp.getInfo(out name, out version, out channels, out w, out h);
								if(retval == FMOD.RESULT.OK)
									str.Append(tabsStr).Append("\t\t- in: ").Append(name).Append(" (").Append(subDsp.handle).AppendLine(")");
								else
									str.Append(tabsStr).Append("\t\t- in: ").Append(subDsp.handle).AppendLine();
							}
						}
					}

					retval = dsp.getNumOutputs(out numOut);
					if (retval != FMOD.RESULT.OK)
					{
						str.Append(tabsStr).Append("\t\t<color=red>ERROR DSP::getNumOutputs(").Append(i).AppendLine("</color>");
					}
					else
					{
						for (int d = 0; d < numOut; d++)
						{
							retval = dsp.getOutput(d, out subDsp, out dspCo);
							if (retval != FMOD.RESULT.OK)
							{
								str.Append(tabsStr).Append("\t\t<color=red>ERROR DSP::getOutput(").Append(i).AppendLine("</color>");
							}
							else
							{
								retval = subDsp.getInfo(out name, out version, out channels, out w, out h);
								if (retval == FMOD.RESULT.OK)
									str.Append(tabsStr).Append("\t\t- out: ").Append(name).Append(" (").Append(subDsp.handle).AppendLine(")");
								else
									str.Append(tabsStr).Append("\t\t- out: ").Append(subDsp.handle).AppendLine();
							}
						}
					}
				}
			}
		}
	}


	int numCG;
	retval = eventCG.getNumGroups(out numCG);
	if (retval != FMOD.RESULT.OK)
	{
		str.Append(tabsStr).Append('\t').AppendLine("<color=red>ERROR getNumGroups</color>");
	}
	else
	{
		FMOD.ChannelGroup subCG;
		for (int i = 0; i < numCG; i++)
		{
			retval = eventCG.getGroup(i, out subCG);
			if (retval != FMOD.RESULT.OK)
			{
				str.Append(tabsStr).Append('\t').Append("<color=red>ERROR getGroup(").Append(i).AppendLine("</color>");
			}
			else
			{
				Editor_VisualiseChannelGroup(str, tabs + 1, subCG);
			}
		}
	}
}
#endif

It showed me the DSP chain.

I added desc.name = Encoding.ASCII.GetBytes("lipsync_capture");

The thing is my DSP was at the end of it, so I changed the code at the “addDSP” :

int numDSP;

result = eventCG.getNumDSPs(out numDSP);
if (VerboseFmodResults)
Debug.Log("getNumDSPs => " + result);

result = eventCG.addDSP(numDSP-1, captureDSP);
if (VerboseFmodResults)
Debug.Log("addDSP => " + result);

Now the DSP seems better placed, but it still seem to be influenced by the VCA.

How can I prevent that?

aleveque · October 21, 2025, 7:13pm

If this can help, here is what showed the method:

Visualizing channel group:
Master Bus (1589916096008)
    - DSP: FMOD Limiter (1585957525760)
        - in: FMOD Send (1586331779360)
        - out: ChanGroup Fader (1586343061776)
    - DSP: FMOD Send (1586331779360)
        - in: FMOD 3-EQ (1581531999824)
        - out: FMOD Limiter (1585957525760)
    - DSP: FMOD 3-EQ (1581531999824)
        - in: FMOD Pan (1590477095776)
        - out: FMOD Send (1586331779360)
    - DSP: FMOD Pan (1590477095776)
        - in: FMOD Fader (1586343052944)
        - out: FMOD 3-EQ (1581531999824)
    - DSP: FMOD Fader (1586343052944)
        - in: FMOD Pan (1590477099472)
        - out: FMOD Pan (1590477095776)
    - DSP: FMOD Pan (1590477099472)
        - in: lipsync_capture (1573133052288)
        - out: FMOD Fader (1586343052944)
    - DSP: lipsync_capture (1573133052288)
        - in: ChanGroup Fader (1586343076496)
        - out: FMOD Pan (1590477099472)
    - DSP: ChanGroup Fader (1586343076496)
        - in: ChanGroup Fader (1586343066192)
        - in: ChanGroup Fader (1586343072816)
        - in: ChanGroup Fader (1586343057360)
        - in: ChanGroup Fader (1586343083856)
        - in: ChanGroup Fader (1586343069136)
        - in: ChanGroup Fader (1586343069872)
        - in: ChanGroup Fader (1586343079440)
        - out: lipsync_capture (1573133052288)
    Group Bus (1589916115464)
        - DSP: ChanGroup Fader (1586343066192)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916092360)
        - DSP: ChanGroup Fader (1586343072816)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916092968)
        - DSP: ChanGroup Fader (1586343057360)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916093576)
        - DSP: ChanGroup Fader (1586343083856)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916108168)
        - DSP: ChanGroup Fader (1586343069136)
            - in: Channel Fader (1586297345952)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916094184)
        - DSP: ChanGroup Fader (1586343069872)
            - out: ChanGroup Fader (1586343076496)
    Group Bus (1589916098440)
        - DSP: ChanGroup Fader (1586343079440)
            - out: ChanGroup Fader (1586343076496)

aleveque · October 21, 2025, 7:47pm

I made a method to visualize from the banks buses.

private void Editor_VisualiseBuses(StringBuilder str)
{
	FMOD.RESULT retval;

	Bank[] banks;
	retval = RuntimeManager.StudioSystem.getBankList(out banks);
	if(retval != FMOD.RESULT.OK)
	{
		str.AppendLine("<color=red>error for getBankList</color>");
		return;
	}

	string path;
	FMOD.Studio.Bus[] buses;

	string busPath;
	FMOD.ChannelGroup busCG;

	foreach (Bank b in banks)
	{
		retval = b.getPath(out path);
		if (retval != FMOD.RESULT.OK)
		{
			str.AppendLine("<color=red>error for getPath</color>");
			continue;
		}

		str.Append("Loaded bank: ").AppendLine(path);

		retval = b.getBusList(out buses);
		if (retval != FMOD.RESULT.OK)
		{
			str.AppendLine("<color=red>error for getBusList</color>");
			continue;
		}

		foreach (Bus bus in buses)
		{
			retval = bus.getPath(out busPath);
			if (retval != FMOD.RESULT.OK)
			{
				str.AppendLine("\t<color=red>error for Bus::getPath</color>");
				continue;
			}

			str.Append("\t- Found bus: ").AppendLine(busPath);

			retval = bus.getChannelGroup(out busCG);
			if (retval != FMOD.RESULT.OK)
			{
				str.AppendLine("\t\t<color=red>error for Bus::getChannelGroup</color>");
				continue;
			}

			Editor_VisualiseChannelGroup(str, 2, busCG);
		}
	}

}

Found out that the volume has been sent to 0 at the “Master bus” parent’s parent

I have like:

The bus “bus:/” → CG “Master Bus” → CG “Group Bus” whose volume is 0 (I guess the VCA integrates here) → CG “Input Bus” → CG “Master Bus” (Master of my event) → the DSP “lipsync_capture”

aleveque · October 21, 2025, 7:56pm

Clamping the volume to a minimum of 0.001 seems to be working. My guess is that volume 0 means muted and there might be some optimization in the process.

I’m still curious about which idea you might have to have this nicely done ^^

Sylafrs.

Topic		Replies	Views
Fmod lipsync question fmod volume affect spectrum extracting Unity	20	959	February 25, 2022
How to get waveform data for lipsync when audio is muted? Unity	5	466	July 28, 2023
Confusion about System addDsp FMOD Engine	31	7155	October 8, 2014
Adding a DSP Unity	3	3996	December 19, 2015
Preventing copyrighted music from being captured by GameDVR FMOD Studio	15	2918	August 31, 2017

Lipsync when instance volume was set to 0 on the VCA

Related topics