Add DSP index, head behaves like tail?

Hi there!

My goal is to generate text-to-speech, play that via FMOD, and do lipsync based on the audio data. The first thing I do therefore is once the audio is generated, I play it via FMOD:

 private void PlayFMODSound(Sound sound)
        // this is to make sure the bus is loaded before you get the channel group
        // otherwise an error will be thrown and FMOD will enter an unstable state
        _fmodBus.getChannelGroup(out var channelGroup);

        FMOD.RESULT result = RuntimeManager.CoreSystem.playSound(sound, channelGroup, false, out var channel);
        if (result != FMOD.RESULT.OK)
            Debug.LogError("Error playing FMOD sound: " + result);
        _currentPlaybackChannel = channel;

_fmodBus is a FMOD.Studio.Bus that I set like this in the beginning:

_fmodBus = RuntimeManager.GetBus("bus:/Dialogue/Character");

As soon as the playback stops, I unlock the bus again. So far so good.
Next, I need to analyse the audio and feed that analysis to Salsa lipsync. For that, I created a custom DSP that I’m adding to the channel group I got when playing the sound above.

First, I’m creating a custom DSP (similar to how it is showcases in the example)

 private void InitializeCustomDSP()
        // Assign the callback to a member variable to avoid garbage collection
        _readCallback = CustomDspReadCallback;

        // Allocate a data buffer large enough for 8 channels, pin the memory to avoid garbage collection
        uint bufferLength;
        int numBuffers;
        FMODUnity.RuntimeManager.CoreSystem.getDSPBufferSize(out bufferLength, out numBuffers);
        _dataBuffer = new float[bufferLength * 8];
        _bufferLength = bufferLength;
        _objHandle = GCHandle.Alloc(this);
        if (_objHandle != null)
            dspDesc.numinputbuffers = 1;
            dspDesc.numoutputbuffers = 1;
   = CustomDspReadCallback;
            dspDesc.userdata = GCHandle.ToIntPtr(_objHandle);
            var result = RuntimeManager.CoreSystem.createDSP(ref dspDesc, out _customDsp);
            if (result != FMOD.RESULT.OK)
                Debug.LogError("Error creating custom DSP: " + result);

Once the audio playback starts and I have the channel group, I add the DSP:

_channelGroup.Value.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, _customDsp);

Now to the part that I don’t understand. My goal is to analyze the raw audio, before effects like a spatializer are applied to the signal (the lips should not move differently just because the character is further away). To my understanding, an index of “HEAD” should do exactly that. But it behaves the other way around: when i add the DSP to head, the values received by the custom DSP were already processed by all DSPs added via FMOD studio, for instance a delay effect that I added to the bus just for testing purposes. When I add the custom DSP to “TAIL” however, it does not get affected by that effect (just as I want it to). What’s interesting too is that the standard fader then does not affect the values received by the custom DSP unless the volume is set to minus infinity, in which case it is affected (lips don’t move). I suspect that the default fader is treated in some special way.

Anyways, I am wondering why “TAIL” behaves like I would except “HEAD” to function; From all documentation i’ve seen, “HEAD” should be the right index for what I want to achieve.

Hope someone can point me in the right direction!

// edit: Hmm, I just realized that I can’t add a spatializer effect to an entire channel group; that seems to be working only for individual events. But I can’t work with events when I have a procedural sound like this?

Signal flows from tail to head / right to left in our DSP Chain, with the head on the left, and the tail on the right.
This is certainly counterintuitive, since we present signal flow as left to right in the FMOD Studio deck, and top to bottom in the mixer. It makes more sense if you think about it in terms of a linked list, with the head “output” node having a growing list of tail “input” nodes appended to it over the lifetime of the application.
In any case, the “input” part of the signal chain is the tail, and the “output” is the head. You can read more about our signal flow in the DSP Architecture and Usage whitepaper for more information.
Please let me know if you have any further questions.

Thank you for clarifying this for me, Jeff!

I do have a follow-up question: I understand now that the signal flows from tail to head; but why does changing the fader to -infinity volume result in the read values of my custom DSP resembling muted audio as well? If I attach it to the tail, shouldn’t the fader that comes later make no difference?

That does sound strange- I have not been able to reproduce this.
Perhaps you can try connecting the Core API Profiler to confirm the DSP is where you think it is:

  1. Download the from the FMOD Engine package.
  2. Enable Live Update in your FMOD Unity Settings.
  3. Connect the Core API Profiler to your running application.
  4. Tick the “Volume” checkbox to display audio levels.

Here is an example with an FMOD Compressor added at the tail input, and a Channel muted with Channel.setVolume(0):

Thank you Jeff, I’ll give it a try and let you know!