Capture only a dynamic subset of channels' mixed audio in real-time (side-chain capture DSP)?

I have many sounds playing on channels under the master ChannelGroup. At runtime I want to capture only a specific subset of those channels’ mixed audio as a separate real-time PCM stream (to drive a secondary output), without affecting normal playback.

Constraint: I can’t pre-assign these sounds to a dedicated ChannelGroup when they’re created — I only learn which channels are “of interest” after they start playing, and the set changes constantly as sounds start and stop. They all just play flat under master.

Approach I’m considering — is this the correct pattern?

  1. Create a capture DSP and connect it as a (silent) input into the master’s head DSP via DSP::addInput, so it gets processed each mix but outputs silence and doesn’t alter the master output.
  2. For each “interesting” channel, captureDSP->addInput(channelHeadDSP) (the channel’s head DSP, via getDSP(FMOD_CHANNELCONTROL_DSP_HEAD)), so the channel feeds the capture DSP in addition to its normal path to master.
  3. Read the mixed result in the capture DSP’s read callback — and output silence there, so these sounds aren’t doubled into the master mix.

Questions:

  1. Is a side-chain capture DSP fed by per-channel DSP::addInput the right/standard way to capture a chosen subset of channels? Or is per-channel Channel::addDSP the intended approach? (per-channel addDSP crashed for me — multiple channels sharing one DSP seemed to corrupt the graph.)
  2. How do I safely manage these connections as channels stop / are released? I’m worried stale DSPConnections after a channel ends will crash. Should I DSP::disconnectFrom proactively when a channel stops, or does FMOD clean up the input connection automatically when that channel’s head DSP is freed?

Thanks!

Hi,

Thank you for the detailed post.

Question 1: From the docs (FMOD Engine | Core Api Dsp - Dsp::Addinput) “When a DSP has multiple inputs the signals are automatically mixed together, sent to the unit’s output(s).” To feed multiple inputs and a single DSP Addinput is the way to go.

Question 2: I would proactively remove the DSP inputs, before the channel stops you can use channelHeadDSP->disconnectFrom(captureDSP)as you will have reference to both.

Would it be possible to share your working test script?

Thanks Connor! Here’s a minimal repro of the pattern (FMOD Core). It plays a looping sound on new channels, taps each by adding its head DSP as an input to a capture DSP (connected silently under the master head), and retires the oldest tap.

My question is the cleanup marked in the code:

  1. Is capture->disconnectFrom(head, conn) before channel->stop() the correct/safe order?
  2. What’s the safe approach if FMOD may have already stopped or stolen the channel before I get there, so the head DSP could be invalid? How do I know when it’s safe to disconnect?

Also — as an alternative, would attaching a per-channel DSP via Channel::addDSP and cleaning up in that DSP’s release callback be a cleaner way to auto-clean exactly when the channel ends (instead of me detecting the stop myself)? Any downside vs the side-chain addInput approach?

cpp
// Minimal FMOD Core repro for:
//   "capture a dynamic subset of channels' mixed audio in real-time via a
//    side-chain capture DSP" + the connection-cleanup timing question.
//
// Build: link FMOD Core (fmod.hpp / fmod_vc.lib / fmod.dll).
// Put any short looping sound as "sound.wav" next to the exe.

#include <fmod.hpp>
#include <fmod_errors.h>
#include <cstdio>
#include <cmath>
#include <cstring>
#include <vector>
#include <windows.h>

#define CK(x) do{ FMOD_RESULT r=(x); if(r!=FMOD_OK) printf("FMOD %d %s @%d\n", r, FMOD_ErrorString(r), __LINE__);}while(0)

static float g_capturePeak = 0.0f;

// Capture DSP: its inputs are auto-mixed into 'in'; we read that (our subset-of-channels
// mix) and output silence so the tapped channels are NOT doubled into the master mix.
static FMOD_RESULT F_CALLBACK captureRead(FMOD_DSP_STATE*, float* in, float* out,
                                          unsigned int length, int inch, int* outch) {
    float peak = 0.0f;
    if (in) for (unsigned int i = 0; i < length * (unsigned)inch; ++i) { float a = fabsf(in[i]); if (a>peak) peak=a; }
    g_capturePeak = peak;                                   // <-- captured mix of tapped channels
    int oc = outch ? *outch : inch;
    if (out) for (unsigned int i = 0; i < length * (unsigned)oc; ++i) out[i] = 0.0f;   // silence out
    return FMOD_OK;
}

int main() {
    FMOD::System* sys = nullptr;
    CK(FMOD::System_Create(&sys));
    CK(sys->init(64, FMOD_INIT_NORMAL, nullptr));

    FMOD::Sound* snd = nullptr;
    CK(sys->createSound("sound.wav", FMOD_LOOP_NORMAL, nullptr, &snd));   // any short looping sound

    // --- capture DSP ---
    FMOD_DSP_DESCRIPTION d = {};
    strncpy(d.name, "capture", sizeof(d.name) - 1);
    d.numinputbuffers = 1; d.numoutputbuffers = 1;
    d.read = captureRead;
    FMOD::DSP* capture = nullptr;
    CK(sys->createDSP(&d, &capture));

    // Run the capture DSP by connecting it as a (silent) input of the master head DSP.
    FMOD::ChannelGroup* master = nullptr; CK(sys->getMasterChannelGroup(&master));
    FMOD::DSP* masterHead = nullptr;      CK(master->getDSP(FMOD_CHANNELCONTROL_DSP_HEAD, &masterHead));
    CK(masterHead->addInput(capture, nullptr, FMOD_DSPCONNECTION_TYPE_STANDARD));

    struct Tap { FMOD::Channel* ch; FMOD::DSP* head; FMOD_DSPCONNECTION* conn; };
    std::vector<Tap> taps;

    for (int step = 0; step < 30; ++step) {
        // Start a new channel and TAP it: add its head DSP as an input to the capture DSP.
        FMOD::Channel* ch = nullptr;
        CK(sys->playSound(snd, master, false, &ch));
        ch->setVolume(0.5f);
        FMOD::DSP* head = nullptr; CK(ch->getDSP(FMOD_CHANNELCONTROL_DSP_HEAD, &head));
        FMOD_DSPCONNECTION* conn = nullptr;
        CK(capture->addInput(head, &conn, FMOD_DSPCONNECTION_TYPE_STANDARD));
        taps.push_back({ ch, head, conn });

        // Exercise cleanup: retire the oldest tap.
        if (taps.size() > 4) {
            Tap t = taps.front(); taps.erase(taps.begin());
            // =============== CLEANUP QUESTION (please review) ===============
            // Is this order correct/safe? Should disconnectFrom happen BEFORE stop()?
            // What is the safe approach if FMOD may have already stopped/stolen the
            // channel before I get here (so 'head' could be invalid)?
            CK(capture->disconnectFrom(t.head, t.conn));
            t.ch->stop();
            // ===============================================================
        }
        CK(sys->update());
        printf("step %2d  taps=%zu  capturePeak=%.3f\n", step, (size_t)taps.size(), g_capturePeak);
        Sleep(200);
    }

    for (auto& t : taps) { capture->disconnectFrom(t.head, t.conn); t.ch->stop(); }
    masterHead->disconnectFrom(capture);
    capture->release();
    snd->release();
    sys->release();
    return 0;
}

Thank you for the code, I have added FMOD_INIT_PROFILE_METER_ALL (FMOD Engine | Core Api System - Fmod::Initflags) which allows me to connect with the FMOD Core Profiler

Which I can see the channel connected to the DSP correctly:

Yes, you should be disconnecting the DSP before stopping the channel when you can just to make sure everything is being cleaned up correctly.

Interacting with a stopped channel will return FMOD_ERR_INVALID_HANDLE(FMOD Engine | Core Api Common - Fmod::Result) so you should be fine if you do end up interacting with an invalid channel.

It will depend on your implementation, both are valid options. An alternative could be adding the channel callback FMOD_CHANNELCONTROL_CALLBACK_END (FMOD Engine | Core Api Channelcontrol - Fmod::Channelcontrol::Callback::Type) to each channel and the disconnect the DSP in the callback that way to could be done automatically and you wouldn’t have to worry about manually calling it.

Hi Connor, just wanted to circle back and say thanks — your answers on the disconnect timing, the INVALID_HANDLE safety net, and confirming both the side-chain and per-channel DSP approaches are valid really helped clarify the design. We went with the per-channel DSP + release-callback approach, and haven’t hit any bugs with it so far. Will follow up on this thread if something comes up during further testing. Appreciate the help!
:smiling_face_with_three_hearts: :smiling_face_with_three_hearts: :smiling_face_with_three_hearts:

Happy to help! If there is anything else we can assist with please do not hesitate to reach out!