Joining several sounds into one single multi-channel audio for output

FMOD Engine
1

Hi everyone,

I’m working on a project in Unity in which several audios are being played from different sources, such as music/audio files and input audio from microphones. We want to be able to adjust these sounds outside of Unity in realtime (for example, make one louder without affecting the others), while trying to keep the latency low and the sounds synced (priority on the latency).

What I was thinking is to take the needed sounds, join them into a single multi-channel sound, and then play that on the external device. Any suggestions on how to achieve this? Or some other recommendations?

I’m not very familiar with FMOD. So far, I have been able to read a multi-channel wav file and play it on an output device using the core audio API. If I then try to play several sounds they get merged in the channels, which seems reasonable behavior but I would like to have them separated.

Looking through the forums I found this:
https://qa.fmod.com/t/how-can-i-make-an-wav-file-with-multichannel-ouputs/9238

To create a multichannel file you need to get the raw PCM data from the FMOD::Sound objects and interleave that into a single buffer then write that to file.

This seems close to what I want to do, but then the post goes into another direction and I’m still a bit lost on how to implement this, or if it’s really what I should be doing.

Any help is appreciated. :slight_smile:

2

After looking further into the documentation/forum, setMixMatrix seems like a good option for what I’m trying to do.

Right now I’m using the following code to set the channels to play the sounds on. Seems to be working fine, but if there’s a better way to do this, please let me know. :slight_smile:

private void PlaySoundOnChannels(FMOD.Sound fmodSound, int startingChannel)
{
    var result = fmodSystem.getMasterChannelGroup(out var masterGroup);

    result = fmodSystem.playSound(fmodSound, masterGroup, false, out var fmodChannel);

    result = fmodChannel.getDSP(FMOD.CHANNELCONTROL_DSP_INDEX.HEAD, out var dspHead);

    result = dspHead.getOutput(0, out _, out var channelDspHeadOutputConnection);

    result = fmodSound.getFormat(out _, out _, out var soundChannels, out _);


    result = fmodSystem.getDriver(out var currentDriver);

    result = fmodSystem.getDriverInfo(currentDriver, out _, 0, out _, out _, out _, out var speakerChannels);

    var matrix = new float[soundChannels * speakerChannels];
    for (int i = 0; i < soundChannels; i++)
    {
        int pos = soundChannels * (startingChannel + i) + i;
        if (pos >= matrix.Length)
        {
            break;
        }
        matrix[pos] = 1.0f;
    }
    result = channelDspHeadOutputConnection.setMixMatrix(matrix, speakerChannels, soundChannels);
}
3

It looks like what you are doing is what we would normally recommend for combining multiple sounds into a single multi-channel sound.

1 Like