intercom and soundbank

Hi,

I’d like to build a system that uses the studio system to output sounds in a 7.1 system.
Next to that I’d like to be able to use some of the remaining inputs and outputs of the same soundcard to build a small intercom system.
I succeeded in building a soundbank and triggering some events in C++.

However I’m getting quite confused of all possibilities in FMOD.
Is it possible to add an intercom system using the lowlevel API ? One of the things I’ve been looking at, is the recording example which I can use to retrieve sound from an input.

What I don’t understand yet is how to send something to a choosen output.
One of the things I came across was the FMOD_RESULT System::attachChannelGroupToPort( FMOD_PORT_TYPE portType,
FMOD_PORT_INDEX portIndex, FMOD::ChannelGroup *channelgroup, bool passThru);

But I wasn’t able to find out what to do with the PORT_TYPE.
Maybe there’s a better approach to set up the Intercom system ?

Kind regards,

Tom

Apologies for the lean docs on ports, I’ve updated the docs already to compensate for this a bit, and i’ll be instructing to get a tutorial section on it in the docs as well.

Basically ports are not for you, they are a hardware specific implementation through hardware specific ‘extra’ headers like fmod_wiiu.h or fmod_psvita.h or fmod_orbis.h to allow the user to attach a channelgroup to a hardware device like a console headset, a controller speaker, or a console music channel.
In theory this could be expanded to address your extra outputs, but you didn’t actually say what those extra outputs were? Is it not a standard 7.1 soundcard? What are the other outputs that you are talking about? Typically if you want to go out of the standard 7.1 setup, you have to use RAW speaker mode and pan everything yourself, for example if you had a 20 channel soundcard, which doesn’t map to 7.1 or any other type of speaker mode.

Hi Brett,

It’s a Focusrite 18i20, with 18 inputs and 20 outputs, reachable through using ASIO.

I would like to build banks in FMOD studio for the first 8 outputs to use them as a 7.1 system. Then I’d also like to use 5 additional inputs and outputs to connect headsets and use them as an intercom.

In the meanwhile I’ve been experimenting a bit.
I’ve made a test bank in FMOD Studio and made a program that loads the bank and is able to play the events in it.
I noticed that, as soon as I switch the soundcard to RAW output, using setSoftwareFormat in the lowlevel part of the system, the loading of the banks is giving me errors.

This makes me wonder if it is possible to use fmod in studiomode, which allows me to use banks in this RAW setup ?

If it is possible. Could you give me directions on how to do that. Can I use the output of the studio system and it’s banks as inputs of a MixMatrix somehow ?

Kind regards,

Tom

You’re supposed to switch the low level speaker mode to that of the project you’re loading. If you change it, it wont know how to upmix/downmix which is a big part of the bus architecture.

If you want to support ASIO and raw output, then just use the low level API, and you can use Channel/ChannelGroup::setMixMatrix to place incoming signals into raw speakers out.

Hi Brett,

So if I also want an intercom on the same soundcard I have to switch to RAW speakermode, which means I’m not able to use the soundbanks ?

Kind regards,

Tom

Banks will not work in raw speaker mode, so no. One convoluted way which would work - would be to

  1. run the soundbank system in FMOD_OUTPUTTYPE_NOSOUND_NRT mode,
  2. add a capture DSP to it (createDSP, addDSP)
  3. copy the 7.1 PCM captured from that system, to a RAW speaker mode system object, with another custom DSP that just copies data into the right speakers from the 7.1 mix from the nosound object.

Then you could use the low level commands to do some intercom system on top of that, using setMixMatrix to address different systems.

As far as the copying stuff is concerned I might have to prototype that to see how feasible it is, if you’d like to try it.

Thanks for the workaround Brett !

Kind regards,

Tom

Hi Brett,

I’ve been trying your solution using the custom dsp example.
I don’t understand everything in the example yet.
I made the following code:

#include "fmod.hpp"
#include "common.h"

int const bufferlength = 1024;
int read{ bufferlength - 1 }, write{ 0 };
float transferbuffer[bufferlength];
bool bypass;

FMOD_RESULT F_CALLBACK myDSPCallback(FMOD_DSP_STATE *dsp_state, float *inbuffer, float *outbuffer, unsigned int length, int inchannels, int *outchannels) 
{
    FMOD_RESULT result;
    char name[256];
    void* userdata;
    FMOD::DSP *raw_outdsp = (FMOD::DSP *)dsp_state->instance; 

    result = raw_outdsp->getInfo(name, 0, 0, 0, 0);
    ERRCHECK(result);

    result = raw_outdsp->getUserData(&userdata);
    ERRCHECK(result);

    for (unsigned int samp = 0; samp < length; samp++) 
    { 		
        {
			if (bypass == false ) transferbuffer[write] = inbuffer[samp ];
			else transferbuffer[write] = 0.0;
			if (write < bufferlength) write++;
			else write =0;
			//Common_Draw(" %f", transferbuffer);
        }
    } 
    return FMOD_OK; 
} 

FMOD_RESULT F_CALLBACK inDSPCallback(FMOD_DSP_STATE *dsp_state, float *inbuffer, float *outbuffer, unsigned int length, int inchannels, int *outchannels)
{
	FMOD_RESULT result;
	char name[256];
	void* userdata;
	FMOD::DSP *raw_indsp = (FMOD::DSP *)dsp_state->instance;

	result = raw_indsp->getInfo(name, 0, 0, 0, 0);
	ERRCHECK(result);

	result = raw_indsp->getUserData(&userdata);
	ERRCHECK(result);
	for (unsigned int samp = 0; samp < length; samp++)
	{
			outbuffer[samp] = transferbuffer[read];
			if ((read < bufferlength)) read++;
			else read = 0;
	}

	return FMOD_OK;
}


int FMOD_Main()
{
    FMOD::System       *system;
	FMOD::System       *systemSoundCard;
    FMOD::Sound        *sound;
    FMOD::Channel      *channel;
    FMOD::DSP          *outdsp;
	FMOD::DSP          *indsp;
    FMOD::ChannelGroup *mastergroup;
	FMOD::ChannelGroup *mastergroupSoundCard;
    FMOD_RESULT         result;
    unsigned int        version;
    void               *extradriverdata = 0;

    Common_Init(&extradriverdata);

    result = FMOD::System_Create(&system);
    ERRCHECK(result);

	


	int recordingSources;
	system->getRecordNumDrivers(0,&recordingSources);

	
	result = FMOD::System_Create(&systemSoundCard);
	ERRCHECK(result);

    result = system->getVersion(&version);
    ERRCHECK(result);

    if (version < FMOD_VERSION)
    {
        Common_Fatal("FMOD lib version %08x doesn't match header version %08x", version, FMOD_VERSION);
    }

	result = system->setOutput(FMOD_OUTPUTTYPE_NOSOUND_NRT);
	ERRCHECK(result);

	int samplerate;
	result = system->getSoftwareFormat(& samplerate,0,0);
	ERRCHECK(result);

	result = systemSoundCard->setSoftwareFormat(samplerate, FMOD_SPEAKERMODE_MONO, 0);
	ERRCHECK(result);

    result = system->init(32, FMOD_INIT_NORMAL, extradriverdata);
    ERRCHECK(result);

	result = systemSoundCard->init(32, FMOD_INIT_NORMAL, extradriverdata);
	ERRCHECK(result);

	
    result = system->createSound(Common_MediaPath("drumloop.wav"), FMOD_LOOP_NORMAL, 0, &sound);
    ERRCHECK(result);

    result = system->playSound(sound, 0, false, &channel);
    ERRCHECK(result);

    /*
        Create the Raw output DSP effect.
    */  
    { 
        FMOD_DSP_DESCRIPTION dspdesc; 
        memset(&dspdesc, 0, sizeof(dspdesc));
        
        strncpy(dspdesc.name, "My first DSP unit", sizeof(dspdesc.name));
        dspdesc.version = 0x00010000;
        dspdesc.numinputbuffers = 1;
        dspdesc.numoutputbuffers = 1;
        dspdesc.read = myDSPCallback; 
        dspdesc.userdata = (void *)0x12345678; 

        result = system->createDSP(&dspdesc, &outdsp); 
        ERRCHECK(result);

    } 

	/*
	Create the Raw input DSP effect.
	*/
	{
		FMOD_DSP_DESCRIPTION dspdescin;
		memset(&dspdescin, 0, sizeof(dspdescin));

		strncpy(dspdescin.name, "Output DSP", sizeof(dspdescin.name));
		dspdescin.version = 0x00010000;
		dspdescin.numinputbuffers = 1;
		dspdescin.numoutputbuffers = 1;
		dspdescin.read = inDSPCallback;
		dspdescin.userdata = (void *)0x12345678;

		result = systemSoundCard->createDSP(&dspdescin, &indsp);
		ERRCHECK(result);
	}


	

    /*
        Attach the DSP, inactive by default.
    */
    result = outdsp->setBypass(true);
    ERRCHECK(result);

	result = outdsp->setBypass(false);
	ERRCHECK(result);

    result = system->getMasterChannelGroup(&mastergroup);
    ERRCHECK(result);

    result = mastergroup->addDSP(0, outdsp);
    ERRCHECK(result);

	result = systemSoundCard->getMasterChannelGroup(&mastergroupSoundCard);
	ERRCHECK(result);

	result = mastergroupSoundCard->addDSP(0, indsp);
	ERRCHECK(result);

    /*
        Main loop.
    */
    do
    {
       // bool bypass;

        Common_Update();
		systemSoundCard->update();

        if (Common_BtnPress(BTN_ACTION1))
        {
            bypass = !bypass;
        }

        result = system->update();
        ERRCHECK(result);

        Common_Draw("==================================================");
        Common_Draw("Custom DSP Example.");
        Common_Draw("Copyright (c) Firelight Technologies 2004-2015.");
        Common_Draw("==================================================");
        Common_Draw("");
        Common_Draw("Press %s to toggle filter bypass", Common_BtnStr(BTN_ACTION1));
        Common_Draw("Press %s to quit", Common_BtnStr(BTN_QUIT));
        Common_Draw("");
        Common_Draw("Filter is %s", bypass ? "inactive" : "active");
		Common_Draw("Number of recording sources is %i", recordingSources);


        Common_Sleep(50);
    } while (!Common_BtnPress(BTN_QUIT));

    /*
        Shut down
    */
    result = sound->release();
    ERRCHECK(result);

    result = mastergroup->removeDSP(outdsp);
    ERRCHECK(result);
    result = outdsp->release();
    ERRCHECK(result);

    result = system->close();
    ERRCHECK(result);
    result = system->release();
    ERRCHECK(result);

    Common_Close();

    return 0;
}

There is some slowed down distorted sound.
Is it a problem that the two systems may be out of sync, causing the intermediate buffer to skip or repeat samples, caused by different speeds in reading and writing ?

What am I doing wrong ? Or can you redirect me to other questions and or examples to get a better understanding of developing DSP’s and solving this problem ?

Best regards,

Tom

Here is a modified FMOD Studio example ‘3d’ that includes feeding from one system to another.
Just overwrite the studio example code with this.

  • Note that the audible system’s update function is not called. This is to avoid a deadlock with the waitsema that is in the audible dsp’s read callback. This is just a simple way to synchronize the 2, if update is needed (ie you want to run 3d code on the audible system or any of the other things that are listed in the docs for what System::update is used for) then a more sophisticated synchronization method is required. (ie a queue).
  • Both systems are set to 5.1, so it just does a memcpy between the shared buffer. If the audible system was higher (like 20ch
    asio) you have to copy each channels buffer out one by one.
  • Note 1024 is not a hard coded requirement. You can change the buffer size with setDSPBufferSize.
  • If the main loop takes longer than the time between mix updates (ie 1024 samples = 21ms) then the mixer will starve and stutter. This is why the loop sleep is down to 5ms instead of 50.

.

/*==============================================================================
Event 3D Example
Copyright (c), Firelight Technologies Pty, Ltd 2012-2016.

This example demonstrates how to position events in 3D for spatialization.
==============================================================================*/
#include "fmod_studio.hpp"
#include "fmod.hpp"
#include "common.h"

const int SCREEN_WIDTH = NUM_COLUMNS;
const int SCREEN_HEIGHT = 16;

int currentScreenPosition = -1;
char screenBuffer[(SCREEN_WIDTH + 1) * SCREEN_HEIGHT + 1] = {0};

void initializeScreenBuffer();
void updateScreenPosition(const FMOD_VECTOR& worldPosition);

HANDLE sema;
bool needsdata = true;

float sharedbuffer[6][1024];

FMOD_RESULT F_CALLBACK dspread_capture(FMOD_DSP_STATE *dsp_state, float *inbuffer, float *outbuffer, unsigned int length, int inchannels, int *outchannels)
{
    memcpy(outbuffer, inbuffer, sizeof(float) * length * inchannels);
    
    memcpy(sharedbuffer, inbuffer, sizeof(float) * length * inchannels);

    ReleaseSemaphore(sema, 1, NULL);

    return FMOD_OK;
}

FMOD_RESULT F_CALLBACK dspread_feedaudible(FMOD_DSP_STATE *dsp_state, float *inbuffer, float *outbuffer, unsigned int length, int inchannels, int *outchannels)
{
    needsdata = true;

    WaitForSingleObject(sema, INFINITE);

    memcpy(outbuffer, sharedbuffer, sizeof(float) * length * inchannels);

    needsdata = false;

    return FMOD_OK;
}

int FMOD_Main()
{
    void *extraDriverData = NULL;
    Common_Init(&extraDriverData);
    
    FMOD::System *audibleLowLevelSystem = 0;
    FMOD::Studio::System* system = NULL;
    ERRCHECK( FMOD::Studio::System::create(&system) );

    // The example Studio project is authored for 5.1 sound, so set up the system output mode to match
    FMOD::System* lowLevelSystem = NULL;
    ERRCHECK( system->getLowLevelSystem(&lowLevelSystem) );
    
    ERRCHECK( lowLevelSystem->setOutput(FMOD_OUTPUTTYPE_NOSOUND_NRT) );
    ERRCHECK( lowLevelSystem->setSoftwareFormat(0, FMOD_SPEAKERMODE_5POINT1, 0) );

    ERRCHECK( system->initialize(32, FMOD_STUDIO_INIT_NORMAL, FMOD_INIT_PROFILE_METER_ALL, extraDriverData) );

    if (1)
    {
        FMOD::ChannelGroup *mcg;
        FMOD::DSP *capturedsp;
        FMOD::DSP *feedaudibledsp;
        FMOD_RESULT result;
        FMOD_DSP_DESCRIPTION description = { 0 };


        /*
            Set up capture DSP
        */
        result = lowLevelSystem->getMasterChannelGroup(&mcg);
        ERRCHECK(result);
        
        description.read = dspread_capture;
        strcpy(description.name, "NRT Capture");

        result = lowLevelSystem->createDSP(&description, &capturedsp);
        ERRCHECK(result);

        result = mcg->addDSP(0, capturedsp);
        ERRCHECK(result);

        /*
            Set up audible system + DSP
        */
        result = FMOD::System_Create(&audibleLowLevelSystem);
        ERRCHECK(result);

        ERRCHECK( audibleLowLevelSystem->setOutput(FMOD_OUTPUTTYPE_WASAPI) );                   /* !!!!!! This could be ASIO / 20 channels etc. */
        ERRCHECK( audibleLowLevelSystem->setSoftwareFormat(0, FMOD_SPEAKERMODE_5POINT1, 0) );   /* !!!!!! This could be ASIO / 20 channels etc. */

        result = audibleLowLevelSystem->init(1024, FMOD_INIT_NORMAL/*FMOD_INIT_PROFILE_METER_ALL*/, 0);
        ERRCHECK(result);

        result = audibleLowLevelSystem->getMasterChannelGroup(&mcg);
        ERRCHECK(result);
        
        description.read = dspread_feedaudible;
        strcpy(description.name, "Feeder");

        result = audibleLowLevelSystem->createDSP(&description, &feedaudibledsp);
        ERRCHECK(result);

        result = feedaudibledsp->setChannelFormat(0, 6, FMOD_SPEAKERMODE_5POINT1);
        ERRCHECK(result);

        result = mcg->addDSP(0, feedaudibledsp);
        ERRCHECK(result);

        /*
            Set up synchronization
        */
        sema = CreateSemaphore(NULL, 0, 0xffff, NULL);
        needsdata = false;
    }


    FMOD::Studio::Bank* masterBank = NULL;
    ERRCHECK( system->loadBankFile(Common_MediaPath("Master Bank.bank"), FMOD_STUDIO_LOAD_BANK_NORMAL, &masterBank) );

    FMOD::Studio::Bank* stringsBank = NULL;
    ERRCHECK( system->loadBankFile(Common_MediaPath("Master Bank.strings.bank"), FMOD_STUDIO_LOAD_BANK_NORMAL, &stringsBank) );

    FMOD::Studio::Bank* vehiclesBank = NULL;
    ERRCHECK( system->loadBankFile(Common_MediaPath("Vehicles.bank"), FMOD_STUDIO_LOAD_BANK_NORMAL, &vehiclesBank) );
    
    FMOD::Studio::EventDescription* eventDescription = NULL;
    ERRCHECK( system->getEvent("event:/Vehicles/Basic Engine", &eventDescription) );

    FMOD::Studio::EventInstance* eventInstance = NULL;
    ERRCHECK( eventDescription->createInstance(&eventInstance) );

    FMOD::Studio::ParameterInstance* rpm = NULL;
    ERRCHECK( eventInstance->getParameter("RPM", &rpm) );

    ERRCHECK( rpm->setValue(650) );

    ERRCHECK( eventInstance->start() );

    // Position the listener at the origin
    FMOD_3D_ATTRIBUTES attributes = { { 0 } };
    attributes.forward.z = 1.0f;
    attributes.up.y = 1.0f;
    ERRCHECK( system->setListenerAttributes(0, &attributes) );

    // Position the event 2 units in front of the listener
    attributes.position.z = 2.0f;
    ERRCHECK( eventInstance->set3DAttributes(&attributes) );
    
    initializeScreenBuffer();

    do
    {
        Common_Update();
        
        if (Common_BtnPress(BTN_LEFT))
        {
            attributes.position.x -= 1.0f;
            ERRCHECK( eventInstance->set3DAttributes(&attributes) );
        }
        
        if (Common_BtnPress(BTN_RIGHT))
        {
            attributes.position.x += 1.0f;
            ERRCHECK( eventInstance->set3DAttributes(&attributes) );
        }
        
        if (Common_BtnPress(BTN_UP))
        {
            attributes.position.z += 1.0f;
            ERRCHECK( eventInstance->set3DAttributes(&attributes) );
        }
        
        if (Common_BtnPress(BTN_DOWN))
        {
            attributes.position.z -= 1.0f;
            ERRCHECK( eventInstance->set3DAttributes(&attributes) );
        }

        if (audibleLowLevelSystem)
        {
           // ERRCHECK( audibleLowLevelSystem->update() );      /* ! Dont call update, dont need it.  Calling update can hang mixer because there is a waitsema in it. */
        }

        if (needsdata)
        {
            ERRCHECK( system->update() );
        }
        
        updateScreenPosition(attributes.position);
        Common_Draw("==================================================");
        Common_Draw("Event 3D Example.");
        Common_Draw("Copyright (c) Firelight Technologies 2016-2016.");
        Common_Draw("==================================================");
        Common_Draw(screenBuffer);
        Common_Draw("Use the arrow keys (%s, %s, %s, %s) to control the event position", 
            Common_BtnStr(BTN_LEFT), Common_BtnStr(BTN_RIGHT), Common_BtnStr(BTN_UP), Common_BtnStr(BTN_DOWN));
        Common_Draw("Press %s to quit", Common_BtnStr(BTN_QUIT));

        Common_Sleep(5);
    } while (!Common_BtnPress(BTN_QUIT));

    ERRCHECK( system->release() );

    Common_Close();

    CloseHandle(sema);

    return 0;
}

void initializeScreenBuffer()
{
    memset(screenBuffer, ' ', sizeof(screenBuffer));

    int idx = SCREEN_WIDTH;
    for (int i = 0; i < SCREEN_HEIGHT; ++i)
    {
        screenBuffer[idx] = '\n';
        idx += SCREEN_WIDTH + 1;
    }

    screenBuffer[(SCREEN_WIDTH + 1) * SCREEN_HEIGHT] = '\0';
}

int getCharacterIndex(const FMOD_VECTOR& position)
{
    int row = static_cast<int>(-position.z + (SCREEN_HEIGHT / 2));
    int col = static_cast<int>(position.x + (SCREEN_WIDTH / 2));
    
    if (0 < row && row < SCREEN_HEIGHT && 0 < col && col < SCREEN_WIDTH)
    {
        return (row * (SCREEN_WIDTH + 1)) + col;
    }
    
    return -1;
}

void updateScreenPosition(const FMOD_VECTOR& eventPosition)
{
    if (currentScreenPosition != -1)
    {
        screenBuffer[currentScreenPosition] = ' ';
        currentScreenPosition = -1;
    }

    FMOD_VECTOR origin = {0};
    int idx = getCharacterIndex(origin);
    screenBuffer[idx] = '^';
    
    idx = getCharacterIndex(eventPosition);    
    if (idx != -1)
    {
        screenBuffer[idx] = 'o';
        currentScreenPosition = idx;
    }
}
1 Like