Using resonance audio FMOD plugin without using FMOD Studio

FMOD Engine
1

Hi there,

we’re trying to use the resonance audio plugin for fmod via the fmod core api’s

I’m manually creating and attaching the Resonance Audio Listener DSP to an appropriate channel group. And manually attaching a Resonance Audio Source DSP to the channel of a sound that I want to play as spatial audio.

during update i’m updating both the fmod 3d position information for the channel, and sending FMOD_DSP_PARAMETER_3DATTRIBUTES to the Resonance Audio Source DSP’s data parameter 7.

in this particular example, the head position is not changing, but the sound position is orbiting around the listener. So i am not updating the spatial channel group position, nor sending 3D information to the Resonance Audio Listener DSP.

this all appears to work, however I get no audio through the spatial channel group with the Resonance Audio Listener DSP.

If i do not attach a Resonance Audio Source to the channel playing audio, but leave the Resonance Listener DSP on the channel group i do get audio from the spatial channel group, and it appears to be attenuated based on the position of the source. however setting the Resonance room values to something that should provide obvious reflections (eg: metal walls) cause no change to the output in this scenario, leading me to assume that this is not the “correct” configuration of these DSP’s.

is there any documentation available on how to hook this up natively? Anything obvious i’ve missed in the configuration of the DSP’s? I’ve been using the fmod.cc / fmod.h implementation from the resonance audio plugin to determine parameter values et al. Im working on the assumption that i’ll need to manually pass position information to the Resonance Source DSP’s in the same way that FMOD Studio’s update tick would, but so far no luck.

currently am testing this on android, but we’d eventually like to use this plugin on pc as well.

2

It sounds like there might be a parameter failing to be set. Are you checking the FMOD_RESULT of each of these calls to attach the dsps, set the parameters etc?
Unfortunately we don’t have any docs on how to setup Resonance Audio in code. If you share a code snippet I can try to reproduce the issue and figure out what might be going wrong?

3

as requested. im skipping the usual FMOD setup code as that works fine without the resonance plugin code.

FMOD_RESULT res = m_fmod_system->loadPlugin("libresonanceaudio.so",&m_resonanceHandle, 0);
ERRCHECK(res);

res = m_fmod_system->createDSPByPlugin(m_resonanceHandle, &m_resonanceDSP);
ERRCHECK(res);

int count;
res = m_fmod_system->getNumNestedPlugins(m_resonanceHandle, &count);
ERRCHECK(res);

unsigned int m_pluginHandles[3];
for(int i = 0; i < count; ++i)
{
    res = m_fmod_system->getNestedPlugin(m_resonanceHandle, i, &m_pluginHandles[i]);
    ERRCHECK(res);

that gets us the handles of all the nested plugins. we figure out which handle is which by doing

res = m_fmod_system->getPluginInfo(m_pluginHandles[i], &ptype, name, namelen, &ver);
ERRCHECK(res);

and string comparing against the names of the plugins, which are.

Resonance Audio Listener
Resonance Audio Source
Resonance Audio Soundfield

we then attach the Resonance Audio Listener DSP to the channel group we want to use for spatial audio. group here is the spatial channel group.

FMOD_RESULT res = group->addDSP(0, m_resAudioListenerDSP);
ERRCHECK( res );

res = m_resAudioListenerDSP->setActive(true);
ERRCHECK( res );

I also set the main plugin active here, because its not clear if the nested Resonance Audio Listener is the same as the main plugin.

res = m_resonanceDSP->setActive(true);
ERRCHECK( res );

in an attempt to give the plugin some data that would be immediately noticable ive set the room values to

m_room.dimensions[0] = 100.0f;
m_room.dimensions[1] = 100.0f;
m_room.dimensions[3] = 100.0f;
m_room.material_names[0] = vraudio::MaterialName::kMetal;
m_room.material_names[1] = vraudio::MaterialName::kMetal;
m_room.material_names[2] = vraudio::MaterialName::kMetal;
m_room.material_names[3] = vraudio::MaterialName::kMetal;
m_room.material_names[4] = vraudio::MaterialName::kMetal;
m_room.material_names[5] = vraudio::MaterialName::kMetal;
m_room.reverb_brightness = 1.5f;
m_room.reverb_time = 1.5f;
m_room.reverb_gain = 1.5f;

FMOD_RESULT res = m_resonanceDSP->setParameterData(1, &m_room, sizeof(vraudio::RoomProperties));
ERRCHECK( res );

when playing an audio source that we want to play via the spatial channel group. calling playSound then attaching the Resonance Audio Source plugin DSP to the channel associated to the sound.

FMOD_RESULT res = m_fmod_system->playSound(sound, channelGroup, false,&channel);
ERRCHECK(res);

FMOD_RESULT res = m_fmod_system->createDSPByPlugin(m_resAudioSourceHandle, &audioclip.m_resonanceAudioSourceDSP);
ERRCHECK(res);

int numDsps = 0;
res = channel->getNumDSPs(&numDsps);
ERRCHECK(res);

res = channel->addDSP(numDsps, audioclip.m_resonanceAudioSourceDSP);
ERRCHECK(res);

res = audioclip.m_resonanceAudioSourceDSP->setActive(true);
ERRCHECK(res);

finally each frame im updating the position of the Resonance Audio Source DSP plugin using

m_attrs.absolute.forward.x = 0.0f;
m_attrs.absolute.forward.y = 0.0f;
m_attrs.absolute.forward.z = 1.0f;
m_attrs.absolute.up.x = 0.0f;
m_attrs.absolute.up.y = 1.0f;
m_attrs.absolute.up.z = 0.0f;
m_attrs.absolute.velocity.x = 0.0f;
m_attrs.absolute.velocity.y = 0.0f;
m_attrs.absolute.velocity.z = 0.0f;
m_attrs.absolute.position.x = x;
m_attrs.absolute.position.y = y;
m_attrs.absolute.position.z = z;
m_attrs.relative.forward.x = 0.0f;
m_attrs.relative.forward.y = 0.0f;
m_attrs.relative.forward.z = 1.0f;
m_attrs.relative.up.x = 0.0f;
m_attrs.relative.up.y = 1.0f;
m_attrs.relative.up.z = 0.0f;
m_attrs.relative.velocity.x = 0.0f;
m_attrs.relative.velocity.y = 0.0f;
m_attrs.relative.velocity.z = 0.0f;
m_attrs.relative.position.x = x;
m_attrs.relative.position.y = y;
m_attrs.relative.position.z = z;

// magic number 7 from querying the input param list for the Audio Source DSP
FMOD_RESULT res = audioClip.m_resonanceAudioSourceDSP->setParameterData(7, &m_attrs, sizeof(FMOD_DSP_PARAMETER_3DATTRIBUTES));
ERRCHECK(res);

we’re just giving simple values here as the sound is simply orbiting the user as an example at this point.changing these values has no effect on the output of the audio, suspect its further upstream

so, when compiling this code out, everything works as expected with bare FMOD, adding this in gives no audio output. hopefully we’re just missing a parameter set somewhere.

4

Thank you for the additional information, and apologies for the delayed response. I can see the problem now.

The “main plugin” description is the same as the plugin description at index 0 of getNestedPlugin. While they are the same plugin description, in this case a Resonance Audio Listener, you have created two separate plugin instances; m_resAudioListenerDSP and m_resonanceDSP.

If you were to add both of these DSPs to your channel group then that could cause bizarre spatialization behavior, but it looks like you are only attaching m_resAudioListenerDSP so that should be fine.

On this line however:

You are calling setParameterData on m_resonanceDSP, which, of the two listener DSPs you have created, is the one that you haven’t attached to the channel group. Try calling it on m_resAudioListenerDSP instead:

FMOD_RESULT res = m_resAudioListenerDSP->setParameterData(1, &m_room, sizeof(vraudio::RoomProperties));

One other thing to note:

There is no index 3 in the dimensions array so this is undefined behavior. Should be:

m_room.dimensions[0] = 100.0f;
m_room.dimensions[1] = 100.0f;
m_room.dimensions[2] = 100.0f;

Other than that it should be fine. Please try calling setParameterData on m_resAudioListenerDSP and let me know if you are still running into issues here.

5

i had tried both m_resonanceDSP and m_resAudioListenerDSP to no avail as i’d completely missed the
m_room.dimensions[3] = 100.0f;

fixing that and now i have insanely reverby reverb. :slight_smile: thanks for the assist! interestingly setting a room size parameter to 0 seems to completely stop any reverb calculation. would be useful if it threw a FMOD_ERR_INVALID_PARAM in that case. ill raise a bug on their side.

thanks again jeff!

6

Great to hear it’s working now! Looks like negative dimension values also cancel out reverb. Some sanitization/errors/warnings would certainly be useful. No problem, let us know if you have any other issues!

7

So, trying to setup a “correct” example i’m noticing.

  • audio sources that are “outside” a room appear to create no reverb in the simulation when using room audio. which is fair enough.
  • with the audio source set to ignore room audio reverb, i get no output.
  • with room audio enabled for a source, it sounds like i’m getting 100% reverb mix with no original audio.
  • its not clear if the kTransparent room material means “transparent reflections” or “no reflections”. as im not getting audio out using that, i’m going with “non reflective material”.

i went poking at the additional source parameters. seeing if there was a way I could get something to sound relatively correct. however nothing seems to produce sensible values.

// ignore room audio
        res = audioclip.m_resonanceAudioSourceDSP->setParameterBool(8, true);
        ERRCHECK(res);

        // set spread to 360deg
        res = audioclip.m_resonanceAudioSourceDSP->setParameterFloat(1, 360.0f);
        ERRCHECK(res);

        // enable near field fx
        res = audioclip.m_resonanceAudioSourceDSP->setParameterBool(9, true);
        ERRCHECK(res);

        FMOD_DSP_PARAMETER_OVERALLGAIN gain;
        gain.linear_gain = 1.0f;
        gain.linear_gain_additive = 1.0f;

        // set overall gain.
        res = audioclip.m_resonanceAudioSourceDSP->setParameterData(11, &gain, sizeof(FMOD_DSP_PARAMETER_OVERALLGAIN));
        ERRCHECK(res);

it’s not clear in the code what the values are for linear_gain or linear_gain_additive. I did usin some logging code to get the default/max/min values for the other parameters though. Ive been assuming this whole time that the output from the resonance plugin would be the correct final “mix” output. Is that a correct assumption? or is the output from the resonance plugin akin to a reverb processor that still needs some of the original audio mixed with it?