Duplicate/mirror event (in Unity)?

I am struggling with designing a proper “rooms and portals” system with FMOD and Unity, where sounds inside a room would be heard through open doors, like they would in real life. At the same time I want to hear the sound from its original position through the thing walls.
Is there a way to “duplicate/mirror” an FMOD event in Unity so I can have it playing both from its original position and a filtered version through the door? I know I can use the transceiver effect, but that doesn’t really scale for a more general system because you have to set it up for all events and you are limited to 16 channels.

Hope there is a good solution for this

4 Likes

We don’t really have a good solution for this yet, but we are aware of this limitation and are still determining what a proper solution would be.

I think Transceivers are probably still the best way to go in the meantime, despite their low channel count.

The 16 channel limitation is inconvenient, but you can design around it- since you can transmit on the same channel with multiple transceivers, you can mix multiple transmitting receivers together with only a single channel, allowing you to have multiple sounds in a room and a single opening, or multiple openings using a single channel. With this approach, you only need 1 channel per audible room, and you can re-use these channels once other rooms become inaudible.

For example, here the listener can hear the audio from Room 1 and Room 2 through the transceivers receiving on channels one and two respectively:

Now if the listener moves into range of the portals in Room 3, it moves out of the range of Room 1’s portals, allowing us to re-use Channel 1 for Room 3’s audio:

With this approach, you would only ever need 16 channels if the user was within audible range of 16 distinct rooms.

As for the additional design time overhead of configuring all these transmitting transceivers, this can be mitigated by instead adding and removing transceiver DSPs to events at runtime. What you would need is something that:

  1. Checks whether there is line of sight to the listener.
  2. Checks whether there is line of sight to events on the other side of the portal.
  3. If there is line of sight to the listener and event, add transceivers to those events and play a receiving transceiver event at the position of the portal.
  4. Attenuate each transmitting transceiver based on its distance to the portal for added realism.
  5. Remove transceivers if line of sight to either the events or listener becomes broken.

Here is a basic script demonstrating how to to achieve this:

/**
    This script demonstrates how to add transceivers to an event at runtime
    to simulate openings within occluding geometry.
    1. Create a looping event inside FMOD Studio containing a receiving
        transceiver and position this event in the opening of your geometry.
    2. Assign two GameObjects to the InnerSide and OuterSide fields.
       These will represent the inner and outer side of your geometry's opening,
       and will be used to determine if the listener or event emitters are in the
       field of view of the opening.
    3. Assign any event emitters in your room to the Source list.
*/
using System.Collections.Generic;
using UnityEngine;

public class DynamicTransceiverExample : MonoBehaviour
{
    public GameObject InnerSide;   // An object sitting inside the room
    public GameObject OuterSide;   // An object sitting outside the room
    public int OcclusionLayer = 8; // The layer you are using for occlusion

    public List<FMODUnity.StudioEventEmitter> Sources = new List<FMODUnity.StudioEventEmitter>();
    private Dictionary<FMOD.GUID, FMOD.DSP> transceivers = new Dictionary<FMOD.GUID, FMOD.DSP>();

    // Checks whether a Vector 3 is in the filed of view of the inner or outer side of the occluding geometry
    private bool PositionInFOV(Vector3 position)
    {
        Color color = Color.red;
        int occlusionMask = 1 << OcclusionLayer;
        Vector3 sourcePosition = new Vector3(position.x, position.y, position.z);

        // Test one side
        float distanceToObject = Vector3.Distance(sourcePosition, InnerSide.transform.position);
        bool inFOV = !Physics.Raycast(sourcePosition, InnerSide.transform.position - sourcePosition, out RaycastHit hit, distanceToObject, occlusionMask);

        // Test other side
        distanceToObject = Vector3.Distance(sourcePosition, OuterSide.transform.position);
        inFOV |= !Physics.Raycast(sourcePosition, OuterSide.transform.position - sourcePosition, out hit, distanceToObject, occlusionMask);

        if (inFOV)
        {
            color = Color.green;
        }

        Debug.DrawRay(sourcePosition, transform.position - sourcePosition, color);

        return inFOV;
    }

    // Add a transceiver to an Event Emitter's event
    private void AddTransceiver(FMODUnity.StudioEventEmitter source)
    {
        FMOD.RESULT result = source.EventInstance.getChannelGroup(out FMOD.ChannelGroup group);
        if (result != FMOD.RESULT.OK) return;

        result = FMODUnity.RuntimeManager.CoreSystem.createDSPByType(FMOD.DSP_TYPE.TRANSCEIVER, out FMOD.DSP transceiver);
        result = transceiver.setParameterBool((int)FMOD.DSP_TRANSCEIVER.TRANSMIT, true);
        result = group.addDSP(FMOD.CHANNELCONTROL_DSP_INDEX.TAIL, transceiver);
        transceivers[source.EventReference.Guid] = transceiver;
        Debug.Log("Transceiver added");
    }

    // Remove a transceiver from an Event Emitter's event
    private void RemoveTransceiver(FMODUnity.StudioEventEmitter source)
    {
        source.EventInstance.getChannelGroup(out FMOD.ChannelGroup group);
        FMOD.RESULT result = group.removeDSP(transceivers[source.EventReference.Guid]);
        result = transceivers[source.EventReference.Guid].release();
        transceivers.Remove(source.EventReference.Guid);
        Debug.Log("Transceiver removed");
    }

    // Cleanup dynamically created transceivers
    private void OnDestroy()
    {
        Sources.ForEach((source) => { RemoveTransceiver(source); });
    }

    void Update()
    {
        FMOD.RESULT result = FMODUnity.RuntimeManager.StudioSystem.getListenerAttributes(0, out FMOD.ATTRIBUTES_3D attributes);
        bool canSeeListener = PositionInFOV(new Vector3(attributes.position.x, attributes.position.y, attributes.position.z));

        Sources.ForEach((source) =>
        {
            bool canSeeEmitter = PositionInFOV(source.transform.position);

            float distanceToObject = Vector3.Distance(source.transform.position, transform.position);
            result = source.EventInstance.getMinMaxDistance(out float minDistance, out float maxDistance);

            if (canSeeListener && canSeeEmitter && distanceToObject <= maxDistance)
            {
                Debug.Log("Listener and Emitter in fov of the portal");

                if (!transceivers.ContainsKey(source.EventReference.Guid))
                {
                    AddTransceiver(source);
                }

                if (transceivers.ContainsKey(source.EventReference.Guid))
                {
                    // Attenuate transceiver proportionally to distance from event emitter
                    result = transceivers[source.EventReference.Guid].setParameterFloat((int)FMOD.DSP_TRANSCEIVER.GAIN, 20 * Mathf.Log10(1.0f - (distanceToObject / maxDistance)));
                }
            }
            else
            {
                Debug.Log("Listener and Emitter not in fov of the portal");
                if (transceivers.ContainsKey(source.EventReference.Guid))
                {
                    RemoveTransceiver(source);
                }
            }
        });
    }
}

Hopefully that makes sense, please let me know if you have any questions.

Thank you a lot for the suggestion! Dynamically adding and removing transceivers seems like a great addition to the workaround I had in mind.
Currently trying to understand everything happening in your script. Am I right this is how you would do that?

I guess you would also need to have some sort of manager handling the transceiver channels dynamically, to avoid using the same channels for audible portals. Do you have a good idea for how to approach this?

I guess an issue with this system is when an emitter is really close to the portal if will be artificially loud, since you would both get the sound from the portal and the emitter itself. So if a NPC e.g. walks of of a room, they would seem louder around the portal than after leaving the room. Do you have any ideas for reducing this?

I’m not sure if it can be apply to you case. But I’m working on audio volume system similar to Unreal. It’s not a ready-to-use solution but I can explain how I approach it if you are interested in details. Main idea is if volumes overlap then I can dynamicaly change level of sound sources from one in another. In this example there is a big “rain” volume and inside there is a room with a door. When you open it, part of a rain sound coming through depending on door rotation.

Sorry if it is not approach you are looking for.

1 Like

Yay, managed to make a prototype based on this that somewhat works. However, I am having problems when there are two instances of the same FMOD event in the room.
Does it actually work using the FMOD GUID for checking if sounds already has a transceiver? What if there are multiple instances of the same event? Wouldn’t it then only add a transceiver to the first instance? Could we use source.gameObject.GetInstanceID instead to get a unique ID?

{
		if (!transceivers.ContainsKey(source.EventReference.Guid))
		{
				AddTransceiver(source);
		}

		if (transceivers.ContainsKey(source.EventReference.Guid))
		{
			// Attenuate transceiver proportionally to distance from event emitter
			result = transceivers[source.EventReference.Guid].setParameterFloat((int)FMOD.DSP_TRANSCEIVER.GAIN, 20 * Mathf.Log10(1.0f - (distanceToObject / maxDistance)));
		}
}

Glad to hear you got it working.

For dynamic channel managing, perhaps something like:

  1. Maintain a static pool of available channels.
  2. When the listener is in range an EventInstance, give it a transceiver, transmitting on that room’s channel.
    • If the room does not yet have a channel, first remove one from the pool.
  3. When the listener moves outside the range of an EventInstance, ask the room how many EventInstances it has reporting an audibility greater than 0.
    • If the answer is 0, return the room’s channel to the pool.

That is true, plus you will get phasing from the slightly delayed signal on the transceiver. Perhaps rather than having the receiving transceiver event play at the position of your opening, you could play it in the center of your room, tweak the max distance to be outside the opening, and stop the event when you enter the room? That should reduce the loudness- otherwise I don’t really have a good answer for that one sorry :confused:

Yes that seems to be working fine for me, good idea!

That is relevant, and also a very understandable volume algorithm- thanks for sharing!
So with this approach, you have a box collider around your door, and if you are inside the collider with the door open you mix in events that are outside the volume?

Thank you for the reply Jeff. I have a decent prototype working now, yaaay.

Good idea with the pooling, I will try that out!

Perhaps rather than having the receiving transceiver event play at the position of your opening, you could play it in the center of your room, tweak the max distance to be outside the opening, and stop the event when you enter the room?

Wouldn’t that defeat the purpose of the portal system, because the sound would no longer play from the position of the portal if you did this?

One more question. I actually don’t think I understand why you have the inner and outer portal thing and use that for calculating the FOV. Could you explain that a bit more in depth? Thank you!

Thanks, the box collider is used for the opening logic and transfer rotation float. Mixed signals effect is achived by overlaping this very big box collider around whole scene and this smaller “room”. When you are outside this room you are hearing rain sound with it default loudness, inside - it depends on float receiving from door. I updated git repository GitHub - doudar41/UnityAudioVolume so you can take a look at code. There is an example of door collider script used in video and prefab of it, “SoundChanger” script which sends float to “AVCrossfades” and it makes changes in audio volumes. I think it’s possible to use other colliders to mix audio volume signals like portals you’ve mentioned, I’d think about it, maybe this is the way to make system more flexable.

1 Like

I made something like portals, so if audio volumes do not overlap, it’s possible to hear signal from them, imitating occlusion system. Basically one collider launch sound sources linked to other collider with low pass filter and loudness adjustments.

Yes it would, good point.

Sure, so the way I have occlusion setup is by assigning a GameObject to a layer for occluding geometry, and controlling an occlusion parameter on an event based on whether or not a raycast from the listener hits this occluding geometry (probably the same as what you have).
To create a “portal” through this geometry, I decided I could just add two other game objects on each side of the wall, and raycast from them to the listener and event respectively, and use that to determine whether the listener could indirectly hear the event. Here is a screenshot to demonstrate better, with the red square representing the inner point, and the green square indicating the outer point:

You could remove the need for this by rearranging the geometry to create an actual opening but I thought this would be more generally applicable. Hopefully that makes sense.

Awesome, so you are zoning in portals with colliders and using them to control the mix of events in overlapping audio volume. That does produce a realistic result- thanks for the explanation!

1 Like

Hi @jeff_fmod ,
We have been built a pretty succesfull custom portal system around the transceivers. However, we are having issues with clicks every time we add a transceiver to an event with code (as in your example). I imagine this is because the transceiver level goes from -inf to its level in not time. Do you have an idea about how we can avoid this?
We tried creating a fade-in to the transceiver level when it is added with code, but the fade-in became way too slow before it worked, so that wasn’t really a good solution.

I imagine it would work if the parameter could have a little bit of seek speed, but I don’t think that is possible when we are adding the transceivers dynamically with code?

Thank you!

Hmm, I see what you mean, it’s very difficult to get a smooth fade in by setting the transceiver’s gain since it’s locked to the Studio System’s update rate.

You could potentially create a temporary DSP to handle the fade for you. Here is an example in C++ to demonstrate:

 struct RAMP_DATA
 {
    float target = 1.0f;
    float alpha = 0.00001f;
    float gainVal = 0.0f;
};

FMOD_DSP_DESCRIPTION fadeInDesc = { 0 };
fadeInDesc.numinputbuffers = 1;
fadeInDesc.numoutputbuffers = 1;
fadeInDesc.create = [](FMOD_DSP_STATE *dsp_state)
{
    RAMP_DATA *data = new RAMP_DATA;
    dsp_state->plugindata = (void *)data;
    return FMOD_OK;
};
fadeInDesc.release = [](FMOD_DSP_STATE *dsp_state)
{
    RAMP_DATA *data = (RAMP_DATA*)dsp_state->plugindata;
    delete data;
    return FMOD_OK;
};
fadeInDesc.process = [](FMOD_DSP_STATE* dsp_state, unsigned int length, const FMOD_DSP_BUFFER_ARRAY* inbufferarray, FMOD_DSP_BUFFER_ARRAY* outbufferarray, FMOD_BOOL inputsidle, FMOD_DSP_PROCESS_OPERATION op)
{
    RAMP_DATA *data = (RAMP_DATA*)dsp_state->plugindata;

    if (op == FMOD_DSP_PROCESS_QUERY)
    {
        if (data->target - data->gainVal < 0.1f)
        {
            // Fading done, skip processing
            return FMOD_ERR_DSP_DONTPROCESS;
        }
        outbufferarray->bufferchannelmask = inbufferarray->bufferchannelmask;
        outbufferarray->buffernumchannels = inbufferarray->buffernumchannels;
        outbufferarray->speakermode = inbufferarray->speakermode;
        return FMOD_OK;
    }
    float *in = inbufferarray->buffers[0];
    float *out = outbufferarray->buffers[0];
    while (length--)
    {
        // Fade in the signal with a gain modifier
        *out++ = *in++ * data->gainVal;

        // Exponential fade to target gain of 1.0f
        data->gainVal = data->alpha * data->target + (1 - data->alpha) * data->gainVal;
    }

    return FMOD_OK;
};

channel1->addDSP(FMOD_CHANNELCONTROL_DSP_FADER, transceiver);
system->createDSP(&fadeInDesc, &fadeInDSP);

For Unity, you could pretty much copy the DSP Capture scripting example and ramp in the buffer instead of just copying it across.

I will let the dev team know about this though and see if we can get rid of this clicking- thank you for bringing this to our attention!