Streaming Procedural Audio

Unity
1

I’m working on an integration for Dissonance Voice Chat which plays out audio through FMOD instead of the normal Unity audio system. I’m having some trouble working out the best way to play streaming audio.

My initial approach has been to create a DSP with a read callback which pumps the Dissonance audio system for however much audio it needs (see example code here). This is roughly how Dissonance noramlly works (normally it uses the OnAudioFilterRead Unity callback).

However, when the FMOD editor is attached with Live Update to the Unity editor sound played in this way does not appear anywhere in the mixer.

I tried playing the DSP out to a specific bus:

RuntimeManager.StudioSystem.getBus("bus:/Test", out var bus);
bus.getChannelGroup(out channel);
// pass `channel` in to playDSP method

but this doesn’t help.

I’ve also experimented with creating a Sound with a PCMReadCallback but haven’t got that working yet (constantly crashing the editor).

What’s the correct way to do this?

2

Bump. Would also love to know how to do this.

3

After grabbing your source pcm data, you can pass it to FMOD either in a PCMReadCallback or by copying it into a Sound and playing that back as a programmer sound.
To copy into a sound you can use Sound.lock to get a pointer to the sound’s buffer. Then you copy your pcm data into the buffer and unlock the sound with Sound.unlock. There has been some discussion about this topic recently in another thread which is worth reading, and has examples of both the Sound.lock approach. They ended up having issues with OnAudioFliterRead and this is due to an old Unity bug that they have no plan of solving, but that shouldn’t be an issue in your case if you aren’t using OnAudioFilterRead.

4

If I create and play a Sound object like this:

RuntimeManager.CoreSystem.playSound(_sound, default(ChannelGroup), false, out _channel);

Will that show up in the FMOD studio mixer somewhere? If not, should I direct it to a specific ChannelGroup for that, or must I use a programmer instrument to play the sound instead?

5

you need to get the channel group out of the mixer group and pass that to playSound using something like this

    FMOD.Studio.Bus bus = FMODUnity.RuntimeManager.GetBus("bus:/master/music");
    bus.lockChannelGroup(); // make sure to unlock it once you're done using it
     
    // this is to make sure the bus is loaded before you get the channel group
    // otherwise an error will be thrown and FMOD will enter an unstable state
    FMODUnity.RuntimeManager.StudioSystem.flushCommands();
    bus.getChannelGroup(out channelGroup);

I did it for mixing music and it works well

2 Likes
6

That worked! I would never have figured out locking/flushing without your hint. Thanks so much :hearts:

I have one final hurdle though.

Since this is a VoIP system, audio must be played back at all times (to keep in sync and prevent buffers filling up etc). However at the moment if the bus is muted in the Studio mixer pcmreadcallback stops being called (and it’s the same if I use a Dsp for playback as well).

Is it possible to tell FMOD to never stop pumping a bus for audio, even when it’s muted?

7

We don’t have a way to explicitly tell FMOD to continue running the pcm read callback when muted. Rather than muting, you can try using a parameter to turn the volume down to 99.99% (-79db), which would be inaudible, but prevent the voice from going virtual and losing sync, and also allow the pcm read callback to continue running.

8

Unfortunately that doesn’t really solve the problem - I was trying to avoid VoIP desyncing when the mixer is muted in FMOD studio (e.g. another Bus is soloed to work on a specific audio effect). It’s not a huge issue though, for now I can just tell people working in FMOD to expect desyncs (i.e. it won’t be a problem for players).

9

I see, we are planning on fixing the virtual voice sync issue in a future release, unfortunately there’s currently no workaround if you want to mute from the mixer.

10

Unfortunately I think I’ve just bumped into the same problem from a different angle - when the voice is muted due to distance (e.g. rolloff = Linear, Distance > maxdistance) it seems that the bus is suspended and the same desync issue happens.

For now I can work around this (cut off VoIP before ever getting to the max distance), but do you have any idea when that fix you mentioned might be available?

11

I agree that sounds like the same problem again- there is no hard date on this fix, but it requires breaking changes so it is scheduled for a major release at this stage.

1 Like