Event callbacks don't work if other audio is playing on iOS

Unity
,
1

We have certain sequences in our mobile game that rely on an fmod event callback in order to proceed.

This works well, even when we have the audio muted, since I guess the audio is virtualized and event callbacks are still emitted.

The problem is specifically on iOS (haven’t tested Android yet), when there is audio already playing in the background from another app, it seems fmod studio doesn’t run at all, and there are no events emitted, which means certain parts of our game break.

I tried “Mute other audio sources” in Unity, to force other audio to stop when our game is launched, to ensure FMOD Studio runs, but it seems it doesn’t work when we used FMOD Studio.

Is there a workflow / setting here that I’m missing that avoids this issue? The best I can come up with is to somehow check if FMOD has initialized or if the event is playing (not sure what is appropriate), and if it doesn’t, just skip the part where it waits for the callback. Or just add a timer to the callback waiting so that it moves on after it has waited a really long time.

Neither are great workarounds since they will require more work, so I’m wondering if there’s a simpler solution I’m missing here.

We use Unity 6 and FMOD 2.02.23

2

Sorry for the delayed response!

The behavior you’re observing is most likely due to how FMOD pauses all events and suspends the mixer on receiving an interruption from iOS - it’s likely that it for some reason, it isn’t resuming properly when it should be.

Could I get you to set FMOD’s logging level to “Log” and enable API error logging in the initialization section of FMOD for Unity’s settings, build your app in as a development build, and share a log from XCode where you’ve reproduced the issue on your iOS device?

3

After further examination, it only happens with the iOS Brave browser’s video player, which automatically makes this less bad than I thought it was. I tried to recreate the issue with the apple music app or with a floating twitch window and fmod worked properly (I mean all audio was playing at the same time and it was a bit of a mess, but it’s fine)

Still, with audio playing from the Brave browser in the background (and potentially others? not sure if it’s exclusive to the brave browser, which makes me a bit uneasy), FMOD seems to not start at all. There is nothing in the log in XCode (I checked that logging actually works and it does).

See the capture from an iPhone for what happens and what is supposed to happen. In the first part I have brave playing some Chopin in the background, then it’s the game playing normally.

I didn’t do it in the video, but if I leave the game on iOS, go stop the background audio from brave, then go back to the game, the audio in the game starts and the track starts from the beginning as if it just started playing and the event does fire eventually.

The steps for repro is to have Brave playing audio in the background (either by a floating video player, or just the audio), and then run anything any app that uses FMOD.

I am willing to accept this is mostly a brave issue, but it would be good if we could at least detect it’s happening, so we could either have code to bypass waiting for callbacks, or at the very least, pop up a message to the user.

4

Thanks for the video demonstrating the issue, and for testing the repro for it so thoroughly yourself!

I’ve done a bit of a deep dive into it, and the issue is due to other applications using a larger buffer size for audio than FMOD. FMOD uses what is half the usual buffer on iOS to provide audio with lower latency (512 samples in FMOD for Unity settings → Platform Specific → DSP Buffer Settings → DSP Buffer Length). When FMOD audio is suspended in favor of other applications, the buffer than iOS provides the active audio session increases to match the other app’s audio, and usually when the Unity FMOD app resumes, the buffer is reduced to match what FMOD is outputting.

What appears to be happening with Brave is that, due to Brave playing backgrounded audio, the buffer iOS provides to be filled by FMOD stays at twice what FMOD is outputting. As a result, FMOD can’t resume properly, Studio processing isn’t performed, and callbacks aren’t fired.

The simplest approach to addressing this is to increase FMOD’s DSP Buffer Size from 512 to 1024, which will cause FMOD to resume correctly which Brave is playing backgrounded audio, and process callbacks even if it’s not outputting audio. Note that this will increase FMOD’s latency, which may or may not be an issue for you.

If latency would be an issue, you can force other applications’ audio to stop when your app comes to the foreground and hopefully avoid the issue with Brave playing backgrounded audio. This can be done by modifying ./Assets/Plugins/FMOD/platform_ios.mm and adding the following line at the start of RegisterSuspendCallback:

extern "C" void RegisterSuspendCallback(void (*callback)(bool))
{
    [[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategorySoloAmbient error:nil]; // New line here
    // ...
}

This should cause your app to stop all other applications from outputting audio at the same time. However, exactly what other apps try to do with audio can’t be guaranteed, so you may run into situations where this doesn’t work.

Give those solutions a try and let me know how they go.

5

Wow, that was thorough, thank you.

I can confirm that with a buffer of 1024 the issue goes away, but unfortunately the latency is significantly more than I would like.

So I’ll stick to the other method (which also works).

It is inevitable that some user will complain about not being able to play other music in the background while playing our game, but I think this is the best compromise.

Thank you.

1 Like
6

No problem, feel free to let us know if you run into any further issues!

7

Setting category to AVAudioSessionCategorySoloAmbient makes the game audio work correctly the first time I start the game with background audio from brave playing (which is already an improvement than before), but if I then leave, restart the audio from brave then go back my game, game audio fails to restart (even on subsequent attempts without other audio playing).

8

Thanks again for the video, super invaluable for understanding what’s going on.

On my end, what seems to fix this issue is to set the preferred buffer size and sample rate for the audio session alongside setting the category, which should ensure that when the FMOD Unity app is foregrounded that iOS gives it the buffer size and sample rate it wants.

You’ll need to make modifications to the following files to do this:

./Assets/Plugins/FMOD/platform_ios.mm

Remove the previous change I suggested where AVAudioSessionCategorySoloAmbient is set, and add the following new function:

extern "C" void SetAudioSessionSettings(double rate, int blockSize)
{
    NSLog(@"Sample rate = %lf", rate);
    NSLog(@"Block size = %d", blockSize);
    
    AVAudioSession *session = [AVAudioSession sharedInstance];
    
    NSLog(@"Setting AVAudioSessionCategory to SoloAmbient");
    NSError *audioSessionError = nil;
    [session setCategory:AVAudioSessionCategorySoloAmbient error:&audioSessionError];
    if (audioSessionError)
    {
        NSLog(@"Error setting AVAudioSessionCategory: %@", audioSessionError);
    }
    
    double bufferDuration = blockSize / rate;
    NSLog(@"Setting PreferredIOBufferDuration to %lf", bufferDuration);
    [session setPreferredIOBufferDuration:bufferDuration error:&audioSessionError];
    if (audioSessionError){
        
        NSLog(@"Error setting PreferredIOBufferDuration: %@", audioSessionError);
    }
    
    NSLog(@"Setting PreferredSampleRate to %lf", rate);
    [session setPreferredSampleRate:rate error:&audioSessionError];
    if (audioSessionError)
    {
        NSLog(@"Error setting PreferredSampleRate: %@", audioSessionError);
    }
}

./Assets/Plugins/FMOD/src/RuntimeManager.cs

Use DllImport to export the new function at line 1462:

        #if (UNITY_IOS || UNITY_TVOS || UNITY_VISIONOS) && !UNITY_EDITOR
        // new code start
        [DllImport("__Internal")]
        private static extern void SetAudioSessionSettings(double rate, int blockSize);
        // new code end
        [DllImport("__Internal")]
        private static extern void RegisterSuspendCallback(Action<bool> func);
        #endif

And then call the new function after the FMOD System has been initialized, at line 408:

            #if (UNITY_IOS || UNITY_TVOS || UNITY_VISIONOS) && !UNITY_EDITOR
            // new code start
            int systemSampleRate;
            coreSystem.getSoftwareFormat(out systemSampleRate, out _, out _);
            uint systemDSPBufferLength;
            coreSystem.getDSPBufferSize(out systemDSPBufferLength, out _);
            SetAudioSessionSettings((double) systemSampleRate, (int)systemDSPBufferLength);
            // new code end
            RegisterSuspendCallback(HandleInterrupt);
            #endif

There does appear to be an edge case where backgrounded audio being played by Brave starts playing while the FMOD Unity application is active and in the foreground - this is less an issue with FMOD, and more an issue with how Brave is handling playing in the background. In this case, FMOD audio will be forced to suspend even while the app is open, but should resume if the app is suspended and resumed again.

Give those changes a try and let me know whether that resolves the issue!