Correct flow for handing mixerResume() failures

This question partially continues from my previous one at [iOS] change to setDSPBufferSize causing increased occurrence of 'Error initializing output device.' where I experienced failures to initialize FMOD at startup on iOS.
I previously fixed this by, as recommended, sleeping and retrying to initialize the FMOD system once more which seemed to resolve my problem.

I am however experiencing the same issue when calling mixerResume() after the application returning to the foreground.

My question is: What are the correct steps to take in the case of mixerResume() failing? and how does FMOD handle this.

I found reference in [iOS] Frequent SystemNotInitializedException saying that the initialization will throw an ‘exception’ once and switch to no sound mode afterwards. Does this still apply to current FMOD? If so, is it safe to accept the single error of Error initializing output device. and to continue to make calls to FMOD as if resuming was successful?

I am uncertain that it is safe to perform a sleep in resuming the app from background and therefore hope to opt for a ‘graceful failure’ in having FMOD continue to run but not throw errors for all calls. (no sound is acceptable) instead of the current crashing that we are experiencing on trying to resume the mixer.

Thanks

I would not expect to see mixerResume() fail due to an initialization error, the system has already been initialized and should be in a suspended state due to being sent to the background. mixerResume() will return an FMOD_RESULT that should provide more information on the ‘failure’.

Trying to call into FMOD while it is still in the background would result in a failure as it is not allowed to run in the background.

Sure, I presume the issue might be happening due to the app still being ‘backgrounded’ and an system timing issue with didResumeActive/ audiosession.

We can’t reproduce the issue locally so can’t really get the FMOD_RESULT back easily, hence this question was more on how to handle the failure in resuming (such as trying to call resume in the background).

We don’t really have a “correct flow” for handling a mixerResume failure, we recommend using system callbacks to call mixerSuspend/Resume so that it gets called at the right time.

https://fmod.com/resources/documentation-api?version=2.0&page=platforms-ios.html#handling-interruptions

I see, thanks. We were actually currently calling mixerResume() as a callback from applicationDidBecomeActive but have switched it to as described by the link (AVAudioSessionInterruptionNotification) hopefully this may resolve the issues.

We’ve also added logging to see the issues with resuming mixer in the future if it reoccurs.

By switching mixerResume from being run on applicationDidBecomeActive to, as documented, AVAudioSessionInterruptionNotification, we’ve seen a drop in crashes of around 80% which is great, however we are still seeing lots of crashes attempting to call mixerResume as documented by FMOD docs.
Is there anything further that we should watch out for? The exact code is:

[[NSNotificationCenter defaultCenter] addObserverForName:AVAudioSessionInterruptionNotification object:nil queue:nil usingBlock:^(NSNotification *notification)
 {
     if ([[notification.userInfo valueForKey:AVAudioSessionInterruptionTypeKey] intValue] == AVAudioSessionInterruptionTypeBegan)
     {
         FMOD_RESULT result = fmodSystem->mixerSuspend();
         ERRCHECK(result, "mixerSuspend");
     }
     else
     {
         [[AVAudioSession sharedInstance] setActive:TRUE error:nil];

         FMOD_RESULT result = fmodSystem->mixerResume();
         ERRCHECK(result, "mixerResume"); // Issue occurs here
     }
 }];

with the same FMOD error of Error initializing output device.
As previously stated, the application does not play background audio or have an audiosession set up to do so, this is simply attempting to handle application resumes correctly.

In our example code we also have this:

BOOL success = [[AVAudioSession sharedInstance] setActive:TRUE error:nil];
assert(success);

If that does not succeed then FMOD will not be able to resume.

I see, that makes sense then.
Is there a graceful way that we can ‘disable’ FMOD or cause it no no-op all functions run against it? (As I previously mentioned I read that there was some form of ‘silent mode’ it can be run in) does this apply here?
We can obviously add our own logic to avoid all future calls to FMOD but if there’s an easier or more graceful way to do this, it’d be great.

This AVAudioSession issue does happen fairly frequently (a few hundred cases a day) and crashing out of the application doesn’t seem like an acceptable behavior in production.

We do something like this in newer versions of the RuntimeManager, if the first attempt at initializing does not work we then try again using FMOD_OUTPUTTYPE_NOSOUND. This will at least allow the app to run without FMOD crashing.

Thanks, this sounds like a promising solution, however is there any way to change the output type without re-initializing the whole FMODSystem? I’ve attempted to call setOutput but it appears that this is invalid after init() has been called

(It doesn’t seem very feasible for app state to have to re-initialize the system and re-load all audio handles when the app resumes from an interruption in order to retry resumeMixer and set automatic/no_audio appropriately based on success/failure.)

Currently only Windows supports changing the output after initialization, all other platforms require it to be done before hand.

That does not seem like the best time to have to reinitialize FMOD.

So far we have been unable to reproduce these results here and are not sure why this would be occurring if the callbacks are triggering as they are supposed to.

Are you able to reproduce this behavior in a smaller/empty project?
Can you confirm the callbacks triggering at the correct time from the OS?
Is there anything common that stands out between devices that have these issues? (eg. iPhone model, OS version, etc.).

Taking a look at the crash metrics, there’s nothing that stands out in terms of devices.
Highest frequency is iOS12 and iPhone 8 but this matches our userbase metrics and it is not iOS12 exclusive (happens on 11 and 13 in very tiny quantities)

The only thing of interest seems to be that devices seem to almost always have <10% RAM available, but this may also be the nature of the game application/ iOS filling RAM where available.

I did manage to reproduce the issue locally by launching multiple heavy games (~4 games + our application) and quickly switching between the applications and starting/stopping background audio from spotify.

I can’t vouch for whether AVAudioSessionInterruptionNotification is being called at the right time since causing the issue requires fairly difficult to track circumstances (sometimes it appears it may be called late once the app has left foreground, but this may be debugger-related and is not a 100% crash)

I captured the following log entry which seems to be the only thing of interest:

2019-09-10 13:26:01.984304+0900 _________ [1488:255401] [aurioc] 1540: AUIOClient_StartIO failed (561015905)

I don’t have time this week to try and write a sample project but might be able to next week, it will likely involve just a template app and running it in the same memory/app switching pressure as tested above.

Sorry to bump this but are there any thoughts on this?
It doesn’t seem to be possible to switch to no_audio on a resume failure and this is definitely something that happens in iOS, so what would be the best course of action?

Without being able to reproduce the issue it is very difficult to say. The problem is the ‘resume failure’, if we can figure out why this is happening then we might be able to figure out a solution.

Figuring out which of the callbacks is or is not firing is a good place to start, because it may not always be obvious which one should be firing.

From the Apple docs on Responding to Interruptions:

Note: There is no guarantee that a begin interruption will have a corresponding end interruption. Your app needs to be aware of a switch to a foreground running state or the user pressing a Play button. In either case, determine whether your app should reactivate its audio session.

The callbacks that we log in the API are:

  • AVAudioSessionRouteChangeNotification
  • AVAudioSessionMediaServicesWereLostNotification
  • AVAudioSessionMediaServicesWereResetNotification
  • AVAudioSessionInterruptionNotification
  • UIApplicationDidBecomeActiveNotification
  • UIApplicationWillResignActiveNotification
  • UIApplicationDidEnterBackgroundNotification
  • UIApplicationWillEnterForegroundNotification