Hi warriors,
iOS background audio going nicely here under Unity, but I need a trigger to start the next audio track when the current track completes. Normally, there is setCallback() + FMOD_CHANNELCONTROL_CALLBACK_END, but apparently under iOS background mode, that callback doesn’t get fired until the app returns to the foreground.
What options are there to detect the end of a sound (stream) under these conditions?
Thanks!
Hi,
As per our docs on Lock Screen & Background Audio on iOS, if you choose an AudioSession category that supports background audio, and enable background audio in your info property list, FMOD should work with iOS background mode as it normally does - have you done both of these things?
Also, I recall from one of your previous posts that you were only using the Core API, so just to clarify, are you using the FMOD for Unity integration, or your own C# wrapper?
Hello!
Indeed, iOS setup seems to be in order – background audio is otherwise working.
On start of a track, we hand a new http stream to FMOD in Unity via:
createStream(url, FMOD.MODE.CREATESTREAM | FMOD.MODE.NONBLOCKING, ...)
Each frame getOpenState() is called. On the first OPENSTATE.READY for a track, system.playSound() is called and channel.setCallback() is bound.
When the channel callback sees CHANNELCONTROL_CALLBACK_TYPE.END it prints to log and do an iOS callback that queues UnityBatchPlayerLoop() to trigger a Unity Update() pass.
When not in background mode, things work as expected (once Update() sees a OPENSTATE.READY during playback, it initiates starting a new track).
In background mode, the CHANNELCONTROL_CALLBACK_TYPE callback does not show up until I switch back to foreground (confirmed by running under Xcode and not seeing that log message when the stream ends).
Perhaps the callback is only pushed from inside something like system.update() so therefore is not made until the Unity update loop resumes?
Thanks for the detailed repro! I’ve done some testing on my end, and I’m able to reproduce what you describe - i.e. CHANNELCONTROL_CALLBACK not firing when in background mode - when the AVAudioSession is configured incorrectly. If background audio capability is enabled in XCode (info.plist or target platform capabilities), and I set the app’s AudioSession to use the option: AVAudioSessionCategoryOptionMixWithOthers, then CHANNELCONTROL_CALLBACK_TYPE.END is fired as expected even when backgrounded. Please give that a shot and see whether it resolves your issue.
Hi, this is great news. Will get back as soon as I can confirm, thank you Louis!!
Hi Louis,
Strangely, adding ...MixWithOthers (with category AVAudioSessionCategoryPlayback) the background event still not not fire here. Unfortunately, even if it did, ...MixWithOthers removes the lock screen play control center (play/pause, prev/next, etc) which would be an issue since we are building a music player.
I have confirmed UIBackgroundModes contains audio (and also added fetch just in case that matters). If I am not mistaken, the app wouldn’t even be playing in the background if this was not set correctly.
This has been quite frustrating and time consuming your assistance is much appreciated! Any other thoughts? Hmm, what else could have caused the change you saw over there?
Drew
Sorry for the delayed response!
In my previous testing, I did notice that while the CHANNELCONTROL_CALLBACK_TYPE.END callback fires, audio that would normally loop while the application is focused doesn’t resume while in background mode, which is likely due to how FMOD is tied to Unity’s update loop with our integration. I should be able to give it a further test with the extra information you’ve provided sometime next week, but in the meantime, by the “background event” not firing do you mean that the CHANNELCONTROL_CALLBACK_TYPE.END callback isn’t fired? Or that the callback fires, but the dependent behaviour (i.e. UnityBatchPlayerLoop()) doesn’t occur?
Great question Louis and feeling good you’re looking at this.
Let’s say I use the above steps to consume a http streamed mp3 or aac that is exactly 10 minutes long. Typically a mobile device is playing in background audio mode, where basically iOS is like a cop itching to shut you down if you are idle or or consume too much.
At the 9:59.9 point, the stream is consumed and getOpenState() would normally return OPENSTATE.READY if the Unity Update() loop was active. But we install the CHANNELCONTROL_CALLBACK_TYPE.END callback, and that is used to kick the Unity machinery into action for ~2 secs. But the callback doesn’t appear until the app is moved back into the foreground app (which could be any number of seconds or minutes later). This StayAwake code is already working well for the managing the buttons on the iOS lock screen (so the events they push are carried out).
So the callback doesn’t seem to be making it out of FMOD since the Unity log message below doesn’t show up until the app is activated:
[AOT.MonoPInvokeCallback(typeof(FMOD.CHANNELCONTROL_CALLBACK))]
public static RESULT ChannelCallback(IntPtr channelcontrol, CHANNELCONTROL_TYPE controltype, CHANNELCONTROL_CALLBACK_TYPE callbacktype, IntPtr commanddata1, IntPtr commanddata2) {
if (callbacktype == CHANNELCONTROL_CALLBACK_TYPE.END) {
Logger.Debug($"ChannelCallback: {controltype}, {callbacktype}");
NowPlaying.StayAwake();
}
return RESULT.OK;
}
iOS:
#define STAY_AWAKE_INTERVAL 2.0
double gAwakeUntil = 0;
extern "C" void iOS_StayAwake() {
bool asleep = gAwakeUntil <= 0;
gAwakeUntil = CACurrentMediaTime() + STAY_AWAKE_INTERVAL; // bump the wakeup time forward
if (asleep) {
[NSTimer scheduledTimerWithTimeInterval:.01 repeats:YES block:^(NSTimer * _Nonnull timer) {
UnityBatchPlayerLoop();
if (gAwakeUntil < CACurrentMediaTime()) {
gAwakeUntil = 0;
NSLog(@"\nENTER SLEEP\n");
[timer invalidate];
}
}];
}
}
// Unrelated but shows StayAwake() used to carry out events pushed to Unity
MPRemoteCommandHandlerStatus dispatchUnityMessage(const char* method, const char* msg) {
UnitySendMessage("NowPlayingEvents", method, msg);
iOS_StayAwake();
return MPRemoteCommandHandlerStatusSuccess;
}
So:
The only way I’m able to replicate your issue with the channel end callback never firing is if I never set up the AudioSession incorrectly/don’t set it up at all. When I set up the AudioSession correctly, it works as expected, but using your code in the channel callback results in calling UnityBatchPlayerUpdate() from the FMOD thread, not the main thread, causing a crash. As a result, you will likely need to move your UnityBatchPlayerLoop() call to the main thread to achieve the desired behavior.
Below is an Objective C plugin that sets up the audio session, and creates the timer used to call UnityBatchPlayerLoop() from the main thread, when and only when the application has been suspended. I’m not sure if it will exactly fit what you’re looking for, especially since the timer is a no-op when the application is in focus, but it should serve as an example of what to do:
#import <Foundation/Foundation.h>
#import <AVFoundation/AVFoundation.h>
bool shouldExecute = false;
extern "C" void SetUpAudioSession()
{
AVAudioSession* audioSession = [AVAudioSession sharedInstance];
[audioSession setCategory: AVAudioSessionCategoryPlayback mode: AVAudioSessionModeDefault options: AVAudioSessionCategoryOptionDuckOthers error: nil];
[audioSession setActive: YES error: nil];
}
extern "C" void SetUpTimer(){
NSLog(@"\nTIMER STARTED\n");
[NSTimer scheduledTimerWithTimeInterval:.01 repeats:YES block:^(NSTimer * _Nonnull timer) {
if (shouldExecute){
UnityBatchPlayerLoop();
}
}];
}
extern "C" void iOS_Suspended() {
NSLog(@"\nSUSPENDED\n");
shouldExecute = true;
}
extern "C" void iOS_Resumed() {
NSLog(@"\nRESUMED\n");
shouldExecute = false;
}
I call SetUpAudioSession() and SetUpTimer() in MonoBehavior.Awake(), which will allow FMOD to remain active in the background, and set up the timer used to update Unity while backgrounded. iOS_Suspended() and iOS_Resumed() are called from MonoBehavior.OnApplicationFocus(focus) on focus = false and focus = true respectively, to make the timer only update Unity when suspended.
I hope that this helps point you in the right direction.
Thank you so much for your support in this Louis. It means a lot here and you’ve won us over in our relationship with FMOD moving forward.
Will report back as we figure this out.
Thanks!