Our project requires enabling voice chat while playing sounds through fmod at the same time. However sound does not play as expected when AVAudioSession category and mode are configured for voice chat.
Reproduction steps are as follows.
Start playing any sound (UFMODAudioComponent::Play)
Call IOSAppDelegate::EnableVoiceChat and IOSAppDelegate::EnableHighQualityVoiceChat. (This is equivalent to calling AVAudioSession::setCategory with AVAudioSessionCategoryPlayAndRecord with mode AVAudioSessionModeVoiceChat with options AVAudioSessionCategoryOptionDefaultToSpeaker | AVAudioSessionCategoryOptionMixWithOthers)
Sound starts playing through both the speaker and the receiver. (If was connected to a bluetooth device, sound stops).
We expect that sound plays though only the speaker (not the reciever), and when connected to a bluetooth device, through that device only.
Everything is ok when voice chat is not enabled, but our voice chat module (Vivox) requires that we set AVAudioSessionCategory to those listed above.
After your reply I found out that the issue might have something to do with the IOS version. We have two iPhone 8 devices one with IOS version 15.5 and one with 15.6.1. I installed the same ipa file to both devices and the issue was reproduced on 15.6.1 and not on 15.5
Thank you for the additional information. After some discussion with the development team I’m not sure there is much we can do about this issue unfortunately; we don’t have any control over which output device we send audio to on iOS as this is handled by AVAudioSessionCategory / AVAudioSessionCategoryOptions, which are expected to be set by the user depending on their unique requirements. If you are finding this issue does not reproduce with regular Unreal audio however then there may be an issue with FMOD that we can investigate further.
Does the bluetooth issue still reproduce on 15.5? I saw in the Vivox docs that you also need to set AVAudioSessionCategoryOptionAllowBluetooth and/or AVAudioSessionCategoryOptionAllowBluetoothA2DP to allow audio to send to a bluetooth device, which may be worth investigating if you haven’t yet.
Yes, the bluetooth issue is still reproduced and it might not have been related to IOS version after all. I tested with another iPhone 12 device with IOS 15.5 installed and it was no different than with 15.6.1. It might be that the device that looked normal happened to have its receiver broken. I’m sorry for the misinformation.
When tested with the default audio library, sound did play with a bluetooth device although it was quite laggy for some reason. Below is the code I used. (EAudioFeature::BluetoothMicrophone sets AVAudioSessionCategoryOptionAllowBluetooth)
Apologies for the delayed response, and thank you for the additional information. Unfortunately I have still been unable to reproduce this issue; does removing the AVAudioSessionCategoryOptionDefaultToSpeaker flag help at all? That is the only one I can see in the AVAudioSession docs that might explain why it is being forced to the receiver and not the bluetooth headset.
Bluetooth does have significantly more latency than a wired connection. There are a few things you can do to try to reduce latency which you can find in the Platforms | iOS section of the docs.
Hi. I’m sorry for the delayed reply. We are still experiencing the issue even with the AVAudioSessionCategoryOptionDefaultToSpeaker suggestion. I’m re-listing the exact setup and reproduction steps here to avoid any confusion.
Mobile Device: iPhone 12 with IOS 15.5
Bluetooth Device: Samsung Galaxy Buds Pro
UE4 version: 4.27 built from source with minimal modification in IOSToolChain.cs to ignore unused variable errors with Xcode 13
FMOD version: 2.02.09 for UE4.27
Issue: Sound does not play with the following code. (Sound plays normally if IOSAppDelegate calls are removed)
Thank you for sending that over, I can reproduce the initial issue of audio outputting through the receiver, but I am finding bluetooth is behaving as expected on my test device.
Audio outputting through the receiver is expected behaviour when using the UseReceiver feature. On iOS the output device is set by IOSAppDelegate calls, as you have done here, and we do not have any additional control over which speaker to send audio to. If I remove the UseReceiver line from your repro or change it to NO, audio outputs through the speaker on iOS:
The issues are still here even after setting NO to for SetFeature: EAudioFeature::UseReceiver, but I found out that everything is normal (almost) when tested with Airpods. It causes problems when the connected device is Samsung Galaxy Buds 2 or Buds Pro (which are the two devices I tested with but might be the same on other non-Apple devices).
I pushed a small change to the repo I shared with you and following the steps below, sound played either not at all, or through the phone (not the bluetooth device).
Open the app with no device connected. (sound should play normally here)
Call gg 1 1 through console. This is an Exec function I added.
Connect a bluetooth device (preferably a samsung device)
The bluetooth device plays the tone that’s usually played when a phone call ends and now sound starts playing through the phone.
If step 4 doesn’t happen, sound stops. In this case you might as well close the app and try again from step 1. I could observe the issue happening once every 2 tries.
Although Airpods did not stop playing sound with the above steps, with gg 1 1 the sound played like from a distance and was not so clear. It was all good with gg 0 0 and gg 1 0.
Apologies for the delayed response, unfortunately I still have not been able to reproduce this issue with your updates to the repo and using your exec function.
That said, I only have access to a generic bluetooth speaker and not Samsung Galaxy Buds or Airpods.
I will see if this is something we can procure and will test it again with a closer matching hardware setup.
Hi. I have found some similar problems on Windows that might help in your investigation.
Vivox is required for the reproduction and you also need a bluetooth device that is displayed as “Hands-Free AG Audio” in Windows sound settings. In my case I tested with a Galaxy Buds Pro and a generic bluetooth headset from Beats by Dre. When the devices were first connected, on the sound panel, “Stereo” of the device was the default playback device and “Hands-Free AG Audio” was the default communication device and default recording device.
Everything is normal at first and sound plays nicely through FMOD but as soon as you enter a Vivox channel, all sound is muted. On the sound panel, the real time volume bar shows that the sound is playing through “Stereo” of the device but nothing is heard from the device itself.
Then if you disable “Stereo”, sound starts playing through “Hands-Free AG Audio” and you can actually hear it playing through the device. However, the sound is somewhat distorted and feels like from a distance (just like what happened on IOS). Still, both FMOD and Vivox do play. If you disable everything (both playback and recording) and turn “Stereo” back on, FMOD is normal but Vivox stops playing.
Since everything was normal before Vivox intervened, it might be that as Vivox starts using some OS functionality that involves Communication (Windows) or VoiceChat (iOS), devices that support playing this type of sound are causing issues trying to do their thing.
I am not familiar with any technical details so it might be something entirely different. My apolgoies if that is the case but I do hope that this is something relevant and can help you resolve the issues.
Might this be the cause of the issues? It might be possible that on iOS what happened was as the voice chat setting was switched back and forth the bluetooth devices were disabled and sound started playing through the main device which is the internal speaker of the phone.
I have checked the setting and it was already at “Do nothing”.
I am still investigating as to why the issues might not be reproduced in your environment but in ours. I will leave a comment when I have more information. Since I’m close to being out of ideas, any suggestions would be greatly appreciated.
Thank you for confirming, I am thinking maybe FMOD and Vivox could compete for the audio device if Exclusive Mode is enabled. Can you please go to your device’s properties and ensure they have Exclusive Mode disabled?
On that properties page, can you please tell me what the Default Format for each of these devices are? If the hands-free device has a much lower sample rate that would explain why sound is coming out muffled. If FMOD is switching to a different device (which sounds like it might be the case) then there should be a log when it occurs. To confirm this, can you please enable Enable API Error Logging and set the Logging Level to “LEVEL LOG” and share the log you get while reproducing this issue?
When you say all sound is muted, is it all of your app’s sound or all of your system’s sound? For example, if you launch another application and play audio, does that play sound or is that muted as well?
This solved the problem of Vivox not playing and now both FMOD and Vivox work great, but only when in Stereo.
The hands-free device had a lower sample rate than Stereo. I wanted to see if I could make it better but the drop down was fixed and I could not change the sample rate. Thank you for this info.
I apologize for the reply being delayed. I got hit by covid and have been working from home. I will have the log you suggested and the answer to your last question when I get back to the office next week.
Attached here is the log I obtained from the steps above.
I tried playing Apple Podcast and running the sample app. At first both Podcast and FMOD were okay but as soon as I typed gg 1 1 all sound (including the Podcast) stopped. When I checked the status bar Podcast had been paused and resuming it worked but once I returned to the sample app Podcast paused. It looked like the sample app was preventing both FMOD and Podcast from playing sound.
Thank you for the additional information- I hope you are feeling better now.
I have reproduced this issue now with the AirPods, I think the exec function isn’t working on my end, and was appearing as a false positive when I was testing previously.
Modifying the BeginPlay method to call gg(1, 1) or gg(1, 0), instead of using the Exec fucntion, is reproducing the behaviour you have described:
Audio plays quietly out of receiver
Connecting AirPods results in no audio coming out of the test app
Audio plays on other applications without issue
Disconnecting AirPods resumes audio playing out of receiver
Modifying the BeginPlay method to call gg(0, 0) or gg(0, 1) is behaving as expected:
Audio plays out of the device’s speakers
Connecting AirPods switches output to AirPods
Audio plays on other applications without issue
Disconnecting AirPods resumes audio playing out of device’s speakers
I have found that Unreal Audio plays back on AirPods when running with gg(1, 1), though the audio is crackling and distorted. In any case, I would expect FMOD’s behavior to match Unreal’s behavior, so I have passed this onto the dev team to investigate further- thank you for the reproduction and for your patience with helping me reproduce this issue.