Crash OnApplicationPause when recording voice

Hi,
I’m new to FMOD - so I maybe missing something here.
We have a basic system that records and plays back the players voice using a dspbuffer.


      void Start()
        {

  _soundInfo = new CREATESOUNDEXINFO
            {
                cbsize = Marshal.SizeOf(typeof(CREATESOUNDEXINFO)),
                length = (uint)(SampleRate * sizeof(byte) * NumBytesPerSample * NumInputChannels),
                numchannels = NumInputChannels,
                channelorder = CHANNELORDER.ALLMONO,
                defaultfrequency = SampleRate,
                format = SOUND_FORMAT.PCM16,
                dlsname = IntPtr.Zero,
            };
            MODE mode = MODE.OPENUSER | MODE.LOOP_NORMAL;

            RuntimeManager.CoreSystem.createSound("Test", mode, ref _soundInfo, out _micSound); 
            RuntimeManager.CoreSystem.recordStart(RecordingDeviceIndex, _micSound, true);


            mReadCallback = CaptureDSPReadCallback;
        	mObjHandle = GCHandle.Alloc(this);
        	if (mObjHandle != null)
        	{
        		// Define a basic DSP that receives a callback each mix to capture audio
        		FMOD.DSP_DESCRIPTION desc = new FMOD.DSP_DESCRIPTION();
        		desc.numinputbuffers = 1;
        		desc.numoutputbuffers = 1;
        		desc.read = mReadCallback;
        		desc.userdata = GCHandle.ToIntPtr(mObjHandle);
        
        		if (FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out mCaptureDSP) == FMOD.RESULT.OK)
        		{
        			if (_channelGroup.addDSP(0, mCaptureDSP) != FMOD.RESULT.OK)
        			{
        				Debug.LogWarningFormat("FMOD: Unable to add mCaptureDSP to the master channel group");
        			}
        		}
        		else
        		{
        			Debug.LogWarningFormat("FMOD: Unable to create a DSP: mCaptureDSP");
        		}
        	}
        	else
        	{
        		Debug.LogWarningFormat("FMOD: Unable to create a GCHandle: mObjHandle");
        	}



      micInstance = FMODUnity.RuntimeManager.CreateInstance(micSfxEvent);
	micInstance.start();

   RuntimeManager.CoreSystem.playSound(_micSound, _channelGroup, false, out _channel);




...
}

This works well and creates a nice echo for singing and cool voices.
When the unity application shuts down, we can capture the moment via OnApplicationPause…

At that point in time all i really want is for things to suspend and then to resume. The moment the application clocks out we get a null pointer dereference (i think because the microphone is no longer available but a callback is trying to be made on it)…

I’m lost and also probably a bit out of my depth. I have tried removing the DSP from the channel group, calling stop and release on the sounds and channels, suspending the mixer, pausing all events and calling recordStop - nothing seems to consistently work. Calling RuntimeManager.CoreSystem.recordStop(index) instantantly causes the dereference issue to take place. If i don’t call it it breaks as soon a it ‘properly clocks off’ and goes into standby. RecordStop might be being called naturally at this point anyway (so i’m just seeing the error sooner).
Calling mixerSuspend / lock dsp / Pauseallevents seemed promising as it seemed like nothing would be processed anymore. But trying to do these on application pause didn’t prove to be a fix.

Would be grateful for any advice or general direction - have been stuck here for a day or so.

Error specifically is:

2024/04/24 15:12:08.911 27277 27297 Error CRASH *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2024/04/24 15:12:08.912 27277 27297 Error CRASH Version '2022.2.6f1 (10bfa6201ced)', Build type 'Development', Scripting Backend 'il2cpp', CPU 'arm64-v8a'
2024/04/24 15:12:08.912 27277 27297 Error CRASH Build fingerprint: 'oculus/hollywood/hollywood:12/SQ3A.220605.009.A1/50837850062000150:user/release-keys'
2024/04/24 15:12:08.912 27277 27297 Error CRASH Revision: '0'
2024/04/24 15:12:08.912 27277 27297 Error CRASH ABI: 'arm64'
2024/04/24 15:12:08.912 27277 27297 Error CRASH Timestamp: 2024-04-24 15:12:08.912323136+1200
2024/04/24 15:12:08.912 27277 27297 Error CRASH pid: 27277, tid: 27297, name: UnityMain  >>> com.ME.singing <<<
2024/04/24 15:12:08.912 27277 27297 Error CRASH uid: 10116
2024/04/24 15:12:08.912 27277 27297 Error CRASH signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr --------
2024/04/24 15:12:08.912 27277 27297 Error CRASH Cause: null pointer dereference
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x0  b40000776a594a10  x1  b4000078b288cc58  x2  0000000000000000  x3  00000000010653ee
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x4  0000000000000000  x5  0000000000000002  x6  0000008000000000  x7  434e4c44ff6f6e73
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x8  000000794498b6a8  x9  0000000000000038  x10 0000000000000002  x11 00000000000f4240
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x12 00000000010653ee  x13 00000791c6c8bc76  x14 000707bdb793ca44  x15 0000000034155555
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x16 00000079449b3ce0  x17 00000079f61aa114  x18 00000078e475aab8  x19 b40000776a594a10
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x20 0000000000000000  x21 0000000000000000  x22 00000078cc36e4fc  x23 00000078c9cb6cd0
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x24 00000078e475ccc0  x25 00000078e475ccc0  x26 0000000000000010  x27 0000000000000001
2024/04/24 15:12:08.912 27277 27297 Error CRASH     x28 00000078e33439e0  x29 00000078e475ab30
2024/04/24 15:12:08.912 27277 27297 Error CRASH     lr  0000007944974b88  sp  00000078e475ab20  pc  000000794498b6bc  pst 0000000060001000
2024/04/24 15:12:08.912 27277 27297 Error CRASH backtrace:
2024/04/24 15:12:08.912 27277 27297 Error CRASH       #00 pc 00000000000776bc  <anonymous:0000007944914000>


2024/04/24 15:23:29.313 28262 28476 Error CRASH       #01 pc 0000000000024348  /system/lib64/libaaudio_internal.so (aaudio::AudioStream::maybeCallDataCallback(void*, int)+260) (BuildId: 81ff7d79a4f780a537d64c2090b8cdeb)
2024/04/24 15:23:29.313 28262 28476 Error CRASH       #02 pc 0000000000026e24  /system/lib64/libaaudio_internal.so (aaudio::AudioStreamLegacy::callDataCallbackFrames(unsigned char*, int)+272) (BuildId: 81ff7d79a4f780a537d64c2090b8cdeb)
2024/04/24 15:23:29.313 28262 28476 Error CRASH       #03 pc 0000000000027650  /system/lib64/libaaudio_internal.so (aaudio::AudioStreamLegacy::processCallbackCommon(int, void*)+584) (BuildId: 81ff7d79a4f780a537d64c2090b8cdeb)
2024/04/24 15:23:29.313 28262 28476 Error CRASH       #04 pc 000000000002aa7c  /system/lib64/libaaudio_internal.so (aaudio::AudioStreamRecord::processCallback(int, void*)+92) (BuildId: 81ff7d79a4f780a537d64c2090b8cdeb)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #05 pc 0000000000058354  /system/lib64/libaudioclient.so (android::AudioRecord::processAudioBuffer()+1260) (BuildId: 41f1fc2d7af80cc20ff755c4ccc59752)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #06 pc 0000000000057bf0  /system/lib64/libaudioclient.so (android::AudioRecord::AudioRecordThread::threadLoop()+268) (BuildId: 41f1fc2d7af80cc20ff755c4ccc59752)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #07 pc 00000000000121d4  /system/lib64/libutils.so (android::Thread::_threadLoop(void*)+348) (BuildId: 5fd4d0ed031dbc845d6e37c986029ae7)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #08 pc 00000000000cba44  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::javaThreadShell(void*)+140) (BuildId: dec3998de38d092e4b6f71d6c870309e)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #09 pc 0000000000011b78  /system/lib64/libutils.so (BuildId: 5fd4d0ed031dbc845d6e37c986029ae7)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #10 pc 00000000000df308  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+132) (BuildId: 0121d6b135c6faf9599fd1ea120a8cf3)
2024/04/24 15:23:29.314 28262 28476 Error CRASH       #11 pc 000000000008b36c  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 0121d6b135c6faf9599fd1ea120a8cf3)

As an addendum - if i call RuntimeManager.CoreSystem.recordStop(index) prior to pausing the application - things actually stop and resume fine. I think this is a case that the ‘OnApplicationPause’ event is just a notification - i don’t unity waits for all methods to complete prior to pausing the application.

1 Like

Hi,

Thank you for sharing the solution. Are you still adding the mCaptureDSP to the master channel group? I think the issue may be linked to its callback. Otherwise, stoping and resuming recording is a valid solution.

Hi Connor, thanks for responding!
While we have a fix - the fix doesn’t resolve the full problem.

If we call ‘stop recording’ while not in the ‘onapplicationpause’ exit method - things are a-ok… And if we go into ApplicationPause post that point things won’t crash…

But because it’s a singing game they could be singing and then just all of a sudden stop and take off their headset - if I call RuntimeManager.CoreSystem.recordStop(0); during this ‘onapplicationpause process’ things break then and there - i think because FMOD itself actually begins to shutdown processes automatically when it’s detected than it’s going to be going into pause mode. If I don’t call recordStop things break a frame later (i think because of the callback of mCaptureDSP).

If i remove the mCaptureDSP callback the the voice playback effect (of a very low latency echo) doesn’t work very well.

The current on application pause code looks like this:


        void ProcessDeviceState()
        {
           
            if(isRecordingActive && (!isApplicationFocused || isApplicationPaused))
            {
                isRecordingActive = false;
                Debug.Log("Pausing record");

         
                _channelGroup.removeDSP(mCaptureDSP);
                mCaptureDSP.release();
                _micSound.release();
                RuntimeManager.CoreSystem.recordStop(0);
                return;

I’ve tried a bunch of variations to this exit strategy - but I am very new here - so could just be doing dumb stuff and not realise it. But none of this stops the crash.

Not sure. I’ve read somewhere else on the forum that just shutting down fmod entirely on application pause could be a valid way to go - but I haven’t actually been able to do that within this ‘onapplication pause’ without getting an error either on the way out (or application resume). Lost still.

Thank you for sharing the code.

Unfortunately, I am still unable to reproduce the issue.

Could you please try suspending the mixer RuntimeManager.StudioSystem.mixerSuspend() in your ProcessDeviceState() func. I would suggest calling RuntimeManager.StudioSystem.flushCommands() immediately after as this will block the calling thread until all pending commands have been executed and ensure that the mixer is suspended.

Could you elaborate a bit more about that? Would it be possible to see what the DSP is doing in code?

If possible, could I please get a stripped-out version of the Unity project displaying the issue uploaded to your Profile? Please note that you must register a project with us before uploading files.

Hi Connor!
Thanks for getting back to me again. I’ve begun the process for registering a new project and am currently awaiting approval. I’ve also made a stripped down version of the project so will send it your way once I’ve been cleared.

I tried the mixerSuspend followed by the flush command step. The program goes through the OnApplicationPause step without issue, but breaks a frame later (just before the oculus device goes idle). If I add the recordingStop step anywhere during this ProcessDeviceState step things break as soon as that method is hit - if i don’t call it, things break a frame later. A frame later I get the null pointer dereference error;

2024/04/24 15:12:08.912 27277 27297 Error CRASH pid: 27277, tid: 27297, name: UnityMain >>> com.ME.singing <<<
2024/04/24 15:12:08.912 27277 27297 Error CRASH uid: 10116
2024/04/24 15:12:08.912 27277 27297 Error CRASH signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr --------
2024/04/24 15:12:08.912 27277 27297 Error CRASH Cause: null pointer dereference

If i call recording.stop during this on application suspend

This is the dsp playback code (ie the code that routes the players audio back out to the speakers)

    void CreateSoundStartRecording()
        {
            MODE mode = MODE.OPENUSER | MODE.LOOP_NORMAL;
            RuntimeManager.CoreSystem.createSound("test", mode, ref _soundInfo, out _micSound);
            RuntimeManager.CoreSystem.recordStart(RecordingDeviceIndex, _micSound, true);

            _micSound.getLength(out uint length, TIMEUNIT.PCM);
            _pcmBuffer = new short[length];

            // Ading dsp capture
            // Assign the callback to a member variable to avoid garbage collection
            // Allocate a data buffer large enough for 8 channels
            uint bufferLength;
            int numBuffers;
            FMODUnity.RuntimeManager.CoreSystem.getDSPBufferSize(out bufferLength, out numBuffers);
            mDataBuffer = new float[bufferLength * 8];
            mBufferLength = bufferLength;
        
            // Get a handle to this object to pass into the callback
            mReadCallback = CaptureDSPReadCallback;
            mObjHandle = GCHandle.Alloc(this);
            if (mObjHandle != null)
            {
                // Define a basic DSP that receives a callback each mix to capture audio
                FMOD.DSP_DESCRIPTION desc = new FMOD.DSP_DESCRIPTION();
                desc.numinputbuffers = 1;
                desc.numoutputbuffers = 1;
                desc.read = mReadCallback;
                desc.userdata = GCHandle.ToIntPtr(mObjHandle);

                testSet = desc;
                if (FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out mCaptureDSP) == FMOD.RESULT.OK)
                {
                    if (_channelGroup.addDSP(0, mCaptureDSP) != FMOD.RESULT.OK)
                    {
                        Debug.LogWarningFormat("FMOD: Unable to add mCaptureDSP to the master channel group");
                    }
                }
                else
                {
                    Debug.LogWarningFormat("FMOD: Unable to create a DSP: mCaptureDSP");
                }
            }
            else
            {
                Debug.LogWarningFormat("FMOD: Unable to create a GCHandle: mObjHandle");
            }

            processInputSequence = StartCoroutine(ProcessInput());
        }

Which becomes tied (and performs a callback to this one…)

		[AOT.MonoPInvokeCallback(typeof(FMOD.DSP_READ_CALLBACK))]
		static FMOD.RESULT CaptureDSPReadCallback(ref FMOD.DSP_STATE dsp_state, IntPtr inbuffer, IntPtr outbuffer, uint length, int inchannels, ref int outchannels)
		{
            FMOD.DSP_STATE_FUNCTIONS functions = (FMOD.DSP_STATE_FUNCTIONS)Marshal.PtrToStructure(dsp_state.functions, typeof(FMOD.DSP_STATE_FUNCTIONS));

			IntPtr userData;
			functions.getuserdata(ref dsp_state, out userData);

			GCHandle objHandle = GCHandle.FromIntPtr(userData);
			RecordMic obj = objHandle.Target as RecordMic;

			// Save the channel count out for the update function
			obj.mChannels = inchannels;

			// Copy the incoming buffer to process later
			int lengthElements = (int)length * inchannels;
			Marshal.Copy(inbuffer, obj.mDataBuffer, 0, lengthElements);

			// Copy the inbuffer to the outbuffer so we can still hear it
			Marshal.Copy(obj.mDataBuffer, 0, outbuffer, lengthElements);

			return FMOD.RESULT.OK;
		}

So i’m not sure. It feels like theres a callback that during my shutdown sequence ends up trying to do something that’s already been packed up ie. FMOD has already begun shutting down processes just a moment before my OnApplicationPause gets hit - so calling StopRecording immediately gets you into a bad place (because the recording stuff has been packed away).
But I could be off in my analysis. If i remove some of these steps, the dsp, the playback or the recording, it can handle the OnApplicationPause step without breaking - it’s the combination that seems to get things upset.

Just to outline it all really clearly - this is the whole class. The process device state method has gone through a billion iterations of what it does - so it may look slightly different to the above code.

using System;
using System.Runtime.InteropServices;
using FMOD;
using UnityEngine.Serialization;

namespace Audio
{
    using FMODUnity;
    using UnityEngine;
    using System.Collections;
    using FMOD.Studio;
    using Sirenix.OdinInspector;

    public class RecordMic : MonoBehaviour
    {
        //public variables
        [Header("Choose A Microphone")]
        public int RecordingDeviceIndex = 0;

		[TextArea]
        public string RecordingDeviceName = null;

        public float Latency = .05f;

        //FMOD Objects
        private Sound _micSound;
        private CREATESOUNDEXINFO _soundInfo;
        private Channel _channel;
        private ChannelGroup _channelGroup;

        private int numOfDriversConnected = 0;
        private int numofDrivers = 0;

        private Guid MicGUID;
        private int SampleRate = 0;
        private SPEAKERMODE FMODSpeakerMode;
        private int NumOfChannels = 0;
        public const int NumBytesPerSample = 2;
        public const int NumInputChannels = 1;
        private DRIVER_STATE driverState;


        const float WIDTH = 0.01f;
        const float HEIGHT = 10.0f;
        const float YOFFSET = 5.0f;

        public static float micVolume;

        protected short[] _pcmBuffer;
        protected int _bufferPos;

        private int _latencySamples;
        private DSP _reverbDSP;


        [FormerlySerializedAs("_micSfxEvent")]
        [Header("Input Monitoring")]
        [SerializeField]
        [Tooltip("Path to the event for input monitoring")]
        EventReference micSfxEvent;

        Coroutine processInputSequence = null;
        EventInstance micInstance;

        private bool isRecordingActive = false;
        private bool isApplicationPaused;
        private bool isApplicationFocused;


        void Start()
        {
            Initialize();
            CreateSoundStartRecording();
        }


        void Initialize()
        {
            RuntimeManager.CoreSystem.createChannelGroup("Microphone ChannelGroup", out _channelGroup);
            LogMicrophoneAvailability();
        
            ConfigureSoundSettings();
            ConfigureReverb();
    
        }

        void LogMicrophoneAvailability()
        {
            RuntimeManager.CoreSystem.getRecordNumDrivers(out numofDrivers, out numOfDriversConnected);
            Debug.Log(numOfDriversConnected == 0 ? "Plug in a Microphone!!!" : $"You have {numOfDriversConnected} microphones available to record with.");
        }

        void ConfigureSoundSettings()
        {
            RuntimeManager.CoreSystem.getRecordDriverInfo(RecordingDeviceIndex, out RecordingDeviceName, 50, out MicGUID, out SampleRate, out FMODSpeakerMode, out NumOfChannels, out driverState);
            _soundInfo = new CREATESOUNDEXINFO
            {
                cbsize = Marshal.SizeOf(typeof(CREATESOUNDEXINFO)),
                length = (uint)(SampleRate * sizeof(byte) * NumBytesPerSample * NumInputChannels),
                numchannels = NumInputChannels,
                channelorder = CHANNELORDER.ALLMONO,
                defaultfrequency = SampleRate,
                format = SOUND_FORMAT.PCM16,
                dlsname = IntPtr.Zero,
            };
            _latencySamples = (int)(SampleRate * Latency);
        }

        void ConfigureReverb()
        {
            REVERB_PROPERTIES reverbRoom = PRESET.ROOM();
            REVERB_PROPERTIES reverbOff = PRESET.OFF();
            REVERB_PROPERTIES reverbArena = PRESET.ARENA();
            //  RuntimeManager.CoreSystem.setReverbProperties(1, ref reverbArena);
            RuntimeManager.CoreSystem.createDSPByType(DSP_TYPE.SFXREVERB, out _reverbDSP);

            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.DECAYTIME, 2.0f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.EARLYDELAY, 0.1f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.LATEDELAY, 0.01f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.HFREFERENCE, 5000f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.HFDECAYRATIO, 50f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.DIFFUSION, 70f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.DENSITY, 70f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.LOWSHELFFREQUENCY, 200f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.LOWSHELFGAIN, 0f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.HIGHCUT, 10000f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.EARLYLATEMIX, 50f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.WETLEVEL, -6f);
            _reverbDSP.setParameterFloat((int)FMOD.DSP_SFXREVERB.DRYLEVEL, 0f);
            _channelGroup.addDSP(0, _reverbDSP);
        }



        FMOD.DSP_DESCRIPTION testSet;

        void CreateSoundStartRecording()
        {
            MODE mode = MODE.OPENUSER | MODE.LOOP_NORMAL;
            RuntimeManager.CoreSystem.createSound("test", mode, ref _soundInfo, out _micSound);
            RuntimeManager.CoreSystem.recordStart(RecordingDeviceIndex, _micSound, true);

            _micSound.getLength(out uint length, TIMEUNIT.PCM);
            _pcmBuffer = new short[length];

            // Ading dsp capture
            // Assign the callback to a member variable to avoid garbage collection
            // Allocate a data buffer large enough for 8 channels
            uint bufferLength;
            int numBuffers;
            FMODUnity.RuntimeManager.CoreSystem.getDSPBufferSize(out bufferLength, out numBuffers);
            mDataBuffer = new float[bufferLength * 8];
            mBufferLength = bufferLength;
        
            // Get a handle to this object to pass into the callback
            mReadCallback = CaptureDSPReadCallback;
            mObjHandle = GCHandle.Alloc(this);
            if (mObjHandle != null)
            {
                // Define a basic DSP that receives a callback each mix to capture audio
                FMOD.DSP_DESCRIPTION desc = new FMOD.DSP_DESCRIPTION();
                desc.numinputbuffers = 1;
                desc.numoutputbuffers = 1;
                desc.read = mReadCallback;
                desc.userdata = GCHandle.ToIntPtr(mObjHandle);

                testSet = desc;
                if (FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out mCaptureDSP) == FMOD.RESULT.OK)
                {
                    if (_channelGroup.addDSP(0, mCaptureDSP) != FMOD.RESULT.OK)
                    {
                        Debug.LogWarningFormat("FMOD: Unable to add mCaptureDSP to the master channel group");
                    }
                }
                else
                {
                    Debug.LogWarningFormat("FMOD: Unable to create a DSP: mCaptureDSP");
                }
            }
            else
            {
                Debug.LogWarningFormat("FMOD: Unable to create a GCHandle: mObjHandle");
            }

            processInputSequence = StartCoroutine(ProcessInput());
        }

        void OnApplicationPause(bool pauseStatus)
        {
           #if !UNITY_EDITOR
            isApplicationPaused = pauseStatus;
            ProcessDeviceState();
           #endif
        }

        void OnApplicationFocus(bool hasFocus)
        {
            #if !UNITY_EDITOR
            isApplicationFocused = hasFocus;
            ProcessDeviceState();
            #endif
        }
   

        void ProcessDeviceState()
        {
           
            if(isRecordingActive && !isApplicationFocused || isApplicationPaused)
            {
                Debug.Log("Pausing record");
                StopCoroutine(processInputSequence);
                testSet.read = null;

                isRecordingActive = false;

                _channel.stop();


                RuntimeManager.CoreSystem.update();
                RuntimeManager.CoreSystem.mixerSuspend();
                RuntimeManager.StudioSystem.flushCommands();
                RuntimeManager.WaitForAllSampleLoading();
                RuntimeManager.CoreSystem.recordStop(RecordingDeviceIndex);
                Debug.Log("All steps in pause record concluded.");

            }
            else if(!isRecordingActive && isApplicationFocused && !isApplicationPaused)
            {
                isRecordingActive = true;
                Debug.Log("Resuming record");

            }
        }


        void LateUpdate()
        {
            if (isRecordingActive)
            {
                FillPCMBuffer(out int numSamples);
                UpdateDSPViz();
            }
        }


		void UpdateDSPViz()
		{
            float frameVolume = 0f;
			// Do what you want with the captured data
			for (int j = 0; j < mBufferLength; j++)
			{
				for (int i = 0; i < mChannels; i++)
				{
                    frameVolume += Mathf.Abs(mDataBuffer[(j * mChannels) + i]);

					//float x = j * WIDTH;
					//float y = mDataBuffer[(j * mChannels) + i] * HEIGHT;

					// Make sure Gizmos is enabled in the Unity Editor to show debug line draw for the captured channel data
					//Debug.DrawLine(new Vector3(x, (YOFFSET * i) + y, 0), new Vector3(x, (YOFFSET * i) - y, 0), Color.green);
				}
			}
            micVolume = frameVolume / mDataBuffer.Length * HEIGHT;
		}

       

		IEnumerator ProcessInput()
        {
            isRecordingActive = true;
            micInstance = FMODUnity.RuntimeManager.CreateInstance(micSfxEvent);
			var createResult = micInstance.start();

            if (createResult != RESULT.OK) {
                Debug.LogError("[Fmod] Had an issue instantiating the input monitoring event.");
            }

            FMOD.Studio.PLAYBACK_STATE playbackState;
            var attemptCount = 0;
            do {
                attemptCount++;
                if (attemptCount > 50) {
                    Debug.LogError("AUDIO IO MANAGER: reached attempt limit preparing input monitoring event instance");
                    yield break;
                }

                createResult = micInstance.getPlaybackState(out playbackState);
                if (createResult != RESULT.OK) {
                    Debug.Log("Too MANY TRIES!!!");
                    yield break;
                }

                yield return new WaitForEndOfFrame();
            } while (playbackState != FMOD.Studio.PLAYBACK_STATE.PLAYING);

            var result = micInstance.getPlaybackState(out playbackState);
            
            RuntimeManager.CoreSystem.playSound(_micSound, _channelGroup, false, out _channel);

            while (isRecordingActive)
            {
                createResult = _channel.setPosition((uint)(int)Mathf.Repeat(_bufferPos - _latencySamples, _pcmBuffer.Length), TIMEUNIT.PCM);
                yield break;
            }
        }

        protected bool FillPCMBuffer(out int numSamples)
        {
            numSamples = 0;
            if (!isRecordingActive)
            {
                return false;
            }
            RESULT result = RuntimeManager.CoreSystem.getRecordPosition(RecordingDeviceIndex, out uint recPos);
            if (result != RESULT.OK) return false;
            if (recPos == _bufferPos) return true;
            numSamples = (int)Mathf.Repeat((int)recPos - _bufferPos, _pcmBuffer.Length);

            result = _micSound.@lock(
                (uint)_bufferPos * NumBytesPerSample,
                (uint)numSamples * NumBytesPerSample,
                out IntPtr ptr1, out IntPtr ptr2, out uint len1, out uint len2);

            if (result != RESULT.OK) return false;
            Marshal.Copy(ptr1, _pcmBuffer, _bufferPos, (int)len1 / NumBytesPerSample);
            if (len2 > 0) Marshal.Copy(ptr2, _pcmBuffer, 0, (int)len2 / NumBytesPerSample);
            result = _micSound.unlock(ptr1, ptr2, len1, len2);
            if (result != RESULT.OK) return false;
            _bufferPos = (int)recPos;
            return true;
        }

		private FMOD.DSP_READ_CALLBACK mReadCallback;
		private FMOD.DSP mCaptureDSP;
		private float[] mDataBuffer;
		private GCHandle mObjHandle;
		private uint mBufferLength;
		private int mChannels = 0;

		[AOT.MonoPInvokeCallback(typeof(FMOD.DSP_READ_CALLBACK))]
		static FMOD.RESULT CaptureDSPReadCallback(ref FMOD.DSP_STATE dsp_state, IntPtr inbuffer, IntPtr outbuffer, uint length, int inchannels, ref int outchannels)
		{
            FMOD.DSP_STATE_FUNCTIONS functions = (FMOD.DSP_STATE_FUNCTIONS)Marshal.PtrToStructure(dsp_state.functions, typeof(FMOD.DSP_STATE_FUNCTIONS));

			IntPtr userData;
			functions.getuserdata(ref dsp_state, out userData);

			GCHandle objHandle = GCHandle.FromIntPtr(userData);
			RecordMic obj = objHandle.Target as RecordMic;

			// Save the channel count out for the update function
			obj.mChannels = inchannels;

			// Copy the incoming buffer to process later
			int lengthElements = (int)length * inchannels;
			Marshal.Copy(inbuffer, obj.mDataBuffer, 0, lengthElements);

			// Copy the inbuffer to the outbuffer so we can still hear it
			Marshal.Copy(obj.mDataBuffer, 0, outbuffer, lengthElements);

			return FMOD.RESULT.OK;
		}
	}
}

Appreciate your time in looking at this. If theres anything else specific that I can provide please let me know.

I have this theory that this is a just in time problem. If i remove some of the logging (and have the class a certain size?) i can get things to not crash. But if I re-add lines to the class (ie. a method thats not even called…) - then it’ll crash. It seems temperamental - I still don’t feel like i have a proper handle on it - but just to give as much info as i can.

1 Like

Have uploaded a copy of the apk (for install on oculus) as well as a stripped down version of the unity project (with the library removed - so perhaps a long inital load while it remakes it).
Editor version is 2022.2.6f1.

If theres anything further I can provide to try and help please let me know. Appreciate the support.

Thank you for providing the project.

I was able to get it running, however I am not getting any mic input at all.

Would it be possible to get some instructions for how to start recording audio in the game.

The project is super cool!

Hi Connor!
Glad to hear from you! Just checking - are you running the project on an Oculus?
If you are - do you mind manually checking what the microphone permissions are. This can be found by going to:

Settings > Apps > Permissions > Microphone > [finding the app] and enabing. The permissions request may not of been pushed in the build you have.

Playback should work ok on the project directly - did things work okay on the project itself - ie. does the project load ok? If you press play on the main scene the game should grab your computer microphone and work.
[Main_VRSingTogether] should be the scene to look at. When you press play there things should start. If you’re on a PC as opposed to a mac look for a gameobject called ‘Util’ in the hierachy and press ‘Switch based on OS’ just to swap things over to OculusSpatializer rather than libOCulusSpatializer- that might of been a problem…

Hopefully these instructions aren’t to convoluted. If anything becomes undone I’m happy to make a video stepping through these things.

HI, since Connor is away, I will be continuing the investigation.

Thanks for the detailed instructions! I’ve followed the steps to enable the microphone permissions on the Oculus. I also confirmed that the project loads properly and the playback works now as expected on my end.

However, I’m still unable to reproduce the crash you mentioned. Could you please provide a bit more detail about the circumstances under which the crash occurs? For example,

Would it be possible to share with me the Android version you have on your Quest 2? As we suspect we are using a different Android version which does not reproduce the issue on our end.

If you are not sure where to look for it, you could connect your headset and run adb shell getprop ro.build.version.release in your command prompt to get the Android version number.

Additionally, a video stepping through things would greatly assist in pinpointing the problem.

Hi Li,
Attached is a video of the logcat output for an Oculus 2 and a Oculus 3. To get the crash I start the app - wait five seconds then put the headset down.
The application must lose focus - this can be defined from the Oculus app - see screenshots on where this setting can be changed in order to hit that case.



So after i put the headset down and 15 seconds elapses the device enters into standby mode. If i call microphone.stoprecording on the OnApplicationSuspend things break immediately (I think because there is an internal call inside FMOD which has already started to shutdown FMOD). If I don’t call micrphone.stoprecording things crash on application resume (ie. when i pickup the headset again).

I suspect the detail I may of neglected to include was that the ‘on lost focus’ needs to be called.

Both versions are using version 12 of android. The crash has been replicated on all 4 devices we’ve tested with without fail.

This is a screenshot of the code thats called on applicationpause

But all this code can be commented out the crash will still take place as long as we’ve called this one;

ie. if we are recording and playing back the low latency echo. Things crash.
If i push ‘stop recording’ prior to putting the application into lost focus mode - things are fine - no crash… But we never know when someone is going to take off their headset.

Happy to provide further detail on anything that’ll help - hopefully this is enough for you to be able to get the issue turning up there.

Video link:

Thank you for the detailed information! That really helps a lot for us to pinpoint the issue.

With the script you provided, I have been able to replicate this crash in multiple projects.

Unfortunately I was not able to find a workaround for this problem, but I have passed the details of this issue to our dev team for further investigation.

Thank you for your patience and assistance with investigating this issue.

If you have any more insights or further queries in the meantime, please feel free to share, and we will be happy to assist.