I know that this isn’t a useful question to anyone and feel really sorry to ask here, but out of curiosity, does the latest Unity Engine internally use FMOD Studio?
I heard they used FMOD Ex for their older versions, but is it still true?
You heard correctly, Unity still uses FMOD Ex, version 4 of our Core API.
1 Like
Oh, I see.
Thanks!
Some follow-up questions.
-
Is Unity still receiving bug fixes or updates (such as AAudio support on Android) from the FMOD development team?
Because I don’t see any FMOD 4 downloads from the download page here. -
Are there any other open game engines/frameworks (not in-house tools) that use FMOD 5?
Even Unreal doesn’t use FMOD 5? -
How do I manually disable the built-in Unity audio to prevent it from consuming the resources?
I know FMOD Setup Wizard of the FMOD Unity plugin does this for us, but I’m currently integrating my previous C++ project myself.
I mean, I’m using FMOD Core API instead of FMOD for Unity, so need to do it manually.
Are removing the Audio Listener component and ticking the ‘Disable Unity Audio’ option in the Project Settings more than enough?
- FMOD 4 is out of support now, so they are not receiving fixes or improvements from us, however they do have a source license and have been maintaining the code themselves.
- None immediately come to mind, there are existing integrations with Unity, Unreal and CryEngine though.
- The “Disable Unity Audio” project option is what the setup wizard ticks for you, doing that and removing the listener should be sufficient to fully disable Unity built-in audio.
1 Like
Thank you.
One last question.
After ticking ‘Disable Unity Audio’, the sound of the Video Player component, which is Unity built-in API for video playback, got muted too.
It’s the expected behavior, but can this issue be solved with FMOD Core API or FMOD for Unity?
If not, do I need to have both audio APIs enabled?
The audio functionality of Unity video is played back with their audio system. FMOD doesn’t provide an integration with the video player system to handle this for you so the simplest solution is to leave the Unity audio system enabled and use it just for video.
Technically it’s possible for us to support this via the AudioSampleProvider experimental API Unity provides however that his an advanced topic and not a task that we have scheduled at this stage.
1 Like
As we can’t leave the Unity audio system enabled (because of building for xbox), we need a way to either play video audio through fmod, or alternatively sync seperate video (unity) and audio (fmod). What is currently possible and or the recommended way to do so? I can imagin this is a relatively common thing FMod Unity users are facing.
Thank you
I think your best bet is still the AudioSampleProvider I linked above. I’ll get someone from the team to write up an example of using it with FMOD.
2 Likes
Thank you, that would be amazing!
1 Like
Here is an example of how to use AudioSampleProvider to provide audio from a VideoPlayer to FMOD when Unity Audio is disabled:
ScriptUsageVideoPlayback.cs
//--------------------------------------------------------------------
//
// This is a Unity behaviour script that demonstrates how to capture
// audio data from a VideoPlayer using Unity's AudioSampleProvider and
// play it back through an FMOD.Sound. This example uses the
// VideoPlayer's APIOnly output mode and can be used to get audio from
// a video when UnityAudio is disabled.
//
// For documentation on writing audio data to an FMOD.Sound. See
// https://fmod.com/docs/2.02/api/core-api-sound.html#sound_lock
//
// This document assumes familiarity with Unity scripting. See
// https://unity3d.com/learn/tutorials/topics/scripting for resources
// on learning Unity scripting.
//
//--------------------------------------------------------------------
using Unity.Collections;
using UnityEngine;
using UnityEngine.Experimental.Audio;
using UnityEngine.Experimental.Video;
using UnityEngine.Video;
using System.Runtime.InteropServices;
using System.Collections.Generic;
using System;
public class ScriptUsageVideoPlayback : MonoBehaviour
{
private const int LATENCY_MS = 50;
private const int DRIFT_MS = 1;
private const float DRIFT_CORRECTION_PERCENTAGE = 0.5f;
private AudioSampleProvider mProvider;
private FMOD.CREATESOUNDEXINFO mExinfo;
private FMOD.Channel mChannel;
private FMOD.Sound mSound;
private List<float> mBuffer = new List<float>();
private int mSampleRate;
private uint mDriftThreshold;
private uint mTargetLatency;
private uint mAdjustedLatency;
private int mActualLatency;
private uint mTotalSamplesWritten;
private uint mMinimumSamplesWritten = uint.MaxValue;
private uint mLastReadPosition;
private uint mTotalSamplesRead;
private void Start()
{
VideoPlayer vp = GetComponent<VideoPlayer>();
vp.audioOutputMode = VideoAudioOutputMode.APIOnly;
vp.prepareCompleted += Prepared;
vp.loopPointReached += VideoEnded;
vp.Prepare();
mSampleRate = (int)(vp.GetAudioSampleRate(0) * vp.playbackSpeed);
mDriftThreshold = (uint)(mSampleRate * DRIFT_MS) / 1000;
mTargetLatency = (uint)(mSampleRate * LATENCY_MS) / 1000;
mAdjustedLatency = mTargetLatency;
mActualLatency = (int)mTargetLatency;
mExinfo.cbsize = Marshal.SizeOf(typeof(FMOD.CREATESOUNDEXINFO));
mExinfo.numchannels = vp.GetAudioChannelCount(0);
mExinfo.defaultfrequency = mSampleRate;
mExinfo.length = mTargetLatency * (uint)mExinfo.numchannels * sizeof(float);
mExinfo.format = FMOD.SOUND_FORMAT.PCMFLOAT;
FMODUnity.RuntimeManager.CoreSystem.createSound("", FMOD.MODE.LOOP_NORMAL | FMOD.MODE.OPENUSER, ref mExinfo, out mSound);
}
private void OnDestroy()
{
mChannel.stop();
mSound.release();
}
private void VideoEnded(VideoPlayer vp)
{
if (!vp.isLooping)
{
mChannel.setPaused(true);
}
}
private void Prepared(VideoPlayer vp)
{
mProvider = vp.GetAudioSampleProvider(0);
mProvider.sampleFramesAvailable += SampleFramesAvailable;
mProvider.enableSampleFramesAvailableEvents = true;
mProvider.freeSampleFrameCountLowThreshold = mProvider.maxSampleFrameCount - mTargetLatency;
vp.Play();
}
private void SampleFramesAvailable(AudioSampleProvider provider, uint sampleFrameCount)
{
using (NativeArray<float> buffer = new NativeArray<float>((int)sampleFrameCount * provider.channelCount, Allocator.Temp))
{
uint written = provider.ConsumeSampleFrames(buffer);
mBuffer.AddRange(buffer);
/*
* Drift compensation
* If we are behind our latency target, play a little faster
* If we are ahead of our latency target, play a little slower
*/
uint samplesWritten = (uint)buffer.Length;
mTotalSamplesWritten += samplesWritten;
if (samplesWritten != 0 && (samplesWritten < mMinimumSamplesWritten))
{
mMinimumSamplesWritten = samplesWritten;
mAdjustedLatency = Math.Max(samplesWritten, mTargetLatency);
}
int latency = (int)mTotalSamplesWritten - (int)mTotalSamplesRead;
mActualLatency = (int)((0.93f * mActualLatency) + (0.03f * latency));
int playbackRate = mSampleRate;
if (mActualLatency < (int)(mAdjustedLatency - mDriftThreshold))
{
playbackRate = mSampleRate - (int)(mSampleRate * (DRIFT_CORRECTION_PERCENTAGE / 100.0f));
}
else if (mActualLatency > (int)(mAdjustedLatency + mDriftThreshold))
{
playbackRate = mSampleRate + (int)(mSampleRate * (DRIFT_CORRECTION_PERCENTAGE / 100.0f));
}
mChannel.setFrequency(playbackRate);
}
}
private void Update()
{
/*
* Need to wait before playing to provide adequate space between read and write positions
*/
if (!mChannel.hasHandle() && mTotalSamplesWritten >= mAdjustedLatency)
{
FMOD.ChannelGroup mMasterChannelGroup;
FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out mMasterChannelGroup);
FMODUnity.RuntimeManager.CoreSystem.playSound(mSound, mMasterChannelGroup, false, out mChannel);
}
if (mBuffer.Count > 0 && mChannel.hasHandle())
{
uint readPosition;
mChannel.getPosition(out readPosition, FMOD.TIMEUNIT.PCMBYTES);
/*
* Account for wrapping
*/
uint bytesRead = readPosition - mLastReadPosition;
if (readPosition < mLastReadPosition)
{
bytesRead += mExinfo.length;
}
if (bytesRead > 0 && mBuffer.Count >= bytesRead)
{
/*
* Fill previously read data with fresh samples
*/
IntPtr ptr1, ptr2;
uint len1, len2;
var res = mSound.@lock(mLastReadPosition, bytesRead, out ptr1, out ptr2, out len1, out len2);
if (res != FMOD.RESULT.OK) Debug.LogError(res);
// Though exinfo.format is float, data retrieved from Sound::lock is in bytes, therefore we only copy (len1+len2)/sizeof(float) full float values across
int sampleLen1 = (int)(len1 / sizeof(float));
int sampleLen2 = (int)(len2 / sizeof(float));
int samplesRead = sampleLen1 + sampleLen2;
float[] tmpBuffer = new float[samplesRead];
mBuffer.CopyTo(0, tmpBuffer, 0, tmpBuffer.Length);
mBuffer.RemoveRange(0, tmpBuffer.Length);
if (len1 > 0)
{
Marshal.Copy(tmpBuffer, 0, ptr1, sampleLen1);
}
if (len2 > 0)
{
Marshal.Copy(tmpBuffer, sampleLen1, ptr2, sampleLen2);
}
res = mSound.unlock(ptr1, ptr2, len1, len2);
if (res != FMOD.RESULT.OK) Debug.LogError(res);
mLastReadPosition = readPosition;
mTotalSamplesRead += (uint)samplesRead;
}
}
}
}
`
Please let me know if you have any questions.
3 Likes