# Audio Stream to programmer instrument
**URL:** https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519
**Category:** Unity
**Created:** [August 9, 2023, 1:26pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519 "2023-08-09T13:26:00Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@sailok](https://qa.fmod.com/u/sailok)
#### Post date: [August 9, 2023, 1:26pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/1 "2023-08-09T13:26:00Z")
Hi everyone,
I’m trying to implement an audio stream from a video in Unity following the example in [https://www.fmod.com/docs/2.02/unity/examples-video-playback.html](https://www.fmod.com/docs/2.02/unity/examples-video-playback.html)
So far, so good, except that now, I want to stream the audio directly into an instance of a programmer instrument in Fmod Studio to have more controls.
And that’s where I’m stuck.
I’ve already checked the following resource [Using FMOD with voice chat utilizing OnAudioFilterRead()](https://qa.fmod.com/t/using-fmod-with-voice-chat-utilizing-onaudiofilterread/17763)
but I think I’m not quite understanding how exactly you can grab the Fmod.sound and put it inside a programmer instrument.
Do I have to use a callback for this, or using only Sound.lock and Sound.unlock, is enough?
If anybody can explain step by step how it would work would be awesome.
Thank you!
---
### Author:  [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)
#### Post date: [August 10, 2023, 10:59pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/2 "2023-08-10T22:59:51Z")
Hi,
What version of the FMOD integration are you using? Could you please explain what controls you are looking for over the programmer instrument? It might be possible to avoid having to use a programmer instrument and control the sound directly.
---
### Author:  [@sailok](https://qa.fmod.com/u/sailok)
#### Post date: [August 11, 2023, 8:48am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/3 "2023-08-11T08:48:03Z")
Hi Connor,
I’m using Fmod Studio 2.02.13.
Initially, I wanted to spatialize the audio stream, and I’ve figured it out using the set3DAttributes on the channel Group.
But now, I need to duck another bus group when the audio stream is present, using a sidechain compressor.
And that’s where I think implementing the audio stream to a programmer instrument would be easier to manage for me, as I’m not sure if I would need to implement other features afterwards (like, adding reverb, or else).
---
### Author:  [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)
#### Post date: [August 15, 2023, 3:31am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/4 "2023-08-15T03:31:09Z")
Hi,
Thank you for the explanation. This can still be done on the sound rather than using a programmer instrument.
In the Video playback example, we play the sound on the `MasterChannelGroup`. Instead what we want to do is create a new group for the video audio:
The only difference is we will want to call `lockChannelGroup()` ([FMOD Engine | Studio API Reference - Bus::lockChannelGroup](https://fmod.com/docs/2.02/api/studio-api-bus.html#studio_bus_lockchannelgroup)) which will ensure that the channel group is kept in memory when we attach the sound. Here is how I retrieved the channel group:
```auto
FMODUnity.RuntimeManager.StudioSystem.getBus("bus:/VideoGroup", out FMOD.Studio.Bus videoGroupBus);
if (videoBus.isValid())
{
videoBus.lockChannelGroup();
FMODUnity.RuntimeManager.StudioSystem.flushCommands();
videoBus.getChannelGroup(out FMOD.ChannelGroup videoChannelGroup);
}
else
{
Debug.Log("Failed to lock Channel Group");
}
```
You will notice I also call `flushCommands()` ([FMOD Engine | Studio API Reference - System::flushCommands](https://fmod.com/docs/2.02/api/studio-api-system.html#studio_system_flushcommands)) which ensures that the lock command is called before the next update.
Just remember to attach your sound to the `videoChannelGroup` as we do to the `mMasterChannelGroup` and to `unlock` the channel group at some point.
Hope this helps!
---
### Author:  [@peacha](https://qa.fmod.com/u/peacha)
#### Post date: [November 20, 2025, 2:29pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/5 "2025-11-20T14:29:27Z")
Hi,
Did you ever find a solution to this? (using Programmer Instrument).
I’m in the same situation looking to pipe the audio output of a Unity video player into FMOD but I’ve had no luck with any other method so far. I can use the Video Playback example to get the video audio working, but without any manipulation abilities.
---
### Author:  [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)
#### Post date: [November 23, 2025, 9:48pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/6 "2025-11-23T21:48:49Z")
Hi,
Could I please grab your FMOD and Unity versions?
How did you go implementing the suggestion above ([Audio Stream to programmer instrument - #4 by Connor\_FMOD](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/4))? Did you encounter any issues?
---
### Author:  [@peacha](https://qa.fmod.com/u/peacha)
#### Post date: [November 24, 2025, 7:44am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/7 "2025-11-24T07:44:08Z")
Hi,
Unity 6.2.
FMOD 2.02.25
So I tried the suggestion above first and got some successes via the VideoGroup bus. It works well right until the point where I stop the VideoPlayer in Unity (or change clips) , and then the audio persistently bugs out after that (typically short sample repetition on Update, or full silent failure for subsequent video clips).
As a a hobbyist developer rather than a professional one I struggle to improvise the code in the absence of full examples, which is why I thought sending to programmer instrument might be more simplified or intuitive.
Thanks.
---
### Author:  [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)
#### Post date: [November 25, 2025, 3:45am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/8 "2025-11-25T03:45:00Z")
Thanks for the info, would it be possible to share the code you are using to test on my side?
---
### Author:  [@peacha](https://qa.fmod.com/u/peacha)
#### Post date: [November 27, 2025, 12:27pm UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/9 "2025-11-27T12:27:09Z")
Sure. Hope this helps.
```csharp
//--------------------------------------------------------------------
//
// This is a Unity behavior script that demonstrates how to capture
// audio data from a VideoPlayer using Unity's AudioSampleProvider and
// play it back through an FMOD.Sound. This example uses the
// VideoPlayer's APIOnly output mode and can be used to get audio from
// a video when UnityAudio is disabled.
//
// Steps to use:
// 1. Add a Unity VideoPlayer component to the same GameObject as this
// script.
// 2. Untick the VideoPlayer component's Play On Awake option.
// 3. Set the VideoPlayer component's Source to a VideoClip.
// 4. Set the VideoPlayer component's Renderer to a Mesh.
//
// More information on how to configure a Unity VideoPlayer component
// can be found here:
// https://docs.unity3d.com/Manual/class-VideoPlayer.html
//
// For documentation on writing audio data to an FMOD.Sound. See
// https://fmod.com/docs/2.02/api/core-api-sound.html#sound_lock
//
// This document assumes familiarity with Unity scripting. See
// https://unity3d.com/learn/tutorials/topics/scripting for resources
// on learning Unity scripting.
//
//--------------------------------------------------------------------
using Unity.Collections;
using UnityEngine;
using UnityEngine.Experimental.Audio;
using UnityEngine.Experimental.Video;
using UnityEngine.Video;
using System.Runtime.InteropServices;
using System.Collections.Generic;
using System;
#if UNITY_EDITOR
using UnityEditor;
#endif
public class ScriptUsageVideoPlayback : MonoBehaviour
{
private const int LATENCY_MS = 50; /* Some devices will require higher latency to avoid glitches */
private const int DRIFT_MS = 1;
private const float DRIFT_CORRECTION_PERCENTAGE = 0.5f;
private VideoPlayer mVideoPlayer;
private AudioSampleProvider mProvider;
private FMOD.CREATESOUNDEXINFO mExinfo;
private FMOD.Channel mChannel;
private FMOD.Sound mSound;
private List mBuffer = new List();
private int mSampleRate;
private uint mDriftThresholdSamples;
private uint mTargetLatencySamples;
private uint mAdjustedLatencySamples;
private int mActualLatencySamples;
private uint mTotalSamplesWritten;
private uint mMinimumSamplesWritten = uint.MaxValue;
private uint mTotalSamplesRead;
private uint mLastReadPositionBytes;
public bool playTest;
private void Start()
{
mVideoPlayer = GetComponent();
if (mVideoPlayer == null)
{
Debug.LogWarning("A VideoPlayer is required to use this script. " +
"See Unity Documentation on how to use VideoPlayer: " +
"https://docs.unity3d.com/Manual/class-VideoPlayer.html");
return;
}
mVideoPlayer.audioOutputMode = VideoAudioOutputMode.APIOnly;
mVideoPlayer.prepareCompleted += Prepared;
mVideoPlayer.loopPointReached += VideoEnded;
mVideoPlayer.Prepare();
#if UNITY_EDITOR
EditorApplication.pauseStateChanged += EditorStateChange;
#endif
}
#if UNITY_EDITOR
private void EditorStateChange(PauseState state)
{
if (mChannel.hasHandle())
{
mChannel.setPaused(state == PauseState.Paused);
}
}
#endif
private void OnDestroy()
{
//FMODUnity.RuntimeManager.StudioSystem.getBus("bus:/VideoGroup", out FMOD.Studio.Bus videoBus);
//videoBus.unlockChannelGroup();
mChannel.stop();
mSound.release();
#if UNITY_EDITOR
EditorApplication.pauseStateChanged -= EditorStateChange;
#endif
}
private void VideoEnded(VideoPlayer vp)
{
if (!vp.isLooping)
{
mChannel.setPaused(true);
}
}
private void Prepared(VideoPlayer vp)
{
mProvider = vp.GetAudioSampleProvider(0);
mSampleRate = (int)(mProvider.sampleRate * mVideoPlayer.playbackSpeed);
mDriftThresholdSamples = (uint)(mSampleRate * DRIFT_MS) / 1000;
mTargetLatencySamples = (uint)(mSampleRate * LATENCY_MS) / 1000;
mAdjustedLatencySamples = mTargetLatencySamples;
mActualLatencySamples = (int)mTargetLatencySamples;
mExinfo.cbsize = Marshal.SizeOf(typeof(FMOD.CREATESOUNDEXINFO));
mExinfo.numchannels = mProvider.channelCount;
mExinfo.defaultfrequency = mSampleRate;
mExinfo.length = mTargetLatencySamples * (uint)mExinfo.numchannels * sizeof(float);
mExinfo.format = FMOD.SOUND_FORMAT.PCMFLOAT;
FMODUnity.RuntimeManager.CoreSystem.createSound("", FMOD.MODE.LOOP_NORMAL | FMOD.MODE.OPENUSER, ref mExinfo, out mSound);
mProvider.sampleFramesAvailable += SampleFramesAvailable;
mProvider.enableSampleFramesAvailableEvents = true;
mProvider.freeSampleFrameCountLowThreshold = mProvider.maxSampleFrameCount - mTargetLatencySamples;
vp.Play();
playTest = true;
}
private void SampleFramesAvailable(AudioSampleProvider provider, uint sampleFrameCount)
{
using (NativeArray buffer = new NativeArray((int)sampleFrameCount * provider.channelCount, Allocator.Temp))
{
uint samplesWritten = provider.ConsumeSampleFrames(buffer);
mBuffer.AddRange(buffer);
/*
* Drift compensation
* If we are behind our latency target, play a little faster
* If we are ahead of our latency target, play a little slower
*/
mTotalSamplesWritten += samplesWritten;
if (samplesWritten != 0 && (samplesWritten < mMinimumSamplesWritten))
{
mMinimumSamplesWritten = samplesWritten;
mAdjustedLatencySamples = Math.Max(samplesWritten, mTargetLatencySamples);
}
int latency = (int)mTotalSamplesWritten - (int)mTotalSamplesRead;
mActualLatencySamples = (int)((0.93f * mActualLatencySamples) + (0.03f * latency));
int playbackRate = mSampleRate;
if (mActualLatencySamples < (int)(mAdjustedLatencySamples - mDriftThresholdSamples))
{
playbackRate = mSampleRate - (int)(mSampleRate * (DRIFT_CORRECTION_PERCENTAGE / 100.0f));
}
else if (mActualLatencySamples > (int)(mAdjustedLatencySamples + mDriftThresholdSamples))
{
playbackRate = mSampleRate + (int)(mSampleRate * (DRIFT_CORRECTION_PERCENTAGE / 100.0f));
}
mChannel.setFrequency(playbackRate);
}
}
void CheckKeyboardInput()
{
if (Input.GetKeyDown(KeyCode.F))
{
Debug.Log("F pressed!");
playTest = false;
//FMODUnity.RuntimeManager.StudioSystem.getBus("bus:/VideoGroup", out FMOD.Studio.Bus videoBus);
//videoBus.unlockChannelGroup();
//mChannel.stop();
//mSound.release();
mVideoPlayer.Stop();
}
if (Input.GetKeyDown(KeyCode.G))
{
Debug.Log("G pressed!");
playTest = true;
mVideoPlayer = GetComponent();
if (mVideoPlayer == null)
{
Debug.LogWarning("A VideoPlayer is required to use this script. " +
"See Unity Documentation on how to use VideoPlayer: " +
"https://docs.unity3d.com/Manual/class-VideoPlayer.html");
return;
}
mVideoPlayer.audioOutputMode = VideoAudioOutputMode.APIOnly;
mVideoPlayer.prepareCompleted += Prepared;
mVideoPlayer.loopPointReached += VideoEnded;
mVideoPlayer.Prepare();
#if UNITY_EDITOR
EditorApplication.pauseStateChanged += EditorStateChange;
#endif
}
}
private void Update()
{
CheckKeyboardInput();
if (!mVideoPlayer.isPrepared)
{
return;
}
/*
* Need to wait before playing to provide adequate space between read and write positions
*/
if (!mChannel.hasHandle() && mTotalSamplesWritten > mAdjustedLatencySamples)
{
if (playTest)
{
FMOD.ChannelGroup mMasterChannelGroup;
FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out mMasterChannelGroup);
// FMODUnity.RuntimeManager.CoreSystem.playSound(mSound, mMasterChannelGroup, false, out mChannel);
FMODUnity.RuntimeManager.StudioSystem.getBus("bus:/VideoGroup", out FMOD.Studio.Bus videoBus);
if (videoBus.isValid())
{
videoBus.lockChannelGroup();
FMODUnity.RuntimeManager.StudioSystem.flushCommands();
videoBus.getChannelGroup(out FMOD.ChannelGroup videoChannelGroup);
FMODUnity.RuntimeManager.CoreSystem.playSound(mSound, videoChannelGroup, false, out mChannel);
}
else
{
Debug.Log("Failed to lock Channel Group");
}
}
}
if (mBuffer.Count > 0 && mChannel.hasHandle())
{
uint readPositionBytes;
mChannel.getPosition(out readPositionBytes, FMOD.TIMEUNIT.PCMBYTES);
/*
* Account for wrapping
*/
uint bytesRead = readPositionBytes - mLastReadPositionBytes;
if (readPositionBytes < mLastReadPositionBytes)
{
bytesRead += mExinfo.length;
}
if (bytesRead > 0 && mBuffer.Count >= bytesRead)
{
/*
* Fill previously read data with fresh samples
*/
IntPtr ptr1, ptr2;
uint lenBytes1, lenBytes2;
var res = mSound.@lock(mLastReadPositionBytes, bytesRead, out ptr1, out ptr2, out lenBytes1, out lenBytes2);
if (res != FMOD.RESULT.OK) Debug.LogError(res);
/*
* Though exinfo.format is float, data retrieved from Sound::lock is in bytes,
* therefore we only copy (len1+len2)/sizeof(float) full float values across
*/
int lenFloats1 = (int)(lenBytes1 / sizeof(float));
int lenFloats2 = (int)(lenBytes2 / sizeof(float));
int totalFloatsRead = lenFloats1 + lenFloats2;
float[] tmpBufferFloats = new float[totalFloatsRead];
mBuffer.CopyTo(0, tmpBufferFloats, 0, tmpBufferFloats.Length);
mBuffer.RemoveRange(0, tmpBufferFloats.Length);
if (lenBytes1 > 0)
{
Marshal.Copy(tmpBufferFloats, 0, ptr1, lenFloats1);
}
if (lenBytes2 > 0)
{
Marshal.Copy(tmpBufferFloats, lenFloats1, ptr2, lenFloats2);
}
res = mSound.unlock(ptr1, ptr2, lenBytes1, lenBytes2);
if (res != FMOD.RESULT.OK) Debug.LogError(res);
mLastReadPositionBytes = readPositionBytes;
mTotalSamplesRead += (uint)(totalFloatsRead / mExinfo.numchannels);
}
}
}
}
```
---
### Author:  [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)
#### Post date: [December 2, 2025, 1:02am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/10 "2025-12-02T01:02:59Z")
Thank you for the code.
I believe the issue was trying to restart the video player without clearing and resetting some of our variables
```csharp
if (Input.GetKeyDown(KeyCode.G))
{
Debug.Log("G pressed!");
// Reset our provider
if (mProvider != null)
mProvider.sampleFramesAvailable -= SampleFramesAvailable;
// Remove subscriptions
mVideoPlayer.prepareCompleted -= Prepared;
mVideoPlayer.loopPointReached -= VideoEnded;
// Remove FMOD handles
if (mChannel.hasHandle()) mChannel.stop();
if (mSound.hasHandle()) mSound.release();
// Reset variables
mBuffer.Clear();
mSampleRate = 0;
mDriftThresholdSamples = 0;
mTargetLatencySamples = 0;
mAdjustedLatencySamples = 0;
mActualLatencySamples = 0;
mTotalSamplesWritten = 0;
mMinimumSamplesWritten = uint.MaxValue;
mTotalSamplesRead = 0;
mLastReadPositionBytes = 0;
// Restart the video player
mVideoPlayer.audioOutputMode = VideoAudioOutputMode.APIOnly;
mVideoPlayer.prepareCompleted += Prepared;
mVideoPlayer.loopPointReached += VideoEnded;
mVideoPlayer.Prepare();
playTest = true;
}
```
Hope this helps!
---
### Author:  [@peacha](https://qa.fmod.com/u/peacha)
#### Post date: [May 29, 2026, 11:16am UTC](https://qa.fmod.com/t/audio-stream-to-programmer-instrument/20519/11 "2026-05-29T11:16:07Z")
Thanks.
In the end we weren’t able to get this working unfortunately.
One way we are exploring is to separate video and audio files entirely, or to go back to Unity Audio.