# General advice on user-friendly process for players to upload audio assets then pipe them through FMOD-powered Unity?

**URL:** <https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005>\
**Category:** Unity\
**Created:** [July 15, 2022, 7:32pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005 "2022-07-15T19:32:04Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![torley-gallium](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/torley-gallium/32/3688_2.png) [@torley-gallium](https://qa.fmod.com/u/torley-gallium)\
**Post date:** [July 15, 2022, 7:32pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/1 "2022-07-15T19:32:04Z")

</div>

I’m looking for Unity-specific ways of handling this, and any limitations that may be part of Unity.

Say there’s a game with UGC (User-Generated Content), where you can record and even upload sounds from local storage into the game, to play for others. For example, you have a sound of your cat meowing, and you import it into the virtual world so that other players can hear it.

In a case like this, the sound originates outside of an FMOD bank, isn’t loaded in at runtime, but FMOD is used to process and effect it — whether that’s spatialization or adding reverb.

How might this be handled?

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [July 19, 2022, 4:07am UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/2 "2022-07-19T04:07:19Z")

</div>

Hi,

An option you have is using Programmer Instrument, explained in the [Glossary](https://fmod.com/docs/2.02/studio/glossary.html#programmer-instrument). These allow for files outside of the FMOD and game project files to be used as audio sources.

You will need to include: `using System;` and `using System.Runtime.InteropServices;` at the top of your code file.

Create an `FMODUnity.EventReference` variable which will be used to instantiate your [`Programmer Instrument`](https://fmod.com/docs/2.02/studio/working-with-instruments.html#programmer-instruments) from the FMOD Project.

Then create an [`FMOD.Studio.EVENT_CALLBACK`](https://fmod.com/docs/2.02/api/studio-api-eventinstance.html#fmod_studio_event_callback) variable which will be used to create the `sound` to be played by FMOD.

In

```auto
void Start()
{
	yourPlayBackVariable = new FMOD.Studio.EVENT_CALLBACK(PlayFileCallBack);
}

```

This instantiates your `EVENT_CALLBACK` to be used in the `PlayAudioFile(string location)` function. Call this when the sound is ready to be played, how it gets the parameter `location` is up to you. Be sure that `location` is the full file path to the audio source.

> **PlayAudioFile(string location) Function**
>
> ```auto
> public void PlayAudioFile(string location)
> {
> var eventInstance = FMODUnity.RuntimeManager.CreateInstance(yourEventReference);
> 	
> // FMOD.RESULT is a very useful debugging tool to ensure that FMOD functions have run properly 
> FMOD.RESULT result; 
> if (yourEventInstance.isValid() == false)
> {
> Debug.Log("Failed to create instance, instance invalid");
> return;
> }
> // Lock the file location in memory to be used in the CALLBACK function to find the audio source
> GCHandl stringLocation = GCHandle.Alloc(location);
> // Assigning this data to the created instance 
> result = eventInstnace.setUserData(GCHandle.ToIntPtr(stringLocation));
> 	
> // Using an if statement like this after most FMOD function calls will help debug errors in code 
> if (result != FMOD.RESULT.OK)
> {
> Debug.Log("Failed to create evenInstance with a result of " + result);
> return;
> }
> 	
> 	
> // Check the result after each function call 
> result = eventInstance.setCallback(yourPlayBackVaraible); 
> result = eventInstance.start();
> // The sound has successfully been created and played 
> }
> 
> ```

This is your `FMOD.Studio.EVENT_CALLBACK` function used to create the sound for FMOD to play using the audio source file location.

> **CALLBACK Function**
>
> ```auto
> [AOT.MonoPInvokeCallback(typeof(FMOD.Studio.EVENT_CALLBACK))]
> static FMOD.RESULT YourCallBackFunction(FMOD.Studio.EVENT_CALLBACK_TYPE type, IntPtr instancePtr, IntPtr parameterPtr)
> {
> FMOD.Studio.EventInstance instance = new FMOD.Studio.EventInstance(instancePtr);
> 
> if (instance.isValid() == false)
> {
> UnityEngine.Debug.Log("Failed to create instance, instance invalid");
> return FMOD.RESULT.ERR_EVENT_NOTFOUND;
> }
> 
> //Retrive the user data 
> IntPtr stringPtr;
> instance.getUserData(out stringPtr);
> 
> // Retrieving the path of the audio source from the user data
> GCHandle stringHandle = GCHandle.FromIntPtr(stringPtr);
> String location = stringHandle.Target as String;
> 
> switch (type)
> {
> case FMOD.Studio.EVENT_CALLBACK_TYPE.CREATE_PROGRAMMER_SOUND:
> {
> FMOD.MODE soundMode = FMOD.MODE.LOOP_NORMAL | FMOD.MODE.CREATECOMPRESSEDSAMPLE | FMOD.MODE.NONBLOCKING;
> var parameter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
> 		
> FMOD.Sound audioSound;
> FMOD.RESULT result;
> 
> // Creating the sound using the location of the audio source 
> result = FMODUnity.RuntimeManager.CoreSystem.createSound(location, soundMode, out audioSound);
> 
> if (result == FMOD.RESULT.OK)
> {
> parameter.sound = audioSound.handle;
> parameter.subsoundIndex = -1;
> Marshal.StructureToPtr(parameter, parameterPtr, false);
> }
> break;
> }
> case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROY_PROGRAMMER_SOUND:
> {
> var paramerter = (FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES)Marshal.PtrToStructure(parameterPtr, typeof(FMOD.Studio.PROGRAMMER_SOUND_PROPERTIES));
> var sound = new FMOD.Sound(paramerter.sound);
> sound.release();
> 
> break;
> }
> case FMOD.Studio.EVENT_CALLBACK_TYPE.DESTROYED:
> {
> // Now the event has been destroyed, unpin the string memory so it can be garbaged collected
> stringHandle.Free();
> break;
> }
> }
> return FMOD.RESULT.OK;
> }
> 
> ```

Once this is all set up users will be able to pass in the location of audio sources which you can then play using FMOD.

A full script example of `Programmer Sounds` being used can be found under [Scripting Examples | Programmer Sounds](https://fmod.com/docs/2.02/unity/examples-programmer-sounds.html). Keep in mind, in the example it is using an `audio table` and not the file location of an audio source, but it may be a useful reference.

Hope this helps!

---

<div class="post-metadata">

**Author:** ![torley-gallium](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/torley-gallium/32/3688_2.png) [@torley-gallium](https://qa.fmod.com/u/torley-gallium)\
**Post date:** [July 20, 2022, 8:14pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/3 "2022-07-20T20:14:17Z")

</div>

@Connor_FMOD You are full of solutions! Thanks for laying this out.

I’ve been reading " [14.3 Loading DLC and UGC Banks](https://www.fmod.com/docs/2.00/studio/supporting-downloadable-and-user-generated-content.html#loading-dlc-and-ugc-banks)", and… are there general pros/cons between these two approaches? The two being:

(1) **Programmer Instruments**  
vs.  
(2) **Loading UGC banks**

It seems like having users do a “UGC Bank” would require more legwork and familiarity with FMOD, but potentially access to other adaptive parameters.

Whereas using Programmer Instruments would restrict the the signal flow to what we (as the game company) dictate in the FMOD Studio Project, but the tradeoff is it makes things easier for audio newcomers who upload audio into our system.

**Are there any other considerations to be aware of when deciding on an approach?**

---

<div class="post-metadata">

**Author:** ![torley-gallium](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/torley-gallium/32/3688_2.png) [@torley-gallium](https://qa.fmod.com/u/torley-gallium)\
**Post date:** [July 20, 2022, 10:37pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/4 "2022-07-20T22:37:52Z")

</div>

@Connor_FMOD And to add another related question about this and Programmer Instruments in particular, how is this use case supported by FMOD?

- [Unity - Manual: Microphone](https://docs.unity3d.com/Manual/class-Microphone.html) is used to live record a player speaking.
- We pass this along internally and store the audio file on a server. At which point it’s out of Unity’s hands.
- FMOD calls this audio file from the server and then it’s piped through an event with a Programmer Instrument.

Are there any known subtleties about how this works? Especially because it’s highly recommended that Unity built-in audio (aka old FMOD Core) is disabled ENTIRELY… but I presume Microphone class exists separately of that, so that functionality would still exist and there is no codependency?

In other words: **FMOD does not conflict with Unity audio input?**

---

<div class="post-metadata">

**Author:** ![torley-gallium](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/torley-gallium/32/3688_2.png) [@torley-gallium](https://qa.fmod.com/u/torley-gallium)\
**Post date:** [July 25, 2022, 4:32pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/5 "2022-07-25T16:32:25Z")

</div>

UPDATE to this is that we found that disabling Unity soundsystem also disables audio input via Microphone (!), so we need to find another way around. Is there a more straightforward guide to achieve that use case?

I’ve been googling and found this video which is one of the closer things I’ve found for start-to-finish documentation: [FMOD & Unity | Recording The Players Voice And Playing It Back At Runtime - YouTube](https://www.youtube.com/watch?v=Q-ivRHMdJ2c)

Are there any other resources we should be aware of? Whether provided by FMOD or user-created.

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [July 29, 2022, 6:15am UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/6 "2022-07-29T06:15:19Z")

</div>

Hi,

There isn’t much more to take into consideration other than what you have pointed out. Which method you end up using will solely depend on the functionality you want to give the user.

Scott’s video is a great resource to get started with, but there are some changes we will want to make ourselves.  
• Includes required

```auto
using FMODUnity;
using UnityEngine;
using System.Collections;
using System.Runtime.InteropServices;
using System.Collections.Generic;
using System.IO;

```

• Extra functions used

> **ERRCHECK()**
>
> ```auto
> /// <summary>
> /// Use to debug FMOD functions with explanation of what the function was doing with an both success and failed messages 
> /// </summary>
> /// <param name="result"></param>
> /// <param name="failed"></param>
> /// <param name="success"></param>
> private void ERRCHEK(FMOD.RESULT result, string failed, string success)
> {
> #if UNITY_EDITOR
> if (result.ToString().Length != 0)
> {
> if (result != FMOD.RESULT.OK)
> Debug.Log(failed + " " + result);
> else
> Debug.Log(success + " " + result);
> }
> #endif
> }
> 
> /// <summary>
> /// Override of no success message is required 
> /// </summary>
> /// <param name="result"></param>
> /// <param name="failed"></param>
> private void ERRCHEK(FMOD.RESULT result, string failed)
> {
> //Will only call debugs if in editor to save on calls in build 
> #if UNITY_EDITOR
> if (result.ToString().Length != 0)
> {
> if (result != FMOD.RESULT.OK)
> Debug.Log(failed + " " + result);
> }
> #endif
> }
> 
> ```

> **SetState()**
>
> ```auto
> /// <summary>
> /// Sets the state of the recording
> /// </summary>
> /// <param name="currentState">Options: Recording, Stopped, Restart</param>
> private void SetState(string currentState)
> {
> switch(currentState)
> {
> case "Recording":
> {
> recording = true;
> stopped = false;
> killed = false;
> #if UNITY_EDITOR
> Debug.Log("RECORDING");
> #endif
> break;
> }
> case "Stopped":
> {
> recording = false;
> stopped = true;
> killed = true;
> soundActive = false;
> #if UNITY_EDITOR
> Debug.Log("STOPPED");
> #endif
> break;
> }
> case "Restart":
> {
> recording = false;
> stopped = false;
> killed = false;
> soundActive = true; 
> #if UNITY_EDITOR
> Debug.Log("RESTARTING");
> 
> #endif
> break;
> }
> }
> }
> 
> ```

• You will need two systems [Core System](https://fmod.com/docs/2.02/api/studio-api-system.html#studio_system_create), one to record and one to write to a `.wav` file

> **Creating Systems**
>
> ```auto
> //System Variables
> private FMOD.System micRecordSystem, wavWriterSystem;
> private string micRecordData = "MICTEST";
> private string wavWiterData = "WAVWRITER";
> 
> //Creating systems to record and output mic input -------------------------------------------------------------
> GCHandle micRecordDataPtr = GCHandle.Alloc(micRecordData);
> GCHandle wavWriterDataPrt = GCHandle.Alloc(wavWiterData);
> 
> FMOD.RESULT result;
> FMOD.OUTPUTTYPE currentOutput;
> //Creating the mic system solely for recording mic input 
> result = FMOD.Factory.System_Create(out micRecordSystem);
> ERRCHEK(result, "Failed to create mic sysytem"); 
> //Initialize mic record system 
> resusystemcRecordSystem.init(100, FMOD.INITFLAGS.NORMAL, GCHandle.ToIntPtr(micRecordDataPtr));
> ERRCHEK(result, "Mic system int failed");
> //Checking current output for the system 
> result = micRecordSystem.getOutput(out currentOutput);
> ERRCHEK(result, "Failed to check output", "Mic system current ouput " + currentOutput.ToString() + " with result"); 
> 
> FMOD.RESULT wavResult; //Just for this snip
> //Creating the wav writer system 
> wavResult= FMOD.Factory.System_Create(out wavWriterSystem);
> ERRCHEK(wavResult, "Failed to create wav system"); 
> //Initialize the wav writer system 
> wavResult= wavWriterSystem.init(100, FMOD.INITFLAGS.NORMAL, GCHandle.ToIntPtr(wavWriterDataPrt));
> ERRCHEK(wavResult, "Failed to init wav write system"); 
> 		
> //Assigning output of Wav system to nosound, till we are ready to record sound
> wavResult= wavWriterSystem.setOutput(FMOD.OUTPUTTYPE.NOSOUND);
> wavWriterSystem.getOutput(out currentOutput); 
> ERRCHEK(wavResult, "Failed to change wav output", "Wav system current output " + currentOutput.ToString() + " with result ");
> //-------------------------------------------------------------------------------------------------------------
> 
> ```

• Because we are creating these systems ourselves, we are responsible for updating and releasing them  
• Call in `Update()`

> **Update systems**
>
> ```auto
> //Updating both the created systems
> if (!killed)
> {
> micRecordSystem.update();
> wavWriterSystem.update();
> }
> 
> ```

> **Release()**
>
> ```auto
> //REMEMBER TO RELEASE SYSTEMS SO THERE ARENT MEMORY LEAKS, this will get called if you stop the Editor running or if the object is destroyed
> private void OnDestroy()
> { 
> micRecordSystem.release();
> wavWriterSystem.release();
> }
> 
> ```

• We can follow Scott’s example of how to set up and retrieve the `recordingDevice` from our new `system`  
• When creating the new `sound` don’t use the `FMODUnity.RuntimeManger.CoreSystem` you will want to you’re your `recordingSystem`  
• When beginning the recording, input will be recorded on the `recordingSystem`, while the `wavWriterSystem` will only be responsible for outputting the file

> **Start Recording**
>
> ```auto
> //START RECORDING----------------------------------------------------------------------------------------------
> if (Input.GetKeyDown(KeyCode.R) && !recording && soundActive)
> {
> SetState("Recording");
> 
> FMOD.RESULT result;
> 
> result = micRecordSystem.recordStart(recordingDeviceIndex, sound, true);
> ERRCHEK(result, "Failed to start recording"); 
> 
> //Playing the sound to the mic record system 
> result = micRecordSystem.playSound(sound, channelGroup, false, out channel);
> ERRCHEK(result, "Failed to play sound on mic system");
> 
> //Playing the sound on the wav system 
> result = wavWriterSystem.playSound(sound, channelGroup, false, out wavChannel);
> ERRCHEK(result, "Failed to play sound on wav system");
> 			
> //Start wav system writing to file, make sure this only happens once 
> result = wavWriterSystem.setOutput(FMOD.OUTPUTTYPE.WAVWRITER);
> ERRCHEK(result, "Wav system failed to start writing to file", "Wav system now writing to file");
> }
> //-------------------------------------------------------------------------------------------------------------
> 
> ```

• When the recording is done, we will need to stop writing to the `.wav` file so it can be used as a source for our `Programmer Sound`.

> **Stop Recording**
>
> ```auto
> //STOP RECORDING-----------------------------------------------------------------------------------------------
> if (Input.GetKeyDown(KeyCode.S) && !stopped && recording)
> {
> SetState("Stopped");
> 
> //Stopping the recording
> FMOD.RESULT result = micRecordSystem.recordStop(recordingDeviceIndex);
> ERRCHEK(result, "Failed to stop recording on mic system");
> 
> //Should stop WASAPI starving------------------------------------------------------------------------------
> result = wavChannel.stop();
> ERRCHEK(result, "Failed to stop wav channel");
> 
> result = channel.stop();
> ERRCHEK(result, "Failed to stop mic channel"); 
> //---------------------------------------------------------------------------------------------------------
> result = sound.release();
> ERRCHEK(result, "Failed to release sound");
> 
> //Stop the wav system writing to file 
> result = wavWriterSystem.setOutput(FMOD.OUTPUTTYPE.NOSOUND);
> ERRCHEK(result, "Failed to change wav system output");
> }
> //-------------------------------------------------------------------------------------------------------------
> 
> ```

• If you want to do another recording, you will have to recreate the `sound` and delete the `.wav` file so it can’t be created again.

> **Recreate Sound**
>
> ```auto
> //RECREATE SOUND-----------------------------------------------------------------------------------------------
> if (Input.GetKeyDown(KeyCode.L) && killed && stopped)
> {
> SetState("Restart"); 
> 
> //Only need to recreate the sound 
> FMOD.RESULT result = micRecordSystem.createSound("Sound", FMOD.MODE.LOOP_NORMAL | FMOD.MODE.OPENUSER,
> ref exinfo, out sound);
> ERRCHEK(result, "Failed to recreate sound");
> 
> //Removing old wav so that the new one can be recorded 
> if (filePath.text.Length != 0)
> {
> File.Delete(filePath.text);
> #if UNITY_EDITOR
> Debug.Log("Deleted .wav file");
> #endif
> }
> }
> //-------------------------------------------------------------------------------------------------------------
> 
> ```

I’ll include the entire file. But it’s a good idea to try to set it up yourself so you understand how it works!  
[RecordMic.txt](https://qa.fmod.com/uploads/short-url/hT8oVFUl1Cn4inof6nWsd07zHDu.txt) (9.4 KB)

I have also included how to use programmer sounds  
[ProgrammerSounds.txt](https://qa.fmod.com/uploads/short-url/we0MRqtROcPPrsiKMSs7GfPcUMD.txt) (3.6 KB)

Hope this helps!

---

<div class="post-metadata">

**Author:** ![torley-gallium](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/torley-gallium/32/3688_2.png) [@torley-gallium](https://qa.fmod.com/u/torley-gallium)\
**Post date:** [July 29, 2022, 3:22pm UTC](https://qa.fmod.com/t/general-advice-on-user-friendly-process-for-players-to-upload-audio-assets-then-pipe-them-through-fmod-powered-unity/19005/7 "2022-07-29T15:22:03Z")

</div>

@Connor_FMOD Thank you for all the specific references! Appreciate your time with this. We’ll take a closer look.