Unity - Get the audio wave of an event while it is playing


I am using the lipsync asset Salsa in Unity which relies on Unity audio sources, unfortunately it still doesn’t support FMOD, however it is possible to feed Salsa with the real time value of the wave form in order to calculate the lipsync.

How can I have the frame to frame (meaning in the Update function) waveform value (I presume from 0.0 to 1.0) of an event that is playing.

Please note my events are 3D events (since they are played on characters positioned in the space within a VR scene) therefore the waveform values should be identical as the ones from the audio clip without any attenuation or filter modifications.

Many thanks in advance for your help!

Any help on this, any suggestion would be much appreciated.

You can use a DSP to get the FFT/spectrum data, we have an example for this: https://www.fmod.com/resources/documentation-unity?version=2.2&page=examples-spectrum-analysis.html

Many thanks for pointing me in the right direction Cameron.
However I am having an issue with the channel groups of the bus.

I’ve reduced the window size to 1 (since I only need a value between 0 and 1 for the lypsinc everyframe). I have created a mixer group and assign the voice over event to it (just a mono file narration).
If I run the code chosing the “bus:/” (which I assume is the master bus) everything works well, however if I chose the “bus:/VO” (which is the one I assign the event) the GetChannelGroup function does not return OK.
I am completely lost here - any help would be much appreciated.

Just in case, I’ve checked the FMOD.RESULT from the GetChannelGroup function and it returns FMOD_ERR_STUDIO_NOT_LOADED.

I am loading the FMOD banks via Asset Bundles and all the audio events play properly so far, is there anything extra I need to do to load the bus channel groups? or this issue is something completely different?

Many thanks again

It is likely that the bus has not been loaded yet, because nothing is playing through it at that point, but you can use Studio::Bus::lockChannelGroup to force FMOD to create the bus and keep it around until you call Studio::Bus::unlockChannelGroup.

Thanks a million! Just calling GetChannelGroup after playig the event instance did the trick.
Working now :slight_smile: