Fmod + Unity + Lip Sync

Unity
1

Hi,

I recently picked up on fmod. I really like the way it works together with my application inside Unity. One thing is not really seamless though, and that is the lip sync. Since I want to drive the dialogues audio through fmod, but I want to use a lip sync solution inside Unity it’s getting tricky. All solutions I know are using the audio clip to drive the lip sync. Since it’s no longer in Unity, it’s not really working. One of these solutions (Salsa3D) is working if I feed the average of the amplitude of the dialogue audio over time. Yeah, it’s not the best quality lip sync, but it seems work.

I have found this “solution”: post on Unity forum.

My questions are:

  • do anyone have better option to do this? (more performant, more precise)
  • is it possible to do this some other way? Since I’m new to fmod, I may missed something.
  • question for the fmod devs: is it possible to change the Unity implementation’s private variables to protected at least in the component files? It’s not the best thing to change the implementation source code because it hinders any updates may occur over time.
2

If you are wanting to use the spectrum data from dialogue, then this is the only way to do it. One improvement to the example shown in the link is to reuse the array used to store the spectrum data, rather than creating a new one each update.

As for changing the StudioEventEmitter variables to protected, that sounds like something we can do in an upcoming release.