Hello! I am wondering if any of the FMOD products provide 3D audio spatialization and/or localization? If so, how do you consider echoes, reverberation, and attenuation? I’d really appreciate any details or links to specifics!
Thanks!
Scott
Thanks for the info Sean. We should be able to provide a migration process for the automation for your set up. I will have a look into it over this week. Would you mind sending me the fdp again? You can email it to thuan@fmod.org. Thanks.
Thuan
Thanks, Thuan! In case my email ends up in your spam folder, I just wanted to let you know that I sent over my FDP.
Sean
Great! Thanks, Gino and Thuan.
I’m using the Occlusion effect in Designer on the 3D Distance parameter, and I have the programmer set the flag that enables Low pass filtering for Occlusion. We haven’t been factoring geometry into occlusion; distance has been our only consideration so far.
Although we haven’t used reverb very much to date, I do set up the automation curve for the reverb aspect of the Occlusion effect (in case an area/level ends up needing a reverb).
Let me know if you need any other info. A month or so ago I sent Brett an fdp for the project in question, in case that’s useful for you to see.
Thanks again,
Sean
A belated thanks for all the info, Gino! One thing that’s holding us back from switching over from Designer to Studio is that (at last check) the importer would discard Occlusion automation. Is it feasible to have the importer translate Occlusion automation to FMOD Low-pass Simple, Volume and Reverb send automation (or whatever is most appropriate)? And have 3D Auto Distance Filtering translated to FMOD High-pass Simple (or whatever is most appropriate)? Having to re-implement this manually after importing seems like a pretty big job.
Thanks,
Sean
We should be able to sort this out… I’ll get back to you soon.
Cheers,
Gino
Hi Sean,
I just had a discussion with Gino regarding the Occlusion automation and 3D Auto Distance Filtering. We can definitely update the Designer importer to convert 3D Auto Distance Filtering (I will add it to my TODO list).
As for Occlusion automation, just wondering what sort of setup are you guys using for your project? Are you setting the occlusion and reverb using the programmer API, or are you using the FMOD geometry and reverb sphere, or the Occlusion effect in Designer (if so, which parameter are you automating it on, Distance parameter or a game parmeter), or a combination of them all?
Thuan
1 Like
Hi Scott,
This is a big topic so I’ll try to summarize how FMOD goes about spatialization before we dive into any details. Each product, FMOD Ex, FMOD Designer and FMOD Studio have various in-the-box solutions to spatialization but can all be augmented with the use of custom DSPs or plugins.
FMOD Ex spatializes 3D sounds by applying amplitude panning, distance attenuation and sending to a reverb unit who’s parameters are blended between different settings based on the listener’s location. FMOD Ex also has an in-built low-pass that is employed for sounds occluded by geometry and can be set up additionally as a very basic HRTF filter. This low-pass and another built-in high-pass can automatically band-pass the signal based on distance as well. Other panning options include blending between the automatic 3D panning and fixed speaker levels and controlling the spread of multi-channel 3D sounds.
FMOD Designer utilises FMOD Ex’s functionality but also provides automation of any effect parameter or internal parameter based on distance and angle. Plugin automation allows for custom spatialization algorithms.
FMOD Studio has a new 3D panner which has an additional section controlling the spread of the sound based on distance for sound-size simulation. The algorithm has also been improved to spatialize multi-channel audio for use in creating 3D ambiences from multichannel field recordings. In FMOD Ex/Designer, panning happens on each individual audio asset whereas Studio has a panner effect which sits on the output (sub-mix) of the event. Reverb is routed via reverb sends which can be placed pre- or post-panning and automated based on 3D position. There can be as many sends as you like sending to different reverbs, delays or any other effect chains in the main mixer. Mixer Snapshots allow effect settings to be blended or different simultaneous reverbs to be mixed based on 3D position. In Studio, 3D audio is as much about mixing as realistic localisation.
FMOD Studio’s panner effect module can be swapped out with custom spatialization plugins which will also listen to the position of the event. Plugins can therefore be used to perform HRTFs, geometry-based early reflections, distance filtering, etc… There are many features being introduced into the plugin architecture to allow game studios and third-party developers to easily implement their own advanced spatialization algorithms into their projects.
We’re happy to discuss any of those things in detail so please let us know what interests you.
Cheers,
Gino
Thanks for the write-up, Gino. I’ve been interested to know how things like Studio’s equivalent of the ‘Occlusion’ effect was going to work. Sorry if this is a bit of a thread hijack:
My understanding is that the LP/HP filtering in FMOD Ex/Designer (via Occlusion) is very light on CPU when compared to adding an LP and HP filter to every 3D event. If so, is there a similar effect coming for Studio, or are the filter’s in Studio light enough on CPU that they can be added to the master out of every 3D event?
As for reverb, it sounds like reverb presets have been replaced with creating a distinct reverb unit for each desired preset. Assuming that’s the case, is the bypassing of effects part of mixer snapshots (since I assume reverbs are fairly CPU heavy)? Also, is there already a system in place for reverb ‘emitters’ (i.e. as the listener moves into the reverb emitter’s sphere, a specific reverb activates and fades-in as the listener gets closer to the center of the sphere)? Is this handled via mixer snap shots too?
Lastly, if you’re trying to gauge interest in 3rd party plugins, I’d say that if the licenses for those cost on the order of several thousand dollars per platform, then it may be a tough sell for studios like ours. We tend to develop $10-15 PSN/XBLA/Steam games, and it can be tough enough to convince the budget gatekeepers of the merit of spending money on new VST plugins, updates to Cubase/Wavelab or better hardware. But maybe these plugins aren’t geared towards lower budget games.
Thanks,
Sean
Hi Sean,
The occlusion effect in FMOD Ex/Designer uses the built-in low-pass effect which lives on each channel. The distance filter uses the same low-pass and an additional high-pass filter on each channel. Studio does not use the built-in effects on each channel, but those effects can be found in the ‘Legacy’ sub-menu as FMOD Low-pass Simple and FMOD High-pass Simple effects. While we haven’t added occlusion support to Studio yet, one way we are planning to support it is to have an Occlusion parameter with which you can automate filters. This parameter will be automatically calculated from game geometry, as it already is, but you’ll be able to create your own occlusion effects from any of the effects available in Studio. This could also lead to setting up different filter automations for different materials.
In most cases, using a Low-pass Simple effect on the master track will be equivalent, if not lighter on CPU then Designer’s way. This is because both built-in and added effects are bypassed if their cut-offs are at max (or min for high-pass); so in Studio, the built-in low-pass will always be bypassed. Possibly lighter because there will only ever be one low-pass applied per event instance rather than one on each channel if you had more than one sound playing simultaneously in your event.
Reverb presets will be done via snapshots in Studio and you would only need the one reverb effect on your mixer’s reverb return bus unless you wanted to mix between multiple simultaneous reverbs on different return buses! Yes, reverbs are heavy on CPU so best keep simultaneously-running reverbs to a minimum if you do plan to mix between reverb units. By saving your reverb settings in a snapshot, you can either simply trigger or automate them inside events based on game logic or you could automate the snapshot’s intensity over distance to achieve a similar result to reverb spheres in FMOD Ex/Designer. This will work somewhat with the current snapshot system but will be fully functional once blending snapshots are introduced later this year.
The plug-in architecture is as much about making it easier to implement and design with custom in-house DSPs as it is about providing for 3rd-party plug-ins. An example of an in-house system would be an early-reflections effect which uses the geometry from the game engine, or in-house sound synthesis or dialogue tech. As for third-party plug-ins, I can’t comment on the pricing or models that companies will decide on over time, or about what will be available for low-budget games, but I’m sure there will be companies looking to make affordable products for smaller developers. The focus on plug-ins is really about deeper and easier integration of external audio tech and Studio’s modular nature is well suited for this.
Cheers,
Gino
Hi Sean,
Just a quick update that the new patch release 1.1.11, should handle importing of Occlusion dsps and 3D auto distance filtering for the events.
Thuan
Excellent! Thanks a lot, Thuan.
Sean
Hi Gino, I just stumbled upon this thread.
I would like to check the following with you.
[1] to play 3D ambiences in FMOD, simply we should import 7.1.4 files with the ambience and play it back through a 3D Event. Would it be necessary to add a spatialiser effect or should it work without it?
[2] i would like to play sounds tin 3D and make them sound within the created ambience. How would you go about it in FMOD and then in Unity? I created ambience in 7.1.4 and played it back in Unity from a Game object. I used other 3D sounds using spatialiser and attached to other objects in the 3D Scene. The result seem to be that the 7.1.4 is all over the world, the individual objects with spatialisers effect seem to play within the ambience. Is this the correct setup?
please let me know
thanks
A “3D event” is one whose behavior changes based on the event’s position in 3D space relative to the listener.
A spatializer effect pans and attenuates a signal based on the event emitter’s position relative to the listener in 3D space, and is thus one of several possible ways in which an aven can be made 3D.
A 7.1.4 audio asset in a 2D event will play through all 12 speakers. If you want the event to be panned and attenuated based on the emitter’s position relative to the listener in 3D space, you should add a spatializer effect. As adding a spatializer effect to an event makes that event 3D by definition, this will make the event 3D.
I’m afraid I don’t understand what you mean by this. Could you please describe in detail what behavior you want to achieve?
Hi Joseph, thanks for your reply.
So imagine you have the 3D field recording of a field of grass in the park.
I want that to play in the game as to surround the whole game. So I used a 7.1.4 file of that recording. THere is no spatialiser on that event, so I assume is treated as 2D, although the channels output is 7.1.4
then i would like to place other synthesised sounds as 3D events (with spatialiser) within the whole area of the game, that in my head is covered by the previous field recording 7.1.4 file.
My question is: is that actually happnening? Would the field recording be a sound in 3D heard as surrounding the whole area into which other sounds could appear, provided the proximity to the listener?
it feels like that, but I am unsure if it is actually correct in terms of signal flow.
I appreciate your help
kind regards
I’m sorry, but I’m afraid I don’t understand what you mean by these questions.
The field recording would be audible from any point within the virtual 3D environment, and it would not change based on the position of the listener in 3D space. The other events in the space would also be audible when they were close to the listener, and would be panned and attenuated based on their positions relative to the listener.
By FMOD Studio’s definitions of 2D and 3D events, the field recording would be a 2D event and the other events would be 3D events.