By default, FMOD for Unity will automatically load all banks when playing, or when the editor enters play mode. You can disable this, or only load specific banks, from the Initialization section of the plugin’s settings menu at FMOD → Edit Settings → Initialization.
If this is disabled, you will indeed have to load the desired banks at runtime - the plugin comes with the Studio Bank Loader component to do this, but you can also load them yourself from your own code using RuntimeManager.LoadBank().