Hey all,
I’ve seen a couple topics on this before, so thus I’m aware that you fmod does not yet support pixelstreaming. Our team is going pretty much all in on pixelstreaming, so we need a solution for fmod sounds on pixelstreaming. That being said I don’t expect you guys to drop everything and write an integration for us, instead I’d just like some advice on where to start.
So thus, where should I start in order to get fmod to send sounds to the audiocapturer.cpp in pixelstreaming?
I’m not sure where fmod sends information to the audio component to tell it to play sounds but I believe that might be the proper place to tell pixelstreaming to bind to or vice versa. Also modifying pixelstreaming may not be the best option as pixelstreaming is an engine plugin and fmod is a project plugin.
Hi,
Unfotunately, I’m not familiar with the specifics of Pixel Streaming besides a cursory look at the AudioCapturer.cpp file and some familiarity with WebRTC, so I can’t be much help on the Pixel Streaming side of things. That said, what you’re looking to accomplish is probably best done by submitting the final mix of the server’s FMOD System to AudioCapturer’s buffer, which will likely require modification of AudioCapturer to expose the buffer, as well as code to access your FMOD System’s audio data.
To access the FMOD System’s mix, the cleanest way to do it would be to create a custom output plugin that sends the System’s output to AudioCapturer to be sent over the network. You can define an FMOD_OUTPUT_MIXER_CALLBACK, use FMOD_OUTPUT_READFROMMIXER_FUNC within it to generate and retrieve audio from the mixer. To make sure the FMOD System and Pixel Streaming sync, you can block within FMOD_OUTPUT_MIXER_CALLBACK until you need to execute a new mix. You would also need to modify FMODStudioModule.cpp to set the Core System’s output to your custom output plugin.
I would recommend taking a look at the “output_mp3” Core API example for a basic example of how you might set up a custom Output plugin, as well as the Output Plugin API Docs.
As an aside, I’ve also noted your interest in Pixel Streaming support on our internal feature/improvement tracker.