Hey all,
I’ve seen a couple topics on this before, so thus I’m aware that you fmod does not yet support pixelstreaming. Our team is going pretty much all in on pixelstreaming, so we need a solution for fmod sounds on pixelstreaming. That being said I don’t expect you guys to drop everything and write an integration for us, instead I’d just like some advice on where to start.
So thus, where should I start in order to get fmod to send sounds to the audiocapturer.cpp in pixelstreaming?
I’m not sure where fmod sends information to the audio component to tell it to play sounds but I believe that might be the proper place to tell pixelstreaming to bind to or vice versa. Also modifying pixelstreaming may not be the best option as pixelstreaming is an engine plugin and fmod is a project plugin.
Hi,
Unfotunately, I’m not familiar with the specifics of Pixel Streaming besides a cursory look at the AudioCapturer.cpp file and some familiarity with WebRTC, so I can’t be much help on the Pixel Streaming side of things. That said, what you’re looking to accomplish is probably best done by submitting the final mix of the server’s FMOD System to AudioCapturer’s buffer, which will likely require modification of AudioCapturer to expose the buffer, as well as code to access your FMOD System’s audio data.
To access the FMOD System’s mix, the cleanest way to do it would be to create a custom output plugin that sends the System’s output to AudioCapturer to be sent over the network. You can define an FMOD_OUTPUT_MIXER_CALLBACK, use FMOD_OUTPUT_READFROMMIXER_FUNC within it to generate and retrieve audio from the mixer. To make sure the FMOD System and Pixel Streaming sync, you can block within FMOD_OUTPUT_MIXER_CALLBACK until you need to execute a new mix. You would also need to modify FMODStudioModule.cpp to set the Core System’s output to your custom output plugin.
I would recommend taking a look at the “output_mp3” Core API example for a basic example of how you might set up a custom Output plugin, as well as the Output Plugin API Docs.
As an aside, I’ve also noted your interest in Pixel Streaming support on our internal feature/improvement tracker.
1 Like
Hi, i am running into the same issue as our team want to use pixel streaming with unreal and Fmod, any updates on this thread ?
We will have to investigate the options in the next weeks, would be good to know if your cpp workaround to link the Fmod output with the pixel streaming AudioCapturer has succeded or if we have to go back to Unreal native audio !
Unfortunately I have no updates to offer on our end for adding support for pixel streaming, but I’ve noted your interest internally.
Hello,
We’re in the same boat: We’re in the middle of multiple Pixel Streaming UE projects right now and the demand for audio solutions in the context of pixel-streamed applications for the Metaverse is increasing each month. Judging from our situation and talks we’re having with brands and agencies, you’d do well to prioritize support for Pixel Streaming.
In the meantime I’d love to know if anyone came up with a workaround?
Thanks for the expression of interest, I’ve noted it internally.