Recording from Stereo Mix and microphone simultaneously

#1

I’ve just implemented audio recording for the purpose of merging the Stereo Mix output with captured video to produce a complete movie of gameplay.

I create a new FMOD::Sound and then use FMOD::System::recordStart to write the output to it. Poll with getRecordPosition and encode to mp3 when there’s enough data.

We’d now like to also record the output of the microphone and merge that with the video and Stereo Mix, to support voice overs. The Stereo Mix volume would be reduced so the voice over could be clearly heard. This would be useful for making tutorials, as well as in-game support for Let’s Play videos.

When I call recordStart for the microphone driver id as well, it returns FMOD_ERR_RECORD. If I call it for the microphone without calling it for the stereo mix first, I can capture the mic fine.

Is this an absolute limitation of FMod that prevents recording of more than once device at a time?

I am working in Windows only at the moment, so the next step would be to attempt to use the Waveform-Audio recording interface instead. Are there any issues with using this along-side FMod?

#2

Are you trying to do this with a non-fmod game? If the game was written with fmod you wouldnt need to use recording of stereo mix, but just insert a capture DSP into the game’s DSP signal chain.

#3

No, it’s using FMod. Are you saying if I use a capture DSP I can record from the mic at the same time?

#4

Yes, you should avoid stereo mix if you want to have it run on more than 1 machine. it is purer (ie you get the direct, unmixed signal from the game) if you put a capture dsp at the head of the signal (ie System::getMasterChannelGroup, ChannelGroup::addDSP). Then all you should have is a single mic being used as normal.

#5

I know it’s not a direct answer to your question, but i’m really interested in how you managed to reliably capture Stereo Mix. I’m using WASAPI and capturing it is not so much a problem but playing it back in a FMOD::Sound while i’m capturing has proven to come with a ton of problems no matter the approach i take. I would love to hear how you did it.

#6

Hi Samuel,

It’s pretty much as I describe it above. I’m selecting the stereo mix driver, calling recordStart with that driver id, which sends data to the Sound object passed. I then poll the recording with getRecordPosition and use lock/unlock to extract the PCM data in my update loop. The Sound is not playing simultaneously, it’s just receiving the data.

The core of it I took from the output_mp3 sample. I just had to change where it got the data and how it decided to encode and write a new chunk. I made the Sound object a discrete number of chunk samples in length (20*2304 for now) and when getRecordPosition passes another 2304 sample boundary I encode and write that chunk.

Hope that helps.

#7

You can record simultaneously from multiple record devices, but not all record devices can record into the correct format (ie spdif in for example).
My own test allowed 2 out of 4 devices in my control panel to record at the same time (a webcam and sb audigy mic input)

Check the log when an error happens, ie I get AUDCLNT_E_DEVICE_IN_USE on one device.

#8

They record fine independently, so I don’t think it’s an unsupported format issue. I did try to use the Waveform-Audio mic recording and that behaved in the same way: one or the other worked, but not both at the same time.

#9

You asked if there was a limitation and the answer was no, there’s not, so multiple device recording is available. If there is an error please check the log (using fmodL.dll for example) and we can determine why there is an error. It is most likely going to be device or OS (or format) related. OUTPUT_DSOUND mode is an alternative to try in this case.

#10

This is what I get and I don’t know how to interpret it. It doesn’t seem to match any of the AUDCLNT_E_ codes, and doesn’t make sense as bit flags. Any idea?

[ERR] OutputWASAPI::recordStart : IAudioClient::Initialize returned 8007001F.

#11

Hi Tim,
The error is listed here https://msdn.microsoft.com/en-us/library/windows/desktop/dd319335(v=vs.85).aspx

ERROR_GEN_FAILURE
0x8007001F
May be used to indicate that the device has stopped responding (hung) or a general failure has occurred on the device. The device may need to be manually reset.

#12

Right, thanks. Sounds serious. I tried using setOutput(FMOD_OUTPUTTYPE_DSOUND). The behavior is the same but there is no error in the log.

Is it worth trying the DSP approach for stereo mix while using recordStart for the microphone?