How to input desktop audio so i can retrieve spectrum data from it WASAPI loopback capture

I want to input the left and right channels of desktop audio so i can retrieve seperate spectrum data of the lefft and right

I recommend you take a look at the record_enumeration example, which can be found in the FMOD Engine download Core API example. That will show you how to record from a WASAPI loopback device and into a Sound object. Getting the spectrum data is then a matter of attaching an FFT DSP to the Sound’s Channel. We have a Spectrum Analysis example of how to achieve this in Unity.

I downloaded and imported the .unitypackage. i am struggling to find the examples you mentioned.

thank you for your previous response

No problem here are the steps in more detail:

  1. Go to the FMOD Download page
  2. Click on the tab labelled “FMOD Engine”
  3. Download the FMOD Engine for Windows
  4. Run the executable to install the FMOD Engine
  5. Navigate to “C:\Program Files (x86)\FMOD SoundSystem\FMOD Studio API Windows”, this is the root directory of the FMOD Engine
  6. Navigate down to “\api\core\examples\vs2019” and open examples.sln inside Visual studio 2019
  7. Once that opens, right click on the record_enumeration project in the Solution Explorer and select Set as Startup Project
  8. F5 to run

The code in the record_enumeration example will show you how to capture and playback from a loopback device. It is written in C++, which if you are familiar with C# should be understandable- just ignore any * or & and imagine that :: and -> are . operators, e.g

//C++
struct RECORD_STATE
{
    FMOD::Sound *sound;
    FMOD::Channel *channel;
};

//C#
struct RECORD_STATE
{
    FMOD.Sound sound;
    FMOD.Channel channel;
};

//C++
result = system->createSound(0, FMOD_LOOP_NORMAL | FMOD_OPENUSER, &exinfo, &record[cursor].sound);

//C#
result = system.createSound(0, FMOD_LOOP_NORMAL | FMOD_OPENUSER, exinfo, record[cursor].sound);

Thank you so much, ill try it latere today

I put this off for a while

How should i implement this in unity?

The basic steps to record and play from a WASAPI loopback are to:

  1. Grab the rate and channel info of the loopback audio device you want audio from
  2. Create an FMOD Sound using the retrieved info
  3. Record from the loopback device into the FMOD Sound
  4. Play the FMOD Sound

A very basic implementation in Unity might look something like this:

void Start()
{
    // grab the rate and channel info of the loopback audio device
    int deviceIndex = 0; // arbitrary index, pick whichever one corresponds to the device you want
    int nativeRate = 0;
    int nativeChannels = 0;
    result = FMODUnity.RuntimeManager.CoreSystem.getRecordDriverInfo(deviceIndex, out _, 0, out _, out nativeRate, out _, out nativeChannels, out _);

    // create extended sound info object using retrieved rate and channel info
    FMOD.CREATESOUNDEXINFO exinfo = new FMOD.CREATESOUNDEXINFO
    {
        cbsize = MarshalHelper.SizeOf(typeof(FMOD.CREATESOUNDEXINFO)),
        numchannels = nativeChannels,
        format = FMOD.SOUND_FORMAT.PCM16,
        defaultfrequency = nativeRate,
        length = (uint)(nativeRate * sizeof(short) * nativeChannels) 
    };

    // create a new FMOD Sound using the extended sound info,
    // and start recording from the device into that sound
    FMOD.Sound sound;
    result = FMODUnity.RuntimeManager.CoreSystem.createSound("", FMOD.MODE.LOOP_NORMAL | FMOD.MODE.OPENUSER, ref exinfo, out sound);
    result = FMODUnity.RuntimeManager.CoreSystem.recordStart(deviceIndex, sound, true);

    // play the FMOD Sound (in this case, on the master channel group)
    FMOD.Channel channel;
    FMOD.ChannelGroup mcg;
    result = FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out mcg);
    result = FMODUnity.RuntimeManager.CoreSystem.playSound(sound, mcg, false, out channel);
}

In the Core API example previous mentioned by Jeff, the record_enumeration example demonstrates swapping between different audio devices, though in your case you may simply want to enumerate the devices and find the specific loopback device you want to use. You might also run into some audio crackling, which is caused by drifting between the drivers of the audio device and FMOD - the record example shows how to dynamically adjust the playback speed to compensate for the drift while keeping the same latency.

To get the spectrum data, you can attach an FFT DSP to the channel the sound is playing on, and retrieve the spectrum data from it. The Spectrum Analysis Scripting Example shows how to do this.

Hope that helps!