Get sound data after dsp processing

Unity
1

hey, I am new in fmod.

I am quite confused how to use sound::lock to get the data from the sound after dsp processing. For my understanding, the dsp processing happens in the channel, so when I use sound::lock it just give the data for the original sound(before processing). So if there is any methods to put the processed sound in ‘channel’ back to the ‘sound’ parameter,so that I can use sound::lock to get the processed data and save them as a txt file.
here is my code:

result = system->createSound((const char*)buff, FMOD_OPENMEMORY | FMOD_OPENRAW, &exinfo, &sound);
ERRCHECK(result);
FMOD::DSP* dsp;
	
system->createDSPByType(FMOD_DSP_TYPE_PITCHSHIFT, &dsp);
dsp->setParameterFloat(FMOD_DSP_PITCHSHIFT_PITCH, 1.7);           //luoli
result = system->playSound(sound, 0, false, &channel);
ERRCHECK(result);
channel->addDSP(0, dsp);
system->update();
bool isplaying = true;
while (isplaying) {
	channel->isPlaying(&isplaying);
}
unsigned int numSamples = 0;
unsigned int mappedMemorySize = 0;
int bitsPerSample = 0;
sound->getFormat(NULL, NULL, NULL, &bitsPerSample);
sound->getLength(&numSamples, FMOD_TIMEUNIT_PCM);
sound->getLength(&mappedMemorySize, FMOD_TIMEUNIT_PCMBYTES);
assert(bitsPerSample == 16);
mappedMemorySize *= 2;
printf("Num samples: %i\tBits per sample: %i\n", numSamples, bitsPerSample);
int16_t* mappedMemory = new int16_t[mappedMemorySize];
unsigned int bytesRead = 0;
int16_t* mem = 0;
int16_t* output = mappedMemory;
FILE* fp;
ofstream outfile("test.txt", ios::trunc);
sound->lock(0, mappedMemorySize / 2, (void**)& mem, 0, &bytesRead, 0);


for (int i = 0; i < bytesRead; i++) {
	int16_t sample = mem[i];
	output[i] = ((int16_t)mem[i]);
	//printf("%d\n", output[i]);
	outfile << output[i] << "\n";

}
outfile.close();
sound->unlock(mem, 0, bytesRead, 0);
result = sound->release();
2

Using Sound::lock will get you access to the RAW audio data before any processing is done.
https://fmod.com/resources/documentation-api?version=2.1&page=core-api-sound.html#sound_lock

You will want to add a capture DSP to the channel after the last DSP you want to include.

3

hey cameron

Thank you for your reply. I have already finished using Sound::lock to access to the audio data. But it seems that the data is from the original sound. What I want is to get the PCM data from the processed sound by some DSP effect.

I try to add a custom DSP and I read the example ‘dsp_custom’, but I can not fully understand this callback function. Is the data in the ‘data->buffer’ PCM data？ I save these data in a text file and when I try to play sound from this file, it just sound like noise. I do some conversion for the type of the data because I need data in Int16.

PS: Is the function ‘getParameterData’ can be used to get the PCM data in this circumstance?

FMOD_RESULT F_CALLBACK myDSPCallback(FMOD_DSP_STATE* dsp_state, float* inbuffer, float* outbuffer, unsigned int length, int inchannels, int* outchannels)
{
	mydsp_data_t* data = (mydsp_data_t*)dsp_state->plugindata;
	//FILE* saveFile;
	ofstream savefile("G:\\app_zhanan\\data.txt", ios::app);
	int16_t buffer;

	for (unsigned int samp = 0; samp < length; samp++)
	{

		for (int chan = 0; chan < *outchannels; chan++)
		{

			data->buffer[(samp * *outchannels) + chan] = outbuffer[(samp * inchannels) + chan] = inbuffer[(samp * inchannels) + chan] * data->volume_linear;
			float num = data->buffer[(samp * *outchannels) + chan];
			int32_t temp = *(int*)& num;
			int16_t conversion = temp & ~(0xFFFF << 16);
			//int16_t conversion = (int16_t) num;
			//int16_t* conversion = reinterpret_cast<int16_t*>(&num);
			savefile << conversion << endl;
			//cout << *conversion << endl;


		}
	}
	data->channels = inchannels;
	savefile.close();
	return FMOD_OK;
}
4

Since we are in the Unity forum, I’m assuming you ultimately want to do this these in C# / Unity. I’ll put some code here to get you started, ultimately you’ll need to extend this for your desired goal. Eventually this code will be put into our documentation to help others.

using System;
using UnityEngine;
using System.Runtime.InteropServices;

class ScriptUsageCaptureDSP : MonoBehaviour
{
    FMOD.DSP mCaptureDSP;
    FMOD.DSP_READCALLBACK mReadCallback;
    float[] mDataBuffer;
    GCHandle mObjHandle;

    [AOT.MonoPInvokeCallback(typeof(FMOD.DSP_READCALLBACK))]
    static FMOD.RESULT CaptureDSPReadCallback(ref FMOD.DSP_STATE dsp_state, IntPtr inbuffer, IntPtr outbuffer, uint length, int inchannels, ref int outchannels)
    {
        FMOD.DSP_STATE_FUNCTIONS functions = Marshal.PtrToStructure<FMOD.DSP_STATE_FUNCTIONS>(dsp_state.functions);

        IntPtr userData;
        functions.getuserdata(ref dsp_state, out userData);

        GCHandle objHandle = GCHandle.FromIntPtr(userData);
        ScriptUsageCaptureDSP obj = objHandle.Target as ScriptUsageCaptureDSP;

        // Copy the incoming buffer to process later
        int lengthElements = (int)length * inchannels;
        Marshal.Copy(inbuffer, obj.mDataBuffer, 0, lengthElements);

        // Copy the inbuffer to the outbuffer so we can still hear it
        Marshal.Copy(obj.mDataBuffer, 0, outbuffer, lengthElements);

        return FMOD.RESULT.OK;
    }

    void Start()
    {
        // Assign the callback to a member variable to avoid garbage collection
        mReadCallback = CaptureDSPReadCallback;

        // Allocate a data buffer large enough for 8 channels with default mix buffer of 1024 samples, pin the memory to avoid garbage collection
        mDataBuffer = new float[1024 * 8];

        // Get a handle to this object to pass into the callback
        mObjHandle = GCHandle.Alloc(this);

        // Define a basic DSP that receives a callback each mix to capture audio
        FMOD.DSP_DESCRIPTION desc = new FMOD.DSP_DESCRIPTION();
        desc.numinputbuffers = 1;
        desc.numoutputbuffers = 1;
        desc.read = mReadCallback;
        desc.userdata = GCHandle.ToIntPtr(mObjHandle);

        // Create an instance of the capture DSP and attach it to the master channel group to capture all audio
        FMOD.ChannelGroup masterCG;
        FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out masterCG);
        FMODUnity.RuntimeManager.CoreSystem.createDSP(ref desc, out mCaptureDSP);
        masterCG.addDSP(0, mCaptureDSP);
    }

    void OnDestroy()
    {
        // Remove the capture DSP from the master channel group
        FMOD.ChannelGroup masterCG;
        FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out masterCG);
        masterCG.removeDSP(mCaptureDSP);

        // Release the DSP and free the object handle
        mCaptureDSP.release();
        mObjHandle.Free();
    }

    void Update()
    {
        // Do something with mDataBuffer here
    }
}

To answer your above questions, indeed Sound::lock is for accessing the audio of the audio asset, it doesn’t include DSP processing, this code does, it just depends on where you attach it. In the example I attach it to the master channel group.

As in your code snippet and mine, the incoming audio is in inbuffer, the outgoing audio is in outbuffer. It’s the job of the DSP to transfer audio from inbuffer to outbuffer, hence why I copy the data, otherwise it would be silent. The format of the data is PCM float, usually within the range of -1.0f to 1.0f, remember to multiply 1<<15 before converting to int32 and clamp before truncating to int16.