Confusion about System addDsp

FMOD Engine
1

Hi

In Fmod ex you had a command FMOD_System_AddDSP or System::AddDSP, but which has been removed in FMOD5. So what command should I now use instead?

Thanks in advance

Sincerely

Keitel

2

To get the same result in FMOD 5 query the master channel group from the System object and add the DSP to it instead.

2 Likes
3

Thank you for your answer. So I’ve put it like this in my program:

FMOD_DSP *mydsp;
FMOD_CHANNELGROUP *channelgroup;

FMOD_System_GetMasterChannelGroup(system, &channelgroup);

FMOD_ChannelGroup_AddDSP(channelgroup,0, mydsp);

I guess it’s correct. However, every time the program comes to the FMOD_ChannelGroup_AddDSP command, it stops or exits. Do you think you could give me any idea why this happens.

Sincerely

Keitel

4

Are you getting an error code back which is causing the exit? Or does the code crash?

5

The code crashes, so I don’t get any error code.

Keitel

6

Can you show us the code you are using to create your DSP object before passing into addDSP?

7

The code I am trying out is from the website of Dirac, combining Dirac with Fmod.

http://www.dspdimension.com/admin/using-dirac-with-fmod-to-change-pitch-and-speed-of-audio-in-real-time/

This code again is based on one of Fmod’s example programs, “dsp_custom." The code is, however, obviously made for Fmod ex, and I have tried to translate it, from the Dirac site, from C++ to C and to compatibility with Fmod5.

As known, Dirac can change pitch of a sound file without changing its speed, and change the speed without changing the pitch. However, it is not by itself able to simultaneously retain a perfect loop of a sound file. So, if I have understood it correctly, that is the purpose of this program, though I myself only understand parts of the code in detail.

While the original code in C++ can be found at the Dirac website, the code below is my edited version:

    /*===============================================================================================
 Dirac Time Stretching Example
 Copyright (c), Firelight Technologies Pty, Ltd 2004-2011.
 Copyright (c), The DSP Dimension 2012

 This example shows how to use Dirac to change pitch and speed of music independently from
 each other during playback by using FMOD's sound streaming capabilities.
 
 This demo inserts a custom DSP callback into the FMOD signal chain and pulls audio data from 
 the input sound through that callback for processing.

 Several other implementations are possible, but this appears to offer the best trade off 
 between code complexity and realtime performance.

 Note that using the DiracFx API would provide even better performance but it would mean
 sacrificing some audio quality in the process.

 Note also that unlike the FMOD stream engine we do not insert a resampler into the signal
 chain. If the file's sample rate does not match the system's sample rate your file will
 play at the wrong speed and sound out of tune. You can use Dirac to correct for this, but
 we left that part out of the code to keep it more readable.

===============================================================================================*/

	#include <conio.h>
	#include "fmod.h"
	#include "fmod_errors.h"
	#include <windows.h>
	#include <stdio.h>
	#include <math.h>

	#include "stdafx.h"
	typedef signed int        int32_t;
	//#define	SOUNDFILE_PATH		"./local_media/test-loop.wav"
	
	#define	SOUNDFILE_PATH		"tamp4.wav"

	#include "Dirac.h"

/* ****************************************************************************
	Set up some handy constants
 **************************************************************************** */
const float div8 = 0.0078125f;			//1./pow(2., 8.-1.);
const double div16 = 0.00003051757812;	//1./pow(2., 16.-1.);
const double div32 = 0.00000000046566;	//1./pow(2., 32.-1.);

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

/* ****************************************************************************
	This is the struct that holds state variables that our callback needs. In
	your program you will want to replace this by a pointer to "this" in order
	to access your instance methods and member variables
 **************************************************************************** */
typedef struct {
	unsigned int sReadPosition, sFileNumFrames;
	int sNumChannels, sNumBits;
	FMOD_SOUND	*sSound;
} userDataStruct;

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

void ERRCHECK(FMOD_RESULT result)
{
    if (result != FMOD_OK)
    {
        printf("FMOD error! (%d) %s\n", result, FMOD_ErrorString(result));
        exit(-1);
    }
}

//-----------------------------------------------------------------------------------------------------------------------------------------------------------

/* ****************************************************************************
	This converts the raw format from the decoded file to the float format that
	Dirac expects. 
 **************************************************************************** */
static void intToFloat(float *dest, void *src, long size, int wordlength)
{
	int32_t *v;
	long wordlengthInBytes = wordlength / 8;
	long numElementsInBuffer = size / wordlengthInBytes;
	long i;
	switch (wordlength) {
		case 8:
		{
			signed char *v = (signed char *)src;
			for ( i = 0; i < numElementsInBuffer; i++)	
				dest[i]=(float)v[i] * div8;
		}
			break;
		case 16:
		{
			signed short *v = (signed short *)src;	
			for ( i = 0; i < numElementsInBuffer; i++) {
				dest[i]=(float)v[i] * div16;
			}
		}
			break;
		case 24:
		{
			unsigned char *v = (unsigned char *)src;
			long c = 0;
			for ( i = 0; i < numElementsInBuffer; i++)	{
				int32_t value = 0;
				unsigned char *valuePtr = (unsigned char *)&value;

				valuePtr[0] = 0;
				valuePtr[1] = v[c]; c++;
				valuePtr[2] = v[c]; c++;
				valuePtr[3] = v[c]; c++;
				
				dest[i]=(double)value * div32;
			}
		}
			break;
		case 32:
		{
			printf("!!! 32bit files are not fully supported. Trying anyway...\n");
#if 0 /* this correctly plays 32 bit AIFF files but not WAV. Byte swapping bug in FMOD? */
			unsigned char *v = (unsigned char *)src;
			long c = 0;
			for (long i = 0; i < numElementsInBuffer; i++) {
				int32_t value = 0;
				unsigned char *valuePtr = (unsigned char *)&value;
				
				valuePtr[3] = v[c]; c++;
				valuePtr[2] = v[c]; c++;
				valuePtr[1] = v[c]; c++;
				valuePtr[0] = v[c]; c++;
				
				dest[i]=(double)value * div32;
			}
#else /* this correctly plays 32bit WAV files but not AIFF. Byte swapping bug in FMOD? */
			
			v = (int32_t *)src;	
			for ( i = 0; i < numElementsInBuffer; i++) {
				dest[i]=(double)v[i] * div32;
			}
#endif
			
		}
			break;
		default:
			break;
	}
}

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/* ****************************************************************************
	Reads a chunk of raw audio from the FMOD sound. If the read access was
	successful we convert the data to float
 **************************************************************************** */
int readFromSound(float *targetBuffer, unsigned int startFrame, unsigned int numFrames, FMOD_SOUND *sound, int numBits, int numChannels)
{
	void  *data1 = NULL;
	void  *data2 = NULL;
	unsigned int length1;
	unsigned int length2;
	
	int ret = -1;
	
	int framesToBytes = numChannels * numBits / 8;

	// lock the buffer
	
	FMOD_Sound_Lock(sound, startFrame * framesToBytes, numFrames * framesToBytes, &data1, &data2, &length1, &length2);
	
	if (data1)
		intToFloat(targetBuffer, data1, length1, numBits);

	// unlock the buffer once you're done
	FMOD_Sound_Unlock(sound, data1, data2, length1, length2);	
	return ret;
}

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

/* ****************************************************************************
	This is the callback function that supplies data from the input stream/file
	when needed by Dirac. The read requests are *always* consecutive, ie. the 
	routine will never have to supply data out of order.
	Should return the number of frames read, 0 when EOF, and -1 on error.
 **************************************************************************** */
long diracDataProviderCallback(float *chdata, long numFrames, void *userData)
{	
	unsigned int framesToReadBeforeEndOfFile = 0;
	userDataStruct *state;
	FMOD_SOUND *mySound;
	
	// The userData parameter can be used to pass information about the caller (for example, "this") to
	// the callback so it can manage its audio streams.
	
	if (!chdata)	return 0;

	state = (userDataStruct*)userData;
	if (!state)	return 0;

	mySound = state->sSound;
	
	
	// demonstrates how to loop our sound seamlessly when the file is done playing:
	// 
	// we have this many frames left before we hit EOF
	
	
	// if our current read position plus the required amount of frames takes us past the end of the file
	if (state->sReadPosition + numFrames > state->sFileNumFrames) {
		
		// we have this many frames left until EOF
		framesToReadBeforeEndOfFile = state->sFileNumFrames-state->sReadPosition;
		
		// read the remaining frames until EOF
		readFromSound(chdata, state->sReadPosition, framesToReadBeforeEndOfFile, mySound, state->sNumBits, state->sNumChannels);
		
		// rewind the file
		state->sReadPosition = 0;
		
		// remove the amount we just read from the amount that we actually want
		numFrames -= framesToReadBeforeEndOfFile;
	}
	
	// here we read the second part of the buffer (in case we hit EOF along the way), or just a chunk of audio from the file (in case we have not encountered EOF yet)
	readFromSound(chdata+framesToReadBeforeEndOfFile*state->sNumChannels, state->sReadPosition, numFrames, mySound, state->sNumBits, state->sNumChannels);
	state->sReadPosition += numFrames;
	
	return numFrames;	
	
}

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/* ****************************************************************************
	This is our custom dsp callback as implemented by the FMOD example
	dsp_custom. 
 **************************************************************************** */
FMOD_RESULT F_CALLBACK myDSPCallback(FMOD_DSP_STATE *dsp_state, float *inbuffer, float *outbuffer, unsigned int length, int inchannels, int outchannels) 
{ 
    unsigned int userdata;
    char name[256]; 
    FMOD_DSP *thisdsp = (FMOD_DSP *)dsp_state->instance;

	int ret = kDiracErrorNoErr;
	
    /* 
        This redundant call just shows using the instance parameter of FMOD_DSP_STATE and using it to 
        call a DSP information function. 
    */
    
	FMOD_DSP_GetInfo(thisdsp, name, 0, 0, 0, 0);
	
	//thisdsp->getInfo(name, 0, 0, 0, 0);

	// userData points to our Dirac instance
    
	FMOD_DSP_GetUserData(thisdsp, (void **)&userdata);

	//thisdsp->getUserData((void **)&userdata);
	
	if (!userdata)
		return FMOD_ERR_NOTREADY;
	
	ret = DiracProcessInterleaved(outbuffer, length, (void*)userdata);
	
	switch (ret) {
		case kDiracErrorDemoTimeoutReached:
			printf("!!! Dirac Evaluation has reached its demo timeout\n\tSwitching to BYPASS\n");
			
			FMOD_DSP_SetBypass(thisdsp, true);
			
			//thisdsp->setBypass(true);
			break;
		default:
			break;
	}
	
    return FMOD_OK; 
} 

//---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

int main(int argc, char *argv[])
{
    FMOD_SYSTEM       *system;
    FMOD_SOUND        *sound;
    FMOD_CHANNEL      *channel;
    FMOD_DSP          *mydsp;
    FMOD_RESULT        result;
    char               key = 'c';
    unsigned int        version;
	int                 sampleRate;

	void *dirac;
	
	float time;
	float pitch;
	
	static bool active;

	FMOD_DSP_DESCRIPTION  dspdesc; 

	FMOD_CHANNELGROUP *channelgroup;

	userDataStruct state;
	state.sReadPosition;
	state.sSound;

	

    /*
        Create a System object and initialize.
    */
    result = FMOD_System_Create(&system);
    ERRCHECK(result);

    result = FMOD_System_GetVersion(system, &version);
    ERRCHECK(result);

    if (version < FMOD_VERSION)
    {
        printf("Error!  You are using an old version of FMOD %08x.  This program requires %08x\n", version, FMOD_VERSION);
        return 0;
    }

    result = FMOD_System_Init(system, 32, FMOD_INIT_NORMAL, 0);
    ERRCHECK(result);
	
	// get the output (=processing) sample rate from the system
	// actually, we should use the file's sample rate, but I'm not sure if there is a call for that in FMOD
	result = FMOD_System_GetSoftwareFormat(system, &sampleRate, NULL, NULL);
    ERRCHECK(result);
	
    
	result = FMOD_System_CreateSound(system, SOUNDFILE_PATH, FMOD_LOOP_NORMAL, 0, &sound);
	
	ERRCHECK(result);

    printf("===============================================================================\n");
    printf("Dirac DSP example. Copyright (c) Firelight Technologies 2004-2011 and \n");
    printf("(c) 2012 The DSP Dimension, Stephan M. Bernsee \n");
    printf("===============================================================================\n");
    printf("Press 'f' to activate, deactivate Dirac\n");
    printf("Press 'Esc' to quit\n");
    printf("\n");

	
	/* ****************************************************************************
		DIRAC SETUP
		-----------
		We put all of Dirac's state variables that we need to access later in a
		struct. In a C++ program you will normally pass your instance pointer
		"this" as Dirac userData, but since this is not a class we cannot do 
		this here
	 **************************************************************************** */
	state.sReadPosition = 0;
	state.sSound = sound;
	
	// obtain the length of the sound in sample frames (needed to loop the sound properly)
	
	FMOD_Sound_GetLength(sound, &state.sFileNumFrames, FMOD_TIMEUNIT_PCM);
	
	//result = sound->getLength(&state.sFileNumFrames, FMOD_TIMEUNIT_PCM);
	ERRCHECK(result);

	// get the number of channels and number of bits. Needed to obtain the correct seek position in the file when using Sound::lock
	
	FMOD_Sound_GetFormat(sound, NULL, NULL, &state.sNumChannels, &state.sNumBits);
	
	ERRCHECK(result);
	
	// create our Dirac instance. We use the fastest possible setting for the Dirac core API here
	
	dirac = DiracCreateInterleaved(kDiracLambdaPreview, kDiracQualityPreview, state.sNumChannels, sampleRate, &diracDataProviderCallback, (void*)&state);
	if (!dirac) {
		printf("!! ERROR !!\n\n\tCould not create DIRAC instance\n\tCheck number of channels and sample rate!\n");
		exit(-1);
	}
	
	// Here we set our time stretch an pitch shift values
	time      = 1.0f;					// 125% length
	pitch     = 0.5f;		// pitch shift (0 semitones)
	
	// Pass the values to our DIRAC instance 	
	DiracSetProperty(kDiracPropertyTimeFactor, time, dirac);
	DiracSetProperty(kDiracPropertyPitchFactor, pitch, dirac);
	
	// Print our settings to the console
	DiracPrintSettings(dirac);
	
	printf("Running DIRAC version %s\nStarting processing\n", DiracVersion());
	
	// start playback
    
	result = FMOD_System_PlaySound(system, sound, NULL, false, &channel);
	    
	ERRCHECK(result);

    /*
        Create the DSP effects.
    */  
    
   { 
        FMOD_DSP_DESCRIPTION  dspdesc; 

        memset(&dspdesc, 0, sizeof(FMOD_DSP_DESCRIPTION)); 

        strcpy(dspdesc.name, "My first DSP unit"); 
     		
	dspdesc.numinputbuffers     = 0;                   // 0 = whatever comes in, else specify.   These two has to be checked ????
        	
	dspdesc.read         = &myDSPCallback; 
        dspdesc.userdata     = (void *)0x12345678; 

        result = FMOD_System_CreateDSP(system, &dspdesc, &mydsp); 
        ERRCHECK(result); 
    } 

    /*
        Inactive by default.
    */

	active = false;
    
	FMOD_DSP_SetBypass(mydsp, active);

	
    /*
        Main loop.
    */
   

   result = FMOD_System_GetMasterChannelGroup(system, &channelgroup);

   result = FMOD_ChannelGroup_AddDSP(channelgroup,0, mydsp);


   ERRCHECK(result); 


   if (result != FMOD_OK)
    {
        printf("FMOD error! (%d) %s\n", result, FMOD_ErrorString(result));
        exit(-1);
    }

    /*
        Main loop.
    */
    do
    {
        if (kbhit())
        {
            key = _getch();

            switch (key)
            {
                case 'f' : 
                case 'F' : 
                {
                    active = !active;
                    FMOD_DSP_SetBypass(mydsp, active);
					if (!active)	printf("\nProcessing with Dirac\n");
					else			printf("\nBYPASS ON\n");
                    break;
                }
            }
        }

        FMOD_System_Update(system);

        Sleep(10);

    } while (key != 'a');

    printf("\n");

    result = FMOD_Sound_Release(sound);
    ERRCHECK(result);

    result = FMOD_DSP_Release(mydsp);
    ERRCHECK(result);

    result = FMOD_System_Close(system);
    ERRCHECK(result);
    result = FMOD_System_Release(system);
    ERRCHECK(result);

	DiracDestroy(dirac);
	
    return 0;
}

Thanks in advance.

Keitel

8

What compiler are you using?

Can you try deleting the first “FMOD_DSP_DESCRIPTION dspdesc” and changing the second to “static FMOD_DSP_DESCRIPTION dspdesc”.

9

I am using Microsoft Visual C++ 2010 Express with Windows 7.

I tried your suggestion, but it didn’t change the situation.

Keitel

10

I am using Microsoft Visual C++ 2010 Express with Windows 7. I tried your suggestion, but it didn’t change the situation.

Keitel

11

This is your bug

dpdesc.userdata     = (void *)0x12345678;

followed by this in the callback

FMOD_DSP_GetUserData(thisdsp, (void **)&userdata);

and

ret = DiracProcessInterleaved(outbuffer, length, (void*)userdata);

you have not stored the direct handle, you have stored 0x12345678

There’s also an example in the Dirac folder that is specifically for FMOD. --DIRAC3LE–\Dirac3-Desktop\DiracFMOD and it uses the correct setup, ie

dspdesc.userdata = (void *)dirac;
1 Like
12

There are several other issues with this code.

  1. It calls playsound, though the whole point of the dirac test code is to use a custom DSP, and read from the sound using lock/unlock. The playSound can be removed.

  2. Next, fmod studio’s FMOD_DSP_DESCRIPTION doesnt have a numchannels field. This makes it default to 0 (or accept whatever comes in), so it is best to call dsp->setChannelFormat and use channels = the sound’s channel count.

  3. Dirac’s example is wrong, it uses the sample rate of the sound to pass to the dirac library during init, but the processing is done in DSP land which is always the sample rate of the system/mixer. FMOD::System::getSoftwareFormat should be used to get the rate.

13

note if you use the master channelgroup, it looks like there’s a bug calling setChannelFormat on the dsp head of the master channel group. I would create a secondary channelgroup, and call addDSP on that instead to get around this issue. I’m just working on fixing that one now.

14

regarding the last issue, calling FMOD_ChannelGroup_AddDSP(cg, 1, mydsp) instead of (cg, 0, mydsp) would also work around that issue.

15

Hi Brett,
Thank you very much for your help. It’s greatly appreciated. I don’t know how the command “dpdesc.userdata = (void *)0x12345678” came about, as it is as you mention not in the original code. But anyway, now the program works. I can change pitch and still maintain correct looping without changed speed, and I can change speed without changing the ptich. This is great. With this functionality added to Fmod I have all what I need for my sound programs. However, there is still one small issue I am a little puzzled about, though I might find out of it when I have processed all your information. I need to run two different sound files through this program, or function, one after the other, to be played in loops simultaneously in two different channels. Could you give me a hint what I need to do to handle this?

Keitel

16

the 0x12345678 probably came from fmod’s user dsp example. For 2 sounds at once, instead of using ChannelGroup::addDSP, use System::playDSP instead. That will put the user DSP on a channel instead of a group bus (which stomps on everything right now that may be coming in), and you can just call playSound for your second sound. You can play them exactly in sync as well, if you use the setDelay command (make both of them play in the future by a small amount, according to the channel’s parent dsp clock, obtained from Channel::getDSPClock).

17

Thank you very much for your answer. I have, however, detected one more issue that I hope you can help me to resolve. When I run the program, the pitch of the sound file is one semitone above normal - compared to when I for instance play it in Audacity. How can I fix this?

Thanks in advance.

Keitel

18

I think I already addressed that. “Dirac’s example is wrong, it uses the sample rate of the sound to pass to the dirac library during init, but the processing is done in DSP land which is always the sample rate of the system/mixer. FMOD::System::getSoftwareFormat should be used to get the rate.” 44khz vs 48khz is most likely your issue.

19

Sorry, but I don’t understand where the sample rate of the sound is passed to the dirac library. The FMOD_System_GetSoftwareFormat command is already in the program. I have tried to remove it, and set the samplerate variable to 44100, but nothing changes.

20

its right there in the code you pasted - dirac = DiracCreateInterleaved(kDiracLambdaPreview, kDiracQualityPreview, state.sNumChannels, sampleRate, &diracDataProviderCallback, (void*)&state); - the point is that the DSP output rate is probably 48000 and you’ve passed the sample rate of the sound to dirac, which is 44100. Use getSoftwareFormat instead to get the rate that the DSP engine uses, which is whta the DSP you’re using is processing at.