Scheduling multiple midi channel in one frame

FMOD Engine
,
1

I’m upgrading a project from FMOD 1.x ages and run into some issues while scheduling multiple midi channels in one frame.

The midi channel is created like:

FMOD::Channel * c = 0;
FMOD_RESULT r = sys_->playSound( sample->sampleSound, nullptr, true, &c );
ERRCHECK( r );
r = c->setDelay( startTime.Hi, startTime.Lo, false );
c->setPause(false);

It looks if I schedule 2 channels (created from the same source sound) in one frame the later one will overwrite the previous even though there is definitely enough channel left in the system. And this is not resolved by calling fmodSys->update() in between the scheduling calls. Wondering if this is a known change in the API and what changes do I need to get them both making sound in 2.02?

cheers,
seph

1 Like
2

After some digging I’ve sound some weird behaviours…

So I have two midi panels, when users move them around the midi notes are scheduled based on the position, and they are scheduled in the same frame but have slightly different parameters passed to channel->setDelay function.

The weird behaviours are:

  1. if I spawn the channels generated by system->playSound all in the master group, then only the second panel notes make sound (later scheduled);
  2. if I assign the channels to a ChannelGroup per panel, then only the first panel notes make sound (first scheduled);
  3. if I assign each sound instance a separate channel, only the first scheduled one on that sound will be heard;
  4. if I duplicate the sound instance the behaviour is the same.

I’ve setup the system to have 256 software channel and 1024 virtual channels, sound are loaded with CREATE_SAMPLE mode, in exinfo I’ve forced it to only using software codec, all channels are non-virtual, its just some of them are not making sound. Is there anything obvious I’m missing?

3

Hi,
A midi defaults to FMOD_OPENSTREAM. It doesnt try to decode the whole song into memory as PCM, that would possibly take hundreds of megabytes of ram. Streaming rules are that 1 sound uses 1 channel only.

https://www.fmod.com/resources/documentation-api?version=2.02&page=glossary.html#stream

You can decode the whole midi into memory with FMOD_CREATESAMPLE, or you can create 2 streams (open the file twice) and play each one seperately on its own channel.

Then you can use setDelay to schedule the 2 channels together.

The behaviour you were seeing was that the 1st playsound was immediately being stopped and retriggered when you called playsound the 2nd time.

4

Also in the fmod installer, under tools there is a tool called FMOD Profiler.exe.
To go with this you need to initialize your app with FMOD_INIT_PROFILE_METER_ALL.

This can be illuminating to the reason some things are silent, you can tick the levels checkbox to see what is getting audio and what isnt, and if you’ve set up your channelgroup heirarchy correctly.

5

Hi Brett,

Thanks for the comment although I’m not very sure if that is the issue.

By midi I actually mean we have a map where each sound clip is mapped with a midi value, and when playing back the frequency is changed to the desired midi note.

The sound clip were created with the following code:

FMOD_CREATESOUNDEXINFO  exinfo      = {0};
memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
exinfo.cbsize           = sizeof(FMOD_CREATESOUNDEXINFO);
exinfo.audioqueuepolicy = FMOD_AUDIOQUEUE_CODECPOLICY_SOFTWAREONLY;
        
FMOD::Sound * snd;
FMOD_RESULT result  = sys_->createSound(filePath, FMOD_CREATESAMPLE | FMOD_2D | FMOD_ACCURATETIME, &exinfo, &snd);
ERRCHECK(result);

And this is the pitch shifting code:

float channelFrequency = 0;
FMOD_RESULT r = channel->getFrequency(&channelFrequency);
ERRCHECK(r);
channel->setFrequency( powf(2.0f,pitchShift/12.0f) * channelFrequency );

I’ve already tried commenting out the frequency shift bit and allocating multiple sound clips but they doesn’t seem to solve the issue. Do you think there might be something else (system setting etc) that could result in this behaviour?

cheers,
seph

6

I’ve also tried loading the sound as stream (using CREATE_STREAM), and I can clearly hear that newer sound cut off the old one sharply. But it is different behaviour compared to use the CREATE_SAMPLE tag. Is there any other possibility you could think of? I think assigning a ChannelGroup or not actually alters the outcome is very strange… but I can’t find where lead to this behaviour.

thanks,
seph

7

After some digging finally found out where the issues are but don’t understand why it is.

So we have a DSP attached the master channel group, does not alter the sound but just to pull a precise DSP Clock from. It is created with the following code:

    FMOD_DSP_DESCRIPTION dspdesc; 
    memset(&dspdesc, 0, sizeof(FMOD_DSP_DESCRIPTION)); 
    
    strcpy(dspdesc.name, "Clock Sync"); 
    dspdesc.read                        = clockSyncDSPCallback;
    dspdesc.version                   = 0x00010000;
    dspdesc.userdata                 = (void *)0x12345678;
    dspdesc.numinputbuffers     = 1; // 0 to generate sound, 1 to process sound
    dspdesc.numoutputbuffers   = 1;
    
    FMOD_RESULT result = pSys->createDSP(&dspdesc, &dspClockSync_); 
    ERRCHECK(result); 

    FMOD::ChannelGroup *channelgroup_master;
    FMOD::DSP* dspHead;
    result = pSys->getMasterChannelGroup(&channelgroup_master);
    result = channelgroup_master->getDSP(FMOD_CHANNELCONTROL_DSP_HEAD, &dspHead);
    ERRCHECK(result );
    result = dspHead->addInput( dspClockSync_, &syncConnection_ );
    dspClockSync_->setActive( true );

And the callback function is just passing the data to out buffer, and grab position from a bgm channel:

    for (unsigned int samp = 0; samp < length; samp++)
       for (int chan = 0; chan < *outchannels; chan++)
           outbuffer[(samp * *outchannels) + chan] = inbuffer[(samp * inchannels) + chan];
    
    unsigned int channelPos = 0;
    syncChannel_->getPosition( &channelPos, FMOD_TIMEUNIT_MS )
    return FMOD_OK;

And then on each note channel, we’ve added a simple envelope DSP to do the fade out. It is created as below:

    FMOD_DSP_DESCRIPTION dspdesc;
    memset(&dspdesc, 0, sizeof(FMOD_DSP_DESCRIPTION));
    
    strcpy(dspdesc.name, "Volume Envelope");
    dspdesc.read                         = volEnvelopeDSPCallback;
    dspdesc.userdata                  = (void *)(envelopeSetIndex);
    dspdesc.version                    = 0x00010000;
    dspdesc.numinputbuffers      = 1; // 0 to generate sound, 1 to process sound
    dspdesc.numoutputbuffers    = 1;
    FMOD_RESULT result = pSys->createDSP(&dspdesc, &dsp_);

these envelope DSP are added to the generated channels per each note, and just does some standard processing.

The weird behaviour is - if I omit the data copying code in the clock DSP, then half the note will be making sound (as described in the previous comments); if I have the clock DSP copying buffer, then no sound is generated; if I remove the envelope DSPs, then all notes start to make sound.

Is there anything wrong in the way I’m creating the DSPs?

  • seph
8

Hi Seph,
It sounds just like the content of your envelope DSP is causing the problem, there’s no issue with your DSP creation.
FMOD provides clock accurate ways to do volume points, with https://www.fmod.com/resources/documentation-api?version=2.02&page=core-api-channelcontrol.html#channelcontrol_addfadepoint

Also I am not sure why you put a call to Channel::getPosition in a DSP callback, that is not necessary and will not improve any accuracy.

All DSP timing should be done with getDSPClock, setDelay, addFadePoint if you want a truly accurate (to the sample) solution, this is the way FMOD Studio does all of its timing.

If your current logic is mostly done though, you just need to check for a bug in the values passed to your envelope DSP. Its possible it is not intepreting channels correctly as it processes the data, or the envelope point logic (that needs to span multiple mixes) is incorrect.

regards,
Brett