Problem with voice chat with core api (c++)

FMOD Engine
1

Hi,
I’ve been trying to create a voice chat using FMod Core api for the recording and playback of sound. I’ve managed to start the recording, extract the data into a char* (using lock/unlock), and playing the sound by creating a new one everytime.

The problem I’ve been having now though is that the audio, based on what i can hear, is:

  1. Playing slower, even though they have the same frequency.
  2. The audio always gets cut-off by the next clip.

Any help would be appreciated!

1 Like
2

Hi,

Could I please grab a code snippet and what version of FMOD Engine are you using?

The FMOD Engine install does come with a record example which may be useful: "C:\Program Files (x86)\FMOD SoundSystem\FMOD Studio API Windows\api\core\examples\record.cpp"

3

Hi,
Yes, absolutely.

Here is the extraction of the data:

FMOD::System* fSys = ClientChatManager::GetInstance()->GetVoiceChatFmodSystem();
FMOD_RESULT fRes;

unsigned int recordPos = 0;
fRes = fSys->getRecordPosition(myRecordIndex, &recordPos);

if (CheckFMODErr(fRes, "Recording Update Failed"))
        return { 0, nullptr };

unsigned int recordDelta = (recordPos >= lastRecordPos) ? (recordPos - lastRecordPos) : (recordPos + myRecordLength - lastRecordPos);

if (recordDelta == 0)
{
    return { 0, nullptr };
}

unsigned int len = 0;

myRecord->lock(lastRecordPos, min(recordDelta, MAX_AUDIO_SIZE), &currentData, nullptr, &len, 0);
myRecord->unlock(currentData, nullptr, len, 0);

lastRecordPos = recordPos;

char* dataPtr = (char*)currentData + '\0';

return { len, dataPtr };

And here i Reconstruct the sound:

FMOD_RESULT fRes;
FMOD::System* fSys = ClientChatManager::GetInstance()->GetVoiceChatFmodSystem();

if (mySound != nullptr)
    mySound->release();

FMOD_CREATESOUNDEXINFO exinfo = { 0 };
exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO);
exinfo.numchannels = (int)myChannelsCount;
exinfo.format = FMOD_SOUND_FORMAT_PCM16;
exinfo.defaultfrequency = myFreq;
exinfo.length = (myFreq * sizeof(short) * myChannelsCount);

//FMOD::Sound* outPutSound = NULL;
fRes = fSys->createSound(aData, FMOD_OPENMEMORY | FMOD_OPENRAW, &exinfo, &mySound);

if (CheckFMODErr(fRes, "Couldn't create sound (" + std::to_string(aLength) + ")"))
    return;

fRes = fSys->playSound(mySound, 0, false, &myChannel);

if (CheckFMODErr(fRes, "Couldn't create Channel"))
    return;

myChannel->setVolume(myVolume);

As for what version of the core API I am using, I dont really know. How would I check that?

4

Thanks for the code.

You can find the revision.txt file here:

"C:\Program Files (x86)\FMOD SoundSystem\FMOD Studio API Windows\doc\revision.txt"

Which has the version you are currently using.

From the record.cpp example outline how to manage drift compensation which may help with this issue:

if (channel)
{
    /*
        Stop playback if recording stops.
    */
    bool isRecording = false;
    result = system->isRecording(DEVICE_INDEX, &isRecording);
    if (result != FMOD_ERR_RECORD_DISCONNECTED)
    {
        ERRCHECK(result);
    }

    if (!isRecording)
    {
        result = channel->setPaused(true);
        ERRCHECK(result);
    }

    /*
        Determine how much has been played since we last checked.
    */
    unsigned int playPos = 0;
    result = channel->getPosition(&playPos, FMOD_TIMEUNIT_PCM);
    ERRCHECK(result);

    static unsigned int lastPlayPos = 0;
    unsigned int playDelta = (playPos >= lastPlayPos) ? (playPos - lastPlayPos) : (playPos + soundLength - lastPlayPos);
    lastPlayPos = playPos;
    samplesPlayed += playDelta;
            
    /*
        Compensate for any drift.
    */
    int latency = samplesRecorded - samplesPlayed;
    actualLatency = (int)((0.97f * actualLatency) + (0.03f * latency));

    int playbackRate = nativeRate;
    if (actualLatency < (int)(adjustedLatency - driftThreshold)) 
    {
        /* Play position is catching up to the record position, slow playback down by 2% */
        playbackRate = nativeRate - (nativeRate / 50); 
    }
    else if (actualLatency > (int)(adjustedLatency + driftThreshold))
    {
        /* Play position is falling behind the record position, speed playback up by 2% */
        playbackRate = nativeRate + (nativeRate / 50);
    }

    channel->setFrequency((float)playbackRate);
    ERRCHECK(result);
}

Would it be possible to elaborate on this issue?

5

Found, what I assume is, the version: build 144645

As for the borrowing the code from record.cpp, I’ve tried using it, with a slight rewrite, without any luck.

As for the cutting-out part, I can pretty much hear myself speaking relativelt clearly, but with gaps of missing data. For example if I were to say “Hello there” I might hear “Hello th-”.

Some people I have spoken with have suggested that the problem might be that the recording pauses when the locked function is called, is that a correct assumption?

6

Thanks for testing that.

Just to confirm, you are using FMOD for the recording and playing of the sound? If so, you shouldn’t have to lock and unlock the sound. In the example, you are recording directly to the sound that is being played.

Another user had some issues with stuttering here: Vivox to Unity OnAudioFilterRead to FMOD programmer sound. Stutters/crackling - #15 by dougmolina. It may have some useful information.

Potentially, it may also be that the buffer is not long enough:

exinfo.length = (myFreq * sizeof(short) * myChannelsCount);

result = system->getRecordDriverInfo(DEVICE_INDEX, NULL, 0, NULL, &nativeRate, NULL, &nativeChannels, NULL);
exinfo.length = nativeRate * sizeof(short) * nativeChannels; /* 1 second buffer, size here doesn't change latency */

Hope this helps!