# Seeking micro stuttering with FFMpeg

**URL:** <https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724>\
**Category:** FMOD Engine\
**Tags:** cpp\
**Created:** [January 11, 2023, 12:50pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724 "2023-01-11T12:50:01Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 11, 2023, 12:50pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/1 "2023-01-11T12:50:01Z")

</div>

Hi, I’m new to FMod so this question may be dumb.

the sound is created in mode FMOD\_OPENUSER | FMOD\_LOOP\_NORMAL | FMOD\_CREATESTREAM, and use a callback to fill data, just like the tutorial on dranger but replace the counterpart of SDL. It works fine but when changing video position, the sound will repeat previous buffer data until new data is decoded, which causes stuttering.  
If I set the decode buffer bigger the stuttering is gone but this breaks the sync, as the sync time stamp is updated by audio packet. workaround is using system clock instead of audio clock, but I want to know the root reason.

I’ll post any code snippets if need.

thanks in advance.

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [January 12, 2023, 5:46am UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/2 "2023-01-12T05:46:41Z")

</div>

Hi,

Could you link the tutorial you are referring to?

What version of FMOD are you using?

---

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 12, 2023, 4:06pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/3 "2023-01-12T16:06:30Z")

</div>

Thanks for the reply

Tutorial: [ffmpeg tutorial](http://www.dranger.com/ffmpeg/tutorial03.html)  
FMod: 2.02

**code snippet:**

**create sound:**

```
// sound name is used to identify sounds, so here it just the pointer address cast to long
inline void FMI_CreateSound(std::wstring&& soundName, void *userdata
    , std::function<void(FMOD_CREATESOUNDEXINFO& exinfo)>callBack) {
    memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
    exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO); /* Required. */

    callBack(exinfo);
    exinfo.userdata = userdata;

    FMOD::Sound* sound;

    result = system->createSound(0, mode, &exinfo, &sound);
    if (result == FMOD_OK) {
        this->soundMap.emplace(soundName, SoundMapItem{sound});
    }
}

```

**creater:**

```
this->cFMI.FMI_CreateSound(std::forward<std::wstring>(_itos((long)ppFFMpeg)), ppFFMpeg, [] (FMOD_CREATESOUNDEXINFO& exinfo) {
			exinfo.numchannels = TARGET_CHANNEL_NUMBER;
			// match the output of swr
			exinfo.defaultfrequency = TARGET_SAMPLE_RATE;
			// if size = SDL_AUDIO_BUFFER_SIZE, will micro shutter when seeking
			// if size >> SDL_AUDIO_BUFFER_SIZE will cause lag due if sync with audio timer
			exinfo.decodebuffersize = SDL_AUDIO_BUFFER_SIZE;
			exinfo.length = SDL_AUDIO_BUFFER_SIZE;
			exinfo.format = FMOD_SOUND_FORMAT_PCM16;              
			exinfo.pcmreadcallback =                              
				[](FMOD_SOUND* sound, void* data, unsigned int datalen) {
				auto pSound = (FMOD::Sound*)sound;

				void* lockPtr_1 = nullptr;
				void* lockPtr_2 = nullptr;

				size_t prtLength_1 = 0;
				size_t prtLength_2 = 0;

				pSound->lock(0, datalen
					, &lockPtr_1, &lockPtr_2
					, &prtLength_1, &prtLength_2);

				void* userdata = nullptr;
				pSound->getUserData((void**)&userdata);

				AudioCallback(userdata, (Uint8*)data, datalen);

				pSound->unlock(lockPtr_1, lockPtr_2
					, prtLength_1, prtLength_2);

				return FMOD_OK;
			};
			exinfo.pcmsetposcallback = nullptr; /* User callback for seeking. */
			});

```

audio callback:

```
static void AudioCallback(void* userdata, Uint8* stream, int len) {
	// No mutex needed here as audio is paused when deleting pFFMpeg
	auto CallBackCore = [&](FFMpeg** ppFFMpeg, Setter setter, Mixer mixer) {
		if (ppFFMpeg == nullptr) {
			setter(stream, 0, len);

			return;
		}

		auto pFFMpeg = *ppFFMpeg;

		if (pFFMpeg == nullptr) {
			setter(stream, 0, len);

			return;
		}

		pFFMpeg->audio_fillData(stream, len, setter, mixer);

		return;
	};
	
	auto ppFFMpeg = (FFMpeg**)userdata;
	CallBackCore(ppFFMpeg, memset, [](void* dst, const void* src, size_t len, int volume) {
		memcpy(dst, src, len);
		});
};

```

pFFMpeg-\>audio\_fillData(stream, len, setter, mixer) is alomst the same of tutorial but add mutex and asbtract the mixer and memory setter to compatible with both SDL & FMod

---

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 12, 2023, 4:11pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/4 "2023-01-12T16:11:53Z")

</div>

Tutorial: [ffmpeg tutorial](http://www.dranger.com/ffmpeg/tutorial03.html)  
FMod: 2.02

Code Snippets:

Create sound:

```plaintext
    inline void FMI_CreateSound(std::wstring&& soundName, void *userdata
        , std::function<void(FMOD_CREATESOUNDEXINFO& exinfo)>callBack) {
        memset(&exinfo, 0, sizeof(FMOD_CREATESOUNDEXINFO));
        exinfo.cbsize = sizeof(FMOD_CREATESOUNDEXINFO); /* Required. */

        callBack(exinfo);
        exinfo.userdata = userdata;

        FMOD::Sound* sound;

        result = system->createSound(0, mode, &exinfo, &sound);
        if (result == FMOD_OK) {
            this->soundMap.emplace(soundName, SoundMapItem{sound});
        }
    }

```

Actual Creater:

```plaintext
            this->cFMI.FMI_CreateSound(std::forward<std::wstring>(_itos((long)ppFFMpeg)), ppFFMpeg, [](FMOD_CREATESOUNDEXINFO& exinfo) {
				exinfo.numchannels = TARGET_CHANNEL_NUMBER;
				// match SWR output
                exinfo.defaultfrequency = TARGET_SAMPLE_RATE;
				
				// if size = SDL_AUDIO_BUFFER_SIZE, will micro shutter when seeking
				// if size >> SDL_AUDIO_BUFFER_SIZE will cause lag due if sync with audio timer
				exinfo.decodebuffersize = SDL_AUDIO_BUFFER_SIZE;
				exinfo.length = SDL_AUDIO_BUFFER_SIZE;
				exinfo.format = FMOD_SOUND_FORMAT_PCM16;              
				exinfo.pcmreadcallback =                              
					[](FMOD_SOUND* sound, void* data, unsigned int datalen) {
					auto pSound = (FMOD::Sound*)sound;

					void* lockPtr_1 = nullptr;
					void* lockPtr_2 = nullptr;

					size_t prtLength_1 = 0;
					size_t prtLength_2 = 0;

					pSound->lock(0, datalen
						, &lockPtr_1, &lockPtr_2
						, &prtLength_1, &prtLength_2);

					void* userdata = nullptr;
					pSound->getUserData((void**)&userdata);

					AudioCallback(userdata, (Uint8*)data, datalen);

					pSound->unlock(lockPtr_1, lockPtr_2
						, prtLength_1, prtLength_2);

					return FMOD_OK;
				};
				exinfo.pcmsetposcallback = nullptr; /* User callback for seeking. */
				});
			this->cFMI.FMI_PlaySound(std::forward<std::wstring>(_itos((long)ppFFMpeg)), false);

```

Callback:

```plaintext
	static void AudioCallback(void* userdata, Uint8* stream, int len) {
		// No mutex needed here as audio is paused when deleting pFFMpeg
		auto CallBackCore = [&](FFMpeg** ppFFMpeg, Setter setter, Mixer mixer) {
			if (ppFFMpeg == nullptr) {
				setter(stream, 0, len);

				return;
			}

			auto pFFMpeg = *ppFFMpeg;

			if (pFFMpeg == nullptr) {
				setter(stream, 0, len);

				return;
			}

			pFFMpeg->audio_fillData(stream, len, setter, mixer);

			return;
		};

		auto ppFFMpeg = (FFMpeg**)userdata;
		CallBackCore(ppFFMpeg, memset, [](void* dst, const void* src, size_t len, int volume) {
			memcpy(dst, src, len);
			});
	};

```

`pFFMpeg->audio_fillData(stream, len, setter, mixer)` is almost the same as tutorial but added mutex and callback to compatible with both SDL & FMod

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [January 17, 2023, 11:47pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/5 "2023-01-17T23:47:29Z")

</div>

Hi,

Thanks for the code snippets and link.

The root reason for the stuttering is the [ring buffer](https://soundprogramming.net/programming/creating-a-ring-buffer/) used by FMOD to output sound is not getting any new data so it will continue to play what is still in the buffer, as it is very small it sounds very harsh.

> [@defisym](#):
>
> decode buffer bigger

While this seems to solve the issue it only delays the time till you experience a stutter.

We recently created a Unity example using FMOD as the audio output which can be found here: [Is Unity's built-in audio engine FMOD Studio (FMOD 5)? - #10 by jeff\_fmod](https://qa.fmod.com/t/is-unitys-built-in-audio-engine-fmod-studio-fmod-5/18911/10).

Hope this helps!

---

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 19, 2023, 2:27pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/6 "2023-01-19T14:27:18Z")

</div>

Thanks for the reply.  
So is there a way to get the current playback progress in the buffer? I can use this to estimate the audio clock offset for sync. Tried following method but the whole program hangs when step to `Channel::getPosition`

```plaintext
    inline void FMI_SetPos(std::wstring&& soundName, size_t pos = 0) {
        FMI_SetSound(std::forward<std::wstring>(soundName), [&](IT it)->void {
#ifdef _DEBUG
            size_t position = 0;
            result = it->second.channel->getPosition(&position, FMOD_TIMEUNIT_MS);
#endif // _DEBUG            
            result = it->second.channel->setPosition(pos, FMOD_TIMEUNIT_MS);
            });
    }

```

`FMI_SetSound` just find if the given sound name exists then returns it’s iterator, and it points to this struct, which records sound & channel info.

```plaintext
    struct SoundMapItem {   
        FMOD::Sound* pSound;
        FMOD::Channel* channel = 0;       
    };    

```

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [January 23, 2023, 11:09pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/7 "2023-01-23T23:09:12Z")

</div>

Hi,

In the Unity example above the `bytesRead` in the update function will give you the current position of the buffer. This will depend on how you are reading through the buffer if it is the same for you.

> **Update**
>
> ```auto
> private void Update()
> {
> /*
> * Need to wait before playing to provide adequate space between read and write positions
> */
> if (!mChannel.hasHandle() && mTotalSamplesWritten >= mAdjustedLatency)
> {
> FMOD.ChannelGroup mMasterChannelGroup;
> FMODUnity.RuntimeManager.CoreSystem.getMasterChannelGroup(out mMasterChannelGroup);
> FMODUnity.RuntimeManager.CoreSystem.playSound(mSound, mMasterChannelGroup, false, out mChannel);
> }
> 
> if (mBuffer.Count > 0 && mChannel.hasHandle())
> {
> uint readPosition;
> mChannel.getPosition(out readPosition, FMOD.TIMEUNIT.PCMBYTES);
> 
> /*
> * Account for wrapping
> */
> uint bytesRead = readPosition - mLastReadPosition; // <- current position
> if (readPosition < mLastReadPosition)
> {
> bytesRead += mExinfo.length;
> }
> 
> if (bytesRead > 0 && mBuffer.Count >= bytesRead)
> {
> /*
> * Fill previously read data with fresh samples
> */
> IntPtr ptr1, ptr2;
> uint len1, len2;
> var res = mSound.@lock(mLastReadPosition, bytesRead, out ptr1, out ptr2, out len1, out len2);
> if (res != FMOD.RESULT.OK) Debug.LogError(res);
> 
> // Though exinfo.format is float, data retrieved from Sound::lock is in bytes, therefore we only copy (len1+len2)/sizeof(float) full float values across
> int sampleLen1 = (int)(len1 / sizeof(float));
> int sampleLen2 = (int)(len2 / sizeof(float));
> int samplesRead = sampleLen1 + sampleLen2;
> float[] tmpBuffer = new float[samplesRead];
> 
> mBuffer.CopyTo(0, tmpBuffer, 0, tmpBuffer.Length);
> mBuffer.RemoveRange(0, tmpBuffer.Length);
> 
> if (len1 > 0)
> {
> Marshal.Copy(tmpBuffer, 0, ptr1, sampleLen1);
> }
> if (len2 > 0)
> {
> Marshal.Copy(tmpBuffer, sampleLen1, ptr2, sampleLen2);
> }
> 
> res = mSound.unlock(ptr1, ptr2, len1, len2);
> if (res != FMOD.RESULT.OK) Debug.LogError(res);
> mLastReadPosition = readPosition;
> mTotalSamplesRead += (uint)samplesRead;
> }
> }
> }
> 
> ```

Is it possible to break-all during the hang and try to find where the code is hanging?  
Or is it possible to get a call-stack dump to see if its hanging in our code?

---

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 27, 2023, 1:24pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/8 "2023-01-27T13:24:46Z")

</div>

Hi, sorry for the delay. The hang and always return zero pos is due to the buffer size too small (1024 bytes)  
I have tried to offset the timestamp but not work, and found that it becauses FMod will trying to read the buffer endlessly, while SDL won’t ask for new data before current buffer is finished.  
now here comes two methods:

1. make FMod pause when playing finishes
2. reading and playing audio synchronously like the unity example

however, remove `FMOD_CREATESTREAM` flag FMod will treat the entire buffer as a whole song and never call the callback again, is there a flag or a method to let FMod only retrieves new data when buffer ends?

---

<div class="post-metadata">

**Author:** ![Connor\_FMOD](https://yyz2.discourse-cdn.com/flex036/user_avatar/qa.fmod.com/connor_fmod/32/3685_2.png) [@Connor\_FMOD](https://qa.fmod.com/u/Connor_FMOD)\
**Post date:** [January 29, 2023, 10:10pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/9 "2023-01-29T22:10:42Z")

</div>

Unfortunately, as I mentioned FMOD uses a [ring buffer](https://soundprogramming.net/programming/creating-a-ring-buffer/) so there isn’t an end to it.

> [@defisym](#):
>
> reading and playing audio synchronously like the unity example

Have you tried this method?

---

<div class="post-metadata">

**Author:** ![defisym](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@defisym](https://qa.fmod.com/u/defisym)\
**Post date:** [January 30, 2023, 2:40pm UTC](https://qa.fmod.com/t/seeking-micro-stuttering-with-ffmpeg/19724/10 "2023-01-30T14:40:06Z")

</div>

Yes, but this requires a bit more work, to rewrite several functions, that’s why I asked.  
Thank you for your reply!