Seeking issue with custom Opus codec in FMOD when playing multiple stems simultaneously

Hi everyone,

I’m facing an issue when using a custom Opus codec with FMOD in my Unity c# test project. I am playing multiple stems at the same time, and I need to be able to seek to different positions in these stems at the same time. The problem occurs when seeking the stems to a new position (e.g., 25000 PCM samples), where FMOD provides an incorrect position that actually ahead of the requested position for some of the stems.

This issue only happens when using the custom Opus codec, and not when using other codecs. Additionally, the issue is only present when using createStream, and not when loading .opus sounds with createSound.

I’ve tried pausing all stems before seeking and then resuming playback, but the issue persists. I’ve also tried using system.lockDSP and unlock, before and after seeking but this just reduces the possibility of the issue happening. I have also verified that the incorrect position is being passed to the codec callback for some stems by logging/debugging the position values in the SetPosition callback as shown in the following screenshots:
Screenshot 2023-04-11 at 0.48.44 - Callback being passed the correct position

Screenshot 2023-04-11 at 0.49.19 - Callback being passed a position that’s ahead of the one requested in the C# code

Here is the relevant C# test code:
`using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using FMODUnity;
using System.IO;
using System;
using System.Runtime.InteropServices;

public class TestFMODOpus : MonoBehaviour
{
List<FMOD.Sound> sounds = new List<FMOD.Sound>();
FMOD.ChannelGroup channelGroup;
List<FMOD.Channel> channels = new List<FMOD.Channel>();

FMOD.System system;

// Start is called before the first frame update
void Start()
{
    system = FMODUnity.RuntimeManager.CoreSystem;
    system.getMasterChannelGroup(out channelGroup);

}

public void LoadAudioFromFolder(string folderPath)
{
    if (Directory.Exists(folderPath))
    {
        string[] opusFiles = Directory.GetFiles(folderPath, "*.opus");

        foreach (string opusFile in opusFiles)
        {
            LoadAudio(opusFile);
        }


        foreach (FMOD.Channel channel in channels)
        {
            channel.setPosition(125 * 48000, FMOD.TIMEUNIT.PCM); //6,000,000 PCM
        }

        foreach (FMOD.Channel channel in channels)
        {
            channel.setPaused(false);
        }

    }
    else
    {
        Debug.LogError("Folder path not found: " + folderPath);
    }
}

private void LoadAudio(string path)
{
    FMOD.RESULT result;
    FMOD.Sound sound;


    // Load the sound from the provided path
    result = system.createStream(path, FMOD.MODE.DEFAULT | FMOD.MODE.ACCURATETIME, out sound);
    result = system.playSound(sound, channelGroup, true, out FMOD.Channel channel);

    channels.Add(channel);

    if (result != FMOD.RESULT.OK)
    {
        Debug.LogError("FMOD Error: " + result + " - Failed to load sound: " + path);
        return;
    }

    sounds.Add(sound);
}

void releaseSounds()
{
    foreach (FMOD.Sound sound in sounds)
    {
        sound.release();
    }
}

void OnDestroy()
{
    releaseSounds();
}

}`

Could you please provide guidance on how to resolve this issue, or let me know if there is a potential bug within the FMOD API that might be causing this problem? I can also send the opus codec code if needed.

Thank you in advance for your assistance.

(I apologize if the topic is duplicated, the spam filter catched the first post and I can’t find it nowhere even after it supposedly being released by an admin.)

Unfortunately, I haven’t been able to reproduce your issue . Your provided C# code seems fine, which makes me think it’s either an issue with your codec (e.g. a timeunit mismatch), an issue with the inherent inexact scheduling of streams, or some combination of the two. If you could provide the code for your Opus codec so that I can debug the issue, that’d be great.

Your previous post got deleted due to some confusion in the spam filter, so no worries regarding the duplicate topic - you’re all good now.

Just sent it to you via DM :slight_smile:

I’ve been looking at this issue a bit more today, I think I have a better idea of what’s going on.

In my case, seeking multiple channels at once, if one of the seeks takes too long to finish and FMOD manages to update in the middle of the channels seeking, the remaining channels will receive the wrong position on the callback that’s ahead of what was requested in the C# code.

On the other hand, if I replace the codec seeking code to always return FMOD_OK immediately, channels will take absolutely nothing to “seek” and, because of that, they all receive the correct position on the callback.

I also tested this by replaced the seeking code in the codec for a function that takes a loooong time just to rule out any possibility of the opus library causing the issue, and sure enough, as the seeks take a long time to finish, FMOD manages to update mid calls and as a result, the rest of the channels received that altered position in their callbacks.

This finding probably also explains why Louis wasn’t able to reproduce the issue, and why system.lockDSP, lowers the chances of the issue happening. (And when it does happen even when locking, it probably means that the seek code just took a ridiculous amount of time to finish).

The question here, if what I’m seeing is correct, would be: Why is FMOD passing another position than requested when it updates mid setPosition calls. That behavior seems wrong as the position passed by the user via setPosition shouldn’t be touched or updated via FMOD update routines, unless I’m missing something.

Hi ,
The issue is unlikely to be with timing. Do you have a known fixed PCM block size that your codec produces? If so you should set that in the codec wave format structure in fmod.

If a seek position is selected, it will possibly seek into the middle of a compressed frame, and will read forward until it finds the next valid frame, that is possibly why it moves forward.

typically the codec code will round down to the seek position before the next valid frame, read it (a complete frame) and throw away samples it doesnt need so the pcm output position is at exactly the right sample.