New Haptics Instrument in Version 2.03.11 and using in Unreal Engine

Unreal Engine
1

Hi, we have recently upgraded to fmod studio 2.03.11 and we’re trying out the new haptics instrument which seems pretty cool so far.

I have the vibration preview working when pressing play play on an event containing a haptics instrument in fmod, however is this something that can be piped to Unreal Engine? (version 5.4) I can’t seem to find any information on this. Thanks in advance and keep these updates coming!

2

The relevant information on how to get additional plugins setup in Unreal can be found here Unreal | Plugins | FMOD Haptics Setup. To audition haptics in PIE it should just be a matter of adding fmod_haptics to your Dynamic Plugins list. Tnd there are a few additional platform-dependent steps for Deploying FMOD plugins in a build.
Please give that a try and let me know if you run into any issues getting it working.

3

Thanks for your reply Jeff, that’s working now :slight_smile:

Next question, can these haptic files/instruments be triggered via code? Instead of authoring them into events in the fmod studio editor? Could they even be fired from a programmer instrument or are these for use in studio solely?

4

You can play a haptics clip purely with code with these steps:

  1. Retrieve a plugin handle with System::loadPlugin
  2. Create a plugin instance with System::createDSPByPlugin
  3. Read the raw bytes from a haptics clip
  4. Pass the plugin instance the haptics clip with DSP::setParameterData
  5. Play the haptics clip with System::playDSP
#include "fmod_haptics.h"

...

FILE *fp = fopen("path/to/your_haptic.haptic", "r");
if (fp)
{
    char buffer[4096]{};
    fgets(buffer, 4096, fp);
    fseek(fp, 0L, SEEK_END);
    int sz = ftell(fp);
    fseek(fp, 0L, SEEK_SET);

    FMOD::System *CoreSystem;
    StudioSystem->getCoreSystem(&CoreSystem);

    unsigned int HapticsHandle;
    FMOD::DSP *HapticsDSP;
    FMOD::Channel *HapticsChannel;

    FMOD_RESULT result = CoreSystem->loadPlugin("path/to/fmod_haptics.dll", &HapticsHandle);
    result = CoreSystem->createDSPByPlugin(HapticsHandle, &HapticsDSP);
    result = HapticsDSP->setParameterData(FMOD_HAPTICS::FMOD_HAPTICS_CLIP, buffer, sz);
    result = CoreSystem->playDSP(HapticsDSP, 0, false, &HapticsChannel);
}

I haven’t attempted this inside a programmer instrument, but I don’t see why you shouldn’t be able to.

5

Thanks for your reply, this is really helpful.

One thing I’ve noticed when testing with both PS5 and Xbox Series controllers, it they behave differently; the Ps5 one is punchy and as expected, whereas the Xbox one I found it behaved like it was waiting for the vibration to finish before triggering another (this is without any stealing behavior applied)
Do you have a recommended app for generating the needed .haptic files? As I am using the Meta Haptics Studio, and wondered if it was related to that?

6

As far as I know Meta Haptics Studio is currently the only tool for creating .haptic files.
The Xbox controller is sluggish as it has more primitive vibration hardware than the PS5 controller, and currently there is a resource contention issue when multiple hosts are open (i.e, FMOD Studio and Unreal Editor open simultaneously), which makes it even worse- we are hoping to correct this in the next release.
It shouldn’t be waiting for voices to finish though, can you please try reconnecting the controller with just FMOD Studio or just Unreal open and let me know if that improves things at all?

7

Thanks again for your replies on this. I tried again with fmod closed and it behaves as expected, so yeah it sounds like the issue you are describing.

We did have one more issue that is hard to track down what happened exactly… but I submitted some banks not related to the haptics, and they caused some other sounds to stop functioning. I think it’s when the Unreal build/project itself doesn’t have the haptics plugin in the dynamic plugins list. Does the haptics instrument write some information to ALL banks?

8

The haptics instrument doesn’t write information to all banks. If you added a “haptics event” to your master bank I can imagine that causing problems if you haven’t added the plugin to your list, but other events should be able to play if there is a missing plugin definition in the plugin list.
Can you please tick Enable API Error Logging, set the Logging Level to “LEVEL LOG”, and share the output of Unreal’s log after trying to play one of these failing events?

9

Hi, I’ve not been able to repro that issue but will save out the log if it occurs again. Thank you.

We are looking to implement the above coded method you decibe, but can I just check with you how Unreal gets the .haptic files at runtime from a bank? Is it best to setup an audio table pointing to a folder? I have them just in a folder named ‘Haptics’ in assets currently.

I am getting the error ‘Failed to open haptics file from path:’ and then ‘Play2DEffect - Could not find a valid HapticsDSP’ even though the haptics plugin is loaded.

10

You cannot currently package non-audio assets into an audio table, though such an idea is intriguing- I will write a suggestion for the Dev team.

For use in editor, putting the haptic files in your Content directory should be sufficient for loading haptic clips. You will also need to ensure that the haptic files are copied into a well-defined location during a build, otherwise you won’t be able to load them at runtime.

I’ll see if I can come up with a more precise code snippet- bear with me.

11

Thanks Jeff, this looks like it should work and would be a nice way to package these up potentially using audio tables!

We are getting somewhere with this now and have some things working, although I still have a few questions if you don’t mind:

  1. We are finding a behaviour with coded haptics where they play once, and then never again. Is this something you’ve experienced before/known issue at all? (This could be our side)
  2. Can coded haptics have a concept of ‘3d’ or distance? For example the strength varies dependent on how far away it is from the player?
  3. Similar to the above, but is there a way in code to adjust the strength/intensity of the haptic feedback? For example, for intensity sliders in an options menu or similar.

Thanks in advance!

12

If you are doing the full process of:

CoreSystem->createDSPByPlugin(HapticsHandle, &HapticsDSP);
HapticsDSP->setParameterData(FMOD_HAPTICS::FMOD_HAPTICS_CLIP, buffer, sz);
CoreSystem->playDSP(HapticsDSP, 0, false, &HapticsChannel);

every time, it should play a fresh haptic clip each time.

It doesn’t currently hook into the 3D attributes like a spatializer does, but creating a relationship between distance and haptic strength is achievable through the FMOD_HAPTICS_INTENSITY_LEFT and FMOD_HAPTICS_INTENSITY_RIGHT parameters. Here is an example of setting the haptic intensity depending on how close the Actor is to the listener.

FMOD_3D_ATTRIBUTES Attr{};
StudioSystem->getListenerAttributes(0, &Attr);

FMOD_VECTOR Lis = Attr.possition;
FMOD_VECTOR Pos = FMODUtils::ConvertWorldVector(GetActorLocation());

float ManhattanDistance = fabs(Lis.x - Pos.x) + 
                          fabs(Lis.y - Pos.y) + 
                          fabs(Lis.z - Pos.z);

float MaxDistance = 20.0f;
float Strength = MaxDistance < ManhattanDistance ? 0.0f
              : (MaxDistance - ManhattanDistance) / MaxDistance;

HapticsDSP->setParameterFloat(FMOD_HAPTICS_INTENSITY_LEFT, Strength);
HapticsDSP->setParameterFloat(FMOD_HAPTICS_INTENSITY_RIGHT, Strength);

For this you could expose something in your settings as a multiplier and feed it to the above equation.

Strength *= UserPreferences.HapticIntensity / 100.0f; // Some float intensity value between 0-100

HapticsDSP->setParameterFloat(FMOD_HAPTICS_INTENSITY_LEFT, Strength);
HapticsDSP->setParameterFloat(FMOD_HAPTICS_INTENSITY_RIGHT, Strength);
13

Thanks Jeff, we have this working via code now. We are now trying to get this working on xbox and ps5, does the code you advised using above to call the plugin library file still apply in the same way? I see the “fmod_haptics.dll” on ps5 is called “libfmod_haptics.prx” for example - can we use the same code to call it?

14

Yes it is all the same rules on PS5 and Xbox so you can use the same code, except on PS5 where you will need to substitute the lib name in your CoreSystem->loadPlugin call with “libfmod_haptics.prx”.

For packaging, also please ensure you add a “plugins.txt” file with the name of the lib you want packaged (libfmod_haptics for PS5, fmod_haptics for Xbox) to the respective platform binary directory (Platforms/PS5/Plugins/FMODStudio/Binaries for PS5, Platforms/XSX/Plugins/FMODStudio/Binaries for Xbox).

15

Thanks again for this, we’ll try it out - and happy holidays! :slight_smile:

1 Like