Can I tell FMOD to pick different transitions?

[niu]

Hello,

It’s very possible that this has already been asked, but I couldn’t find any answer on the forum. Maybe I’m looking for the wrong thing.

Usually the way that transitions are quantized to the current bpm is by setting up transition regions. So a track in 4/4 with a grid of quantization intervals of quarter notes will look something like this:

The trigger will activate the transition timeline on the nearest following quarter and then the next loop will follow, regardless of the length of the transition.

Now I’ve been thinking for a while about a system that allows the second loop to always start on the downbeat of the bar, regardless of when the transition is being triggered. Like this:

This means that in this case there need to be 4 different transitions and the music system will have to determine which one needs to play in order to keep loop 2 starting on the downbeat.

I’ve seen a similar approach used in Spider-Man for PS4:

Spider-Man PS4: Building Dive Music

It’s not exactly the same thing because there’s no layering involved here (actually not even a loop 1, it starts directly with the “transition”), but the same concept applies since there is some calculation needed to be done in order to make the the musical climax match the sweeping when Spiderman almost reaches the street (or it could be just a coincidence).

I’m pretty sure that this requires coding, if possible at all. Has anyone tried to do something similar? Any script/example to share?
Thanks

[Alcibiade]

Yeah, that’s why I suggested a feature somewhere in the forum to add some quantization options as “every bars on 2d (or 3d, or 4th) beat”. That could handle your case, and also transitions containing an upbeat, which is quite common. Starting the transition earlier and wait for the right beat isn’t a good solution, since it makes the transition very unresponsive.
Using callbacks in the gamecode is a viable solution, which I used once to trigger an effect (snapshot and short event) on the 4th beat of a 4/4 bar. However it may require a more complicated setup for seamless music transition trigger…

[niu]

Hey Alcibiade, thanks for replying

@Alcibiade

Yeah, that’s why I suggested a feature somewhere in the forum to add some quantization options as “every bars on 2d (or 3d, or 4th) beat”. That could handle your case, and also transitions containing an upbeat, which is quite common.

That would only work if I knew in advance that I will need transitions only on 2nd/3rd/4th beat, but in this case the transition might happen anywhere within the bar. Unless you meant to set up different transition regions on top of each other, and each one with a different transition in its timeline. That could get a bit clunky, but should work fine. It would also allow transitions to be more musical, not just rhythmically, but also at a melodic and harmonic level, as they could also get you to totally different segments depending on where the transition is triggered.

The more I think about it, the best and cleanest way to manage this would probably be to keep just one transition region and inside it have the option to have multiple transitions for different quantization intervals. Maybe that’s what you meant since the beginning, I’m not sure.

@Alcibiade

Starting the transition earlier and wait for the right beat isn’t a good solution, since it makes the transition very unresponsive.

Can you please clarify this part? I’m not sure what you mean by starting the transition earlier. What I’m talking about is an approach similar to what is described in this video. But instead of automating the volume at the time of transition (which btw would be interesting to recreate), FMOD would play a different transition altogether.

@Alcibiade

Using callbacks in the gamecode is a viable solution, which I used once to trigger an effect (snapshot and short event) on the 4th beat of a 4/4 bar. However it may require a more complicated setup for seamless music transition trigger…

Did you have to deal with GCHandle and InteropService? That’s the part where I get lost, I don’t understand it…

[Alcibiade]

Ok sorry, I was talking of a related question, but not exactly what you were talking about. I re-read you post and I think I get it.
If my feature suggestion would exist, as you suggested you’d just need to make 4 transition regions in top of each other, each one with its own transition timeline, its own quantization setting (with my suggested feature), and going to the same destination. Right?
But since this feature doesn’t exist, you could do the same except every transition regions would be quarter quantized, and conditioned with a parameter (with different values for each start beat). The game code should then keep track of where we are in the bar, using callbacks, at the quarter level. When the trigger is received, the code looks at the last beat callback received (for instance 2d beat) and set the parameter to 3d beat and triggers the quantized event. Does that make sense?

@niu

Can you please clarify this part? I’m not sure what you mean by starting the transition earlier.

I mean despite the lack of a quantization option to start “on 4th beat”, it’s always possible to add 3 beat of silence before the content and trigger at the beginning of the bar. But that means you’d have to wait a long time between the input and the audible effect, which would make the effect very unresponsive. (And by the way, it makes no sense in your particular case.)

@niu

Did you have to deal with GCHandle and InteropService?

I don’t even know what you’re talking about That’s above my knowledge.

[niu]

@Alcibiade

…and going to the same destination. Right?

Well yes, but not necessarily. At this point, having different regions would allow for custom transitions with custom destinations, which is a feature that I wanted to see in FMOD for a long time. A transition triggered on the 3rd beat of measure 42 could lead somewhere completely different than the one triggered on the downbeat of 66. Sometimes composers (maybe for lack of time, technical difficulties, lack of skills, whatever) tend to compose adaptive music in long chunks, with repetitive figures, static harmonies, etc. because they know that matching transitions is not an easy task. More often than not we only have a single transition to the next segment and that’s it. I know that there are workarounds, but it would be awesome to see this kind of features implemented in one of the next releases.

@Alcibiade

But since this feature doesn’t exist, you could do the same except every transition regions would be quarter quantized, and conditioned with a parameter (with different values for each start beat).

Actually I was thinking about quantizing them all every bar, but each region would start on a different beat. This way region A (and its transition) would always trigger every downbeat, region B only on beat 2, etc. Not sure if it’ll work though, I have to try that.

@Alcibiade

I don’t even know what you’re talking about That’s above my knowledge.

Did you reference the FMOD callback example for your script, or did you do something else?

[Alcibiade]

@niu

A transition triggered on the 3rd beat of measure 42 could lead somewhere completely different than the one triggered on the downbeat of 66.

Couldn’t you get this by simply putting different transition regions above those beats?
I watched the beginning of your video link, which is related to what you were talking about at the beginning of that thread. But I struggle to find the link with your last post, and to understand what you’re looking for. Everything seems doable with some coding, but you’d like a more direct workflow directly in FMOD Studio, right?

@niu

Actually I was thinking about quantizing them all every bar, but each region would start on a different beat. This way region A (and its transition) would always trigger every downbeat, region B only on beat 2, etc. Not sure if it’ll work though, I have to try that.

No it doesn’t work. Quantization is always based on the timeline metronome mark, not the start position of the quantized object. Hence the workarounds and my feature suggestion.

@niu

Did you reference the FMOD callback example for your script, or did you do something else?

No, I used callbacks in a blueprint in UE4. That’s probably why I didn’t have to manage both code lines you were talking about

[niu]

@Alcibiade

Couldn’t you get this by simply putting different transition regions above those beats?

Sure, but can you imagine the mess after doing that for every bar/section? We were talking about one single region just for transitions on downbeats, then one for 2nd beats, one for 3rd beats, one for 4th beats. And for the sake of mental sanity we’ll assume that the whole music event is in 4/4. A system like this would work only if every bar looks pretty much the same, because each of those transitions has to work exactly the same way and lead to the same destination.
Now you’re proposing different regions above those beats, but again, this would require a very complex system with at least 4-8 transition regions across the music segment and possibly more if there are time signature changes and/or multiple destinations required.

I guess my point is: could there be a way to incorporate all the features we talked about in just one single transition region? That is, in addition of what we already have:

• A new quantization interval based on which beat of the measure (every 1st, 2nd, 3rd, 4th, etc.)
• A new condition based on position on the timeline, in conjunction with…
• An option to add multiple custom transitions and destinations based on those conditions

I think this should cover it all. It would open up a new set of possibilities for more natural and seamless transitions.

@Alcibiade

I watched the beginning of your video link, which is related to what you were talking about at the beginning of that thread. But I struggle to find the link with your last post, and to understand what you’re looking for. Everything seems doable with some coding, but you’d like a more direct workflow directly in FMOD Studio, right?

Sorry if I made it confusing, I started the thread trying to find a solution for a problem and then new ideas arised from the discussion. I should have now clarified my points at the beginning of this post.

@Alcibiade

No it doesn’t work. Quantization is always based on the timeline metronome mark, not the start position of the quantized object. Hence the workarounds and my feature suggestion.

Yeah that’s what I was afraid of

@Alcibiade

No, I used callbacks in a blueprint in UE4. That’s probably why I didn’t have to manage both code lines you were talking about

[Alcibiade]

@niu

• A new quantization interval based on which beat of the measure (every 1st, 2nd, 3rd, 4th, etc.)

Yes, that’s my feature suggestion.

@niu

• A new condition based on position on the timeline

This is simply done by doing this:

@niu

• An option to add multiple custom transitions and destinations based on those conditions

This is simply done by doing this:

I don’t really get your two last points. Sure it will be a visually mess in the logic track timeline, but you seems to want to replace this mess with another mess it will inevitably be in a maze of menus and sub-menus… I mean: you’ll still have to enter somewhere that you want between bar 56 and 62, each 2d beat transitions to…, each 3d beat transitions to…, etc. There wouldn’t be any real shortcut in the quantity of data to input, so not sure there would be any benefit for this. But I’m sure you still have a few arguments or exemples to share

[niu]

@Alcibiade

This is simply done by doing this:

Yes, sure, I’m aware of transition regions, thanks But they have a different purpose from what I’m trying to achieve. You already know, the problem with this is that the same condition is carried over that portion of timeline. For example in your picture the same transition and condition are the same from beat 1 of measure 6 till beat 4 of measure 7. A more fitting example would have been using just transitions (not regions), but you see that this could quickly become hell:

Please ignore that the transition points are all aligned in this example. Actually what I have in mind would be even messier since I’d like to be able to map the whole music segment.

@Alcibiade

This is simply done by doing this:

Yes, let’s also put these on top of the others Hopefully now you’re seeing what I meant before.

@Alcibiade

I don’t really get your two last points. Sure it will be a visually mess in the logic track timeline, but you seems to want to replace this mess with another mess it will inevitably be in a maze of menus and sub-menus… I mean: you’ll still have to enter somewhere that you want between bar 56 and 62, each 2d beat transitions to…, each 3d beat transitions to…, etc.

It would be a mess if the interface is kept the same. There would have to be some changes inevitably, as for any other software that is being constantly updated. I don’t expect it to happen any time soon (or at all, for that matters), but hopefully someone will at least consider it at some point.

An idea would be to add a “timeline condition” or something like that, with a list of sync points that we could add and edit. The benefits are that it cleans up the timeline window from the mess of all the single transition points, and that they’re all editable and stored in one place. You talked about a maze of menus and submenus, but we already have it for the “add parameter condition” voice, with even a dedicated browser, so I don’t see what the big deal is.

If the right conditions are met, these sync points could override the current transition region and trigger a different transition and/or destination. This way we could keep the transition region for all the usual quantization points EXCEPT these particular moments in the music. If something in the gameplay triggers right before those moments I want the music to take a different direction.

Alternatively, instead of adding a new condition of the transition region, we could rethink transitions a bit, at least visually. Maybe instead of piling things up, there might be a better way. Here for example the sync points are put directly on the waveform.

I believe that there are solutions, but the interface has to be reworked a little bit.

@Alcibiade

There wouldn’t be any real shortcut in the quantity of data to input, so not sure there would be any benefit for this.

No there wouldn’t be and there shouldn’t either, if one wants to be detailed with this kind of stuff. I personally think that the benefit would be to have a more compact and cleaner interface, with more options for customization and workflow that encourages experimentation and creativity, instead of keeping the user from trying something different just because it’s visually too chaotic and it takes too long to set up.

@Alcibiade

But I’m sure you still have a few arguments or exemples to share

Yep

[Alcibiade]

@niu

If the right conditions are met, these sync points could override the current transition region and trigger a different transition and/or destination. This way we could keep the transition region for all the usual quantization points EXCEPT these particular moments in the music. If something in the gameplay triggers right before those moments I want the music to take a different direction.

You can place other transitions in top of the main one, the priority will apply. But I know you know that!

Do you have in mind some exemples with lots and lots of possible transitions, which would benefit from the interface you described? We know each other a bit outside fmod forum, and I’m sure you have some exemples in mind But I mean: there’s often only 4 beats in a bar ; there’s only a few points on the timeline where we would like some special transitions ; this is not a huge amount of transitions.

[niu]

You’re thinking just in terms of beats, but it doesn’t always necessarily sound right even if tempo and rhythm are aligned. Melody, harmony, timbre and dynamics are also factors to take in consideration. Sure one can just crossfade between segments and that’s it, but sometimes the difference between the segments is so evident that sticks out too much, at least for me.

I don’t have examples of systems that work like this. I haven’t found any GDC talks or similar where stuff like this has been discussed (except maybe that series by Kejero that I linked you in private), so if such systems have been implemented in games I probably wouldn’t even notice because the transitions would be totally seamless. And if someone did create something similar I bet they didn’t used FMOD for this specific purpose

I have however an example of a music system that does everything phenomenally well, except maybe handling transitions. I mean, they work fine and with SFXs and dialogues on top it’s not super noticeable… but I noticed Don’t get me wrong, I love this game to bits and the work they’ve done with the music is stellar (both composition and implementation), but it could have been even better.

FINAL FANTASY VII REMAKE - JENOVA Dreamweaver Boss - Music Only

Every boss in the game has multiple phases with pretty much the same classic pattern:
[loop1] -transition-> [loop2] -transition-> [loop3] -transition-> [ending cutscene]

Being a music only gameplay you can immediately tell the “problem” between sections, especially between loop 2 and 3. This one’s not even that bad, but I played tens of hours and I can guarantee you that there are many occasions where, if you don’t trigger the transition at an optimal time, there would be a clash with the previous section and it will sound wrong.

The only solution is to write custom transitions that serve as bridge in the most musical way possible. No abrupt cuts in the melody, no weird cadences, no clashes in the harmony, no interruptions in the instrumentation, no sudden jumps in dynamics, no rhythmic fills after one bar or less during 2-bar phrases, etc. At the very least it should be done whenever there’s a big change in mode/tonality, because when you suddenly come back that’s the first thing that the ear notices.

Because this would inevitably take a long time to plan and execute, I would probably keep it just for the most salient parts of the game. But it would be a great challenge for a composer and maybe someone could even come up with ways to create a whole game around this idea.

[dominik.weltenbauer]

Bump.
Because transitions to random marker are a thing that keeps coming up in the forum.
Luckily some smart guys have proposed a way to do this with tons of regions and programmer instruments. See this thread:
How to setup one Transition Marker with a random Destination? - #11 by Johanenglund

But that is a solution that quickly spirals out of control and could be greatly simplified if this was a built in feature.

Tanks

[joseph]

@dominik.weltenbauer

Bump.

Exactly which feature are you requesting? The earlier posts in this thread discuss multiple possibilities.

[dominik.weltenbauer]

I’d love to see transition regions with a pool of random markers to jump to.
Cheers

[joseph]

Thanks for the clarification! I’ve already added you to the list of people who’ve asked for that feature.

[dominik.weltenbauer]

@joseph

e clarification! I’ve already added you to the lis

Thank you!