Hi,
Apologies for the delayed response.
Yes, the limit is 8 (FMOD API | Core API Reference).
Yes, that is why we limit it to 8.
Unfortunately, this is not something we support. You would have to create your own DSP. We provide an example of creating your own DSP with the FMOD Engine here:
X:\Program Files (x86)\FMOD SoundSystem\FMOD Studio API Windows\api\core\examples
I would suggest using the Inverse Square Law when calculating the loudness, which is essentially that every doubling off distance equates to 6dB off attenuation, so if you are 1m from a sound source and it is 100dB, at 2m it will be 94dB and at 4m it will be 88dB. There are other factors such as occlusion and sound reflections, but you can approximate it pretty well using the Inverse Square Law.
Hope this helps!