Let's diagnose this. Can we try routing the program change to the computer (via Wi-Fi or cable) and then get it to the Roland Fantom? There are a lot of ways to do this, but one hint is to use "live routings" on the MIDI Network Settings to send the prog changes onward.
The advantage to going through the computer, for diagnostic purposes, is that we can get a MIDI monitor to watch the whole thing.
Looking forward to pinpointing (and then resolving) the latency.
Thanks!
Dan