Using iPhone BTT when Airplay is active is laggy (unusable)

I have bought this app in the past, and completely forgotten about it, until today I was playing some point&click game, and I remembered it. I got the "great idea" of playing the game on my TV by using:
a) Airplay through Apple TV to project the game on TV, and then
b) use BTT to control the mouse using the BTT iPhone app.

Anyway, I tried the setup - both of those things work for me fine on their own. I turned the Airplay on, and of course you can use the touchpad on the Mac as usual, things work fine, and you can see them on TV working fine. Also when the AirPlay is turned off, moving the mouse and tapping etc... through the BTT iPhone app is fast and responsive.

However using iPhone BTT to control the cursor WHILE using airplay is not possible. There is a lag between what I do on the iPhone and the movement of the cursor. Sometimes it moves just a bit, sometimes it jumps quite a bit after a lag.

As the CPU usage is low, the only thing that makes sense to me is that the BTT network communication is getting congested because of the AirPlay network communication. What is weird though, is that AirPlay network communication is mostly from the Apple Mac to the Apple TV, and the data flow for BTT, I guess is mostly from the phone to the Mac.

Any idea if there is something I can do to fix this?