I did add a few more logs in the latest alpha version (get via check for alpha version updates). Could you upgrade to that, then try to reproduce the issue and then go to "Help => Export Diagnostic Debug Data" and send the result to andreas@folivora.ai? Maybe I can see what's causing the error for you then.
Just chiming in to say that I'm experiencing the same thing as @charleshan after upgrading to 12.3.1, on 3.703. I can generate those pop-up windows in two ways:
1/ Put the machine to sleep, and then wake it back up. After waking up, a number of those pop-up windows will be present (the longer the machine has been asleep, the more there are)
2/Quit BTT and re-launch. That will generate a single pop-up window.
you need to install a newer BTT version 
I realized that Setapp wasn't applying the updates. You have to open Setapp > Check for Updates... > Close applications to update
1 Like
Well I feel silly! When I first started noticing the issue, I thought I had run an update check, but apparently not. Belated thanks for fixing the issue!