I'd rather not have to make a BTT mapping for each hyper key mapping though I do see how I think that could work.
That said while I can see how to create a mapping that then sends a different set of key presses I'm still not sure how to create a "caps + j" mapping. Pressing the two when trying to record a shortcut ends up with only "caps lock" being registered.
I just stumbled across this bug report and have a similar problem with Caps Lock as Hyperkey. My workaround is to press Caps Lock alone if it hasn't been pressed for a while, and then press the global shortcut, e.g. Caps Lock + j. Otherwise it won't work reliably.
Note that the first press of Caps Lock is recognized, as a HUD overlay shows. But for some reason that's not enough to trigger the global shortcut.
Like @bdarfler, I used other apps to process the Hyperkey, and they worked fine. But BTT has its advantages, and I don't want to have multiple apps with overlapping functionality. Maybe someone knows how to improve this?
Are you aware that BTT can distinguish between left and right modifiers? Perhaps depending on the keyboard you use and what you need it for, mapping Caps Lock to right control is more reliable for you.
@Andreas_Hegenberg I experience the same issue as @bdarfler, and I do not use other keyboard-mapping tools, only BTT. Occasionally there is a <1s failure for caps lock to act as all modifiers. This is extremely disruptive as it results in keyboard shortcuts sending text or other shortcuts to my applications.
Important note: for many years I used Karabiner Elements to re-map caps lock to hyper, and never had any such issues. I noticed BTT had this feature around a year ago and switched to get rid of Karabiner and reduce the complexity of my setup.
I will probably re-install Karabiner for this one feature, as I had been using before.
@Andreas_Hegenberg I know it's not your job to teach us about how the software works, but if you can comment on possible differences between how Karabiner re-maps versus how BTT does, I would really appreciate it.
I know this is an old thread but I only recently started using BTT's hyperkey function and I'm having this issue. It sometimes does not work the first time, usually if I haven't used it in a little while, but it always works on the second attempt. I can't use Frank1's workaround because I'm on a Mac Mini and the keyboard I'm using does have a right ctrl and is something I've already mapped to other macros. Anyways, this is a bit of a nuisance and would be nice if there was a fix.
which version of BTT are you on? (This should be resolved in versions >= 4.999)
I'm on 5.160
Weird, I haven't encountered any issues with current BTT versions.
Are you using this with shortcuts defined in BTT or with shortcuts defined somewhere else?
No, not necessarily...does they hyper key only work consistently if it's a keyboard shortcut that has been set in BTT? In the Raycast app I set Hyper Key + V to bring up my clipboard manager. I suppose I could switch that to BTT by executing a deeplink to the clipboard but I'd have to try to switch everything that I use Hyper Key for to BTT and that would be a task I'd prefer to avoid.
It should also work outside of BTT, but there it can be influenced by more things (e.g. other keyboard related apps running)
How does it behave in the cases where it doesn't work? Does it then type the character or do nothing?
It simply does nothing. I have it set so that actions executed on (unused) triggers a keyboard shortcut but hyper key otherwise. The only two apps that intercept key events that I can think of that I have are Keyboard Maestro and BetterMouse.
(and raycast 
)
I'll check whether I can reproduce it somehow. How often does it happen approximately?
It happens frequently...usually if I haven't used it for more than say 5 or 10 minutes...but if I use it and then need to use it shortly again afterwards in under that time it works fine. I'm not sure if this helps but if I simply tap Caps Lock to use the keyboard shortcut I have mapped to the "actions executed on (unused)" that always works...there's never an issue.
Would be great if you could also try upgrading to the latest alpha, 5.172. I recently made some little adjustments to hyperkey and most importantly added some more logging. If you could reproduce the issue with that version and then go to Help =>Export Diagnostic Debug Information and send the resulting logs to me (andreas@folivora.ai) that could help a lot.
Ok I'll see if I can do that 
I just added one more log / check in 5.173 alpha (uploading now).
This version now adds an extra check to see whether the system believes all required modifier keys are pressed - and if not retries to activate them.
On 5.172 I couldn't reproduce the issue for Raycast, for whatever reason, but it did happen for another app called Sidenotes. However, what you did in 5.173 seems to have fixed it. Perhaps I have to do more testing since it's a bit tedious to wait every 5 to 10 minutes before attempting. But I've tried it twice so far to activate Sidenotes and it worked each time so consider it solved for the time being I'll reach out if it happens again but so far...Thanks for the help and the quick fix!
I spoke too soon...was able to reproduce it with both Raycast and Sidenotes on 5.173...sent you the logs as requested.