Haven't had similar reports for current versions yet. Maybe secure input is enabled on your machine? - this would break all FN based shortcuts. (BTT will show a warning about that when clicking the menubar icon).
I haven't done a lot of testing on 10.13 during the last two years, if necessary you can switch to the old keyboard implementation in the settings:
A little grey button was displayed at the top right of the window with this message "BTT is not allowed, click here to wake it up" (or something like that). I don't understand where it came from (Is this new? In any case, I hadn't seen him until now). I clicked and now BTT works fine (without checking the box you proposed).