Does anybody know how BTT gets these UI elements/variables?

Hi, all.

I'm a long-time user and huge evangelist of BTT. I bought my first license way back in 2013, and I've been using it religiously ever since! Sometimes, however, I don't have the luxury of being able to use BTT, either because the computer I'm using doesn't have a copy, or because a script I'm writing needs to be able to work on any machine. And that's what brings me here today.

Specifically, I'm wondering if anybody has any idea how BTT gets the data for these UI element variables:

  • focused_window_title
  • focused_window_app_name
  • active_app_name
  • active_app_bundle_identifier

The reason I ask is because I cannot for the life of me figure out how to detect when one of those weird non-activating windows is frontmost/focused. I'm not sure what the technical term for them is, but I'm basically trying to describe one of those app windows that you can interact with but which doesn't cause that app to become frontmost/activated. For example, if I have Safari open and then invoke Numi to do some calculations, typing in the Numi window that pops up does NOT result in Numi becoming the frontmost app; Safari is still frontmost, as far as macOS is concerned (i.e., Safari is still the app shown in the menu bar). This causes problems when I try to detect if Numi is frontmost using AppleScript or JXA, because the scripts always return false even when Numi IS the focused window/app since they still think Safari is.

This obviously isn't a Numi-only problem, but Numi is the only app I haven't been able to figure out a workaround for. I even spent an hour or two chatting with GPT4 looking for a solution, but that didn't help, either. I eventually gave up and came here out of desperation because somehow, some way, BTT is able to detect when this window is focused. I can tell because all four of the aforementioned variables in the UI element explorer (focused_window_title, focused_window_app_name, active_app_name, and active_app_bundle_identifier) correctly identify Numi as the focused window/active app.

So please, if anybody here knows how I can get this info myself via AppleScript, JXA, or something similar, I'm begging you—take pity on me and reveal your secrets! :sob::pray::joy:

Thanks. Fingers crossed! :crossed_fingers:t2:
-Sean

BTT uses the macOS accessibility API and various tricks for specific situations to get this stuff. It should be possible to replicate the same in Apple Script, but it might be tricky in some cases.

However I don't see anything special in Numi, it seems to be a standard, activating window - or am I missing some setting?

Until a year ago, the PFAssistive Framework was still available for download on the website of the developer of the UI Browser.
Sadly, Bill Cheeseman has closed his website at the age of 79.

You may still be able to download the framework somewhere. You can use it relatively easily in AppleScript.

Here is an example of how you can intercept the change of the active application with AppleScript even without the framework:

use framework "Foundation"
use scripting additions

property ca : current application
property workspaceClass : class "NSWorkspace"

set ws to workspaceClass's sharedWorkspace()
set nc to ws's notificationCenter()
tell nc to addObserver:me selector:"applicationDidActivate:" |name|:"NSWorkspaceDidActivateApplicationNotification" object:(missing value)

on applicationDidActivate:theNSNotification
	set userInfo to theNSNotification's userInfo()
	set theApp to userInfo's valueForKey:(ca's NSWorkspaceApplicationKey)
	set theAppName to (theApp's localizedName()) as text
	set theBundleIdentifier to (theApp's bundleIdentifier()) as text
	
	display notification (theAppName & "(" & theBundleIdentifier & ") was activated.") with title "Observer"
end applicationDidActivate:

delay 120 -- for testing in Script Editor