Find and click on word on the screen using BTT "find/search text on screen and move mouse" action as a subroutine from Keyboard Maestro

[bremin001]

I've been struggling to create a subroutine to find and click on a word on the screen using Better Touch Tool (BTT) "find/search text on screen and move mouse" action from Keyboard Maestro.

I hope there is a way to assign the a search word to a variable in KM, evoke BTT action from KM, pick up the variable in BTT, execute it and click on the word found with the mouse and return the control back to KM. Is it doable? How can one transfer variables between KM and BTT? Is there a way to run a specific BTT macro from KM?

I am aware of the MoveMouseToWord macro for KM created by @Airy but find it more difficult to use to achieve the same result.

[Frank1]

@bremin001

Is there a way to run a specific BTT macro from KM?

Is this what you mean?

[bremin001]

Great - thanks - this is how I execute KM macros from BTT, I was not aware I can do it in reverse as well.

Do you know how to share a variable in between the two apps? Can I do it via AppleScript as well?

[Andreas_Hegenberg]

Yes you can do everything with Apple Script (or with the CLI tool that's included in the lates BTT alphas)

For example you could also copy the apple script for the find text action:

tell application "BetterTouchTool"
trigger_action "{
  BTTIsPureAction: 1,
  BTTPredefinedActionType: 426,
  BTTPredefinedActionName: 'Find or Search Text on Screen & Move Mouse',
  BTTGenericActionConfig2: '{\\\"BTTFindImageSearchRegion\\\":0,\\\"BTTFindTextSearchDirection\\\":0,\\\"BTTActionWaitForConditionsInterval\\\":1,\\\"BTTActionWaitForConditionsTimeout\\\":10,\\\"BTTFindTextMatchMode\\\":1,\\\"BTTFindTextContinueActionExecution\\\":0,\\\"BTTFindText\\\":\\\"lallaa\\\",\\\"BTTFindTextRequiredOccurences\\\":0,\\\"BTTFindTextMoveMouseTo\\\":2,\\\"BTTFindTextSearchOn\\\":0}',
}"

end tell

In general I recommend to use named triggers, they are the easiest to call externally:

tell application "BetterTouchTool" to trigger_named "do-something"

To set a variable you can use

tell application "BetterTouchTool" to set_string_variable "variableName" to "variableValue

[bremin001]

@Andreas_Hegenberg

tell application "BetterTouchTool" to set_string_variable "variableName" to "variableValue

Fantastic, thanks!

What is the correct way to add variable to the Text to search box in BTT action?

[Andreas_Hegenberg]

putting it in curly braces should work {variableName}

[bremin001]

OK I am struggling here. What is wrong with this one? I am trying to use "FindWord" variable from KM, and bring it to BTT, assigning its value to ClickWord variable (do I need to do it?). Is there a way to do it in a simpler manner?

[Andreas_Hegenberg]

you are only telling keyboard maestro, but not transferring the variable to BTT

I don't know the KM interfaces at all, but it should be something like

tell application "Keyboard Maestro Engine"
    set kmValue to getvariable "FindWord"
end tell

tell application "BetterTouchTool"
    set_string_variable "ClickWord" to kmValue
end tell

[August]

FYI, the flip side of this question is on the KBM forum at:

Is there a KM equivalent to the Better Touch Tool action "search for text on screen and move mouse"? - Questions & Suggestions - Keyboard Maestro Discourse

Just as in this forum, anyone can read posts but you have to sign up to comment.

[August]

@Andreas_Hegenberg

I don't know the KM interfaces at all, but it should be something like

tell application "Keyboard Maestro Engine"
    set kmValue to getvariable "FindWord"
end tell

There are two ways for AppleScript to access KBM variables. One is similar to what has been suggested above. The other is available when Keyboard Maestro initiates the AppleScript script. Then it make all KBM variables available to the AppleScript environment.

Here is the relevant section from the KBM Wiki:

action:Execute an AppleScript [Keyboard Maestro Wiki] action:Execute an AppleScript [Keyboard Maestro Wiki]

---

Using Keyboard Maestro Variables

Keyboard Maestro sets the environment variables for the script to include all (by default) your variables, using a prefix of KMVAR_ and your variable name with spaces changed in to underscores (_). For example, your Keyboard Maestro “File Name” variable will be available as the environment variable KMVAR_File_Name. By default, all variables are included, but you can limit which variables are included by using the popup menu next to the script (v11.0+). Note that these provide a readonly copy of the value of the variables when the script starts.

Also, you can (7.1+) access Keyboard Maestro variables directly like this:

snippet.applescript

tell application "Keyboard Maestro Engine"
    set v to getvariable "<KM Variable Name>"
    setvariable "<KM Variable Name>" to "<New Value>"
end tell

where both the <KM Variable Name> and <New Value> are text values.

---