Reason(s) for applescript to not be executed

xidiot · November 19, 2024, 5:21pm

I'm missing something super simple - I've lost too much time wasting investigating why a persistent variable is not set when an applescript is executed by a shortcut (HUD is shown, so the tigger is executed, right?). The same var is set, if I run the script manually using the "Run Script" therefore seeking for your wisdom.

I have other scripts which are successfully setting some vars, but for some reason this script does not work. The script is working if executed via Script Editor.

Appreciate any hints, what I'm missing here. Here's the script:

on itemScript(itemUUID)
	-- Define the path to your Python script
	set pythonScriptPath to "~/Documents/btt/scripts/youtube_get_video_id.py"

	-- Define the path to your virtual environment
	set venvPath to "~/tmp/venv_youtube/"

	-- Define the channel name and search query
	tell application "BetterTouchTool"
		set searchQuery to get_string_variable "BTTNowPlayingInfoTitle"
		set channelName to get_string_variable "BTTNowPlayingInfoArtist"
	end tell

	-- Construct the command to execute the Python script
	set command to "source " & quoted form of (venvPath & "/bin/activate") & " && python3 " & quoted form of pythonScriptPath & " " & quoted form of channelName & " " & quoted form of searchQuery
	-- Execute the command
	try
		set video_id to do shell script command
		-- Set the persistent string variable in BetterTouchTool
		tell application "BetterTouchTool"
    		set_persistent_string_variable "custom_var_video_id" to video_id
   		end tell
		return video_id

	on error errMsg
		display dialog "Error: " & errMsg
	end try

end itemScript

Andreas_Hegenberg · November 19, 2024, 6:39pm

is it maybe related to environment variables? As far as I know they are available in Script Editor, but BTT can’t load them.

In general I’d always recommend to
use BTT’s real java script instead of apple scripts - it’s easier to debug. If you want I can post the equivalent JS

xidiot · November 19, 2024, 8:09pm

Which env variables are you referring to?

I'm curious why the persistent variable "custom_var_video_id" is being set if I manually run the script via BTT's "Run Script" button, but not when I press the shortcut?

If you have time you can post the JS equivalent, chatGPT was not able to "convert" it.

Andreas_Hegenberg · November 19, 2024, 8:18pm

I mean system environment variables that might be used by your python script. These are often the reason for problems. One moment I'll post a converted version of your code.

Andreas_Hegenberg · November 19, 2024, 8:27pm

async function itemScript(itemUUID) {

    let pythonScriptPath = "~/Documents/btt/scripts/youtube_get_video_id.py";
    let venvPath = "~/tmp/venv_youtube/"

    let searchQuery = await get_string_variable({variableName: "BTTNowPlayingInfoTitle"});
    let channelName = await get_string_variable({variableName: "BTTNowPlayingInfoArtist"});

    // UNCOMMENT this:
    //let command = `source '${venvPath}/bin/activate' && python3 '${pythonScriptPath}' '${channelName}' '${searchQuery}'`;
    let command = `date`;
    let video_id = await runShellScript({"script": command});

    await set_persistent_string_variable({variableName: "custom_var_video_id", to: video_id})
	console.log('video_id', video_id);
	return video_id;
}

As I can't test your terminal command I replaced it with date you can uncomment the commented command and delete the date one.

To view the output of console.logs, you can enable this option Safari, then it will automatically open a debugger once this starts running:

I assume your youtube_get_video_id.py could also be converted to JS, chatgpt is good at converting python to JS. Then you would not need to call a separate file. Also in general always use the full path to a cli tool in BTT - BTT does not have your PATH environment variable available, thus it does not necessarily know where python3 is. Better use /usr/bin/python3

xidiot · November 19, 2024, 8:46pm

Your JS script worked out-of-the-box, this is amazing!

My python code, does not have any env variables, it is only accepting 2 variables from the applescript (now the javascript) wrapper script.

Everywhere in the scripts I use absolute path, in the code snippet above, I've replaced /User/myUser with ~ for privacy reasons, but I'm always trying to use absolute paths, not sure if this is the reason you said - "use the full path".

I still does not understand why the manual run (via the Run Script) was working, and when triggered via the shortcut it does not.

Thank you once again! Really appreciate it!

Andreas_Hegenberg · November 19, 2024, 8:48pm

Nice!

I was just referring the call to python3, which may or may not work. Better use /usr/bin/python3

Btw. I'm always using JS when somehow possible because I really dislike Apple Scripts syntax
You can still call Apple Script from BTT's JavaScript if necessary, e.g.

let appleScript = `
    set theDialogText to "The curent date and time is " & (current date) & "."
    set result to display dialog theDialogText
    return result
`;

let result = await runAppleScript(appleScript);

xidiot · November 19, 2024, 8:51pm

I have 0 knowledge in applescript and javascript, but decided to go with applescript hoping that it would cause less issue, since it is "native" to macOS.

Andreas_Hegenberg · November 19, 2024, 8:53pm

Yes you will definitely need Apple Script in some places... but I try to only use it when required for UI scripting. ChatGPT is also much better at generating Java Script (you just need to tell ChatGPT about BTT's functions to call Apple Script or Shell Scripts (runAppleScript and runShellScript and the get / set variable functions - Using Java Script (not JXA) · GitBook)

fortred2 · November 20, 2024, 3:36am

@Andreas_Hegenberg @xidiot It helps ChatGPT if you provide the BTT docs as Markdown text. I created a Custom Context Menu that displays the different formats of the current clipboard (dynamically via JavaScript) and shows an option to convert the HTML format of the clipboard as Markdown. Now, I have really good conversations with ChatGPT about BTT's docs

I'll share a preset soon.

Andreas_Hegenberg · November 20, 2024, 8:03am

that’s great. I’m currently working on a markdown definition of all of BTTs actions, including all their possible JSON properties. My hope is this will allow ChatGPT to automatically generate full BTT action sequences.

fortred2 · November 20, 2024, 2:09pm

fortred2 · November 20, 2024, 2:14pm

@Andreas_Hegenberg As soon as you share the Markdown definition of all of BTT's actions, I'll start working on producing a JSON Schema and OpenAPI specification. While documentation in Markdown works, JSON Schema / OpenAPI spec is much more reliable.

Andreas_Hegenberg · November 20, 2024, 2:17pm

I think that will be hard for many actions, but yes for many it will be great to have (and maybe chatgpt can produce something like this mostly by itself once it got the markdown)

There is lots of crazy stuff like this that has somehow evolved during the last 15 years

fortred2 · November 22, 2024, 3:12am

@Andreas_Hegenberg OpenAI's Structured Output doesn't support all keywords in JSON Schema which is annoying, but I was still able to create a JSON Schema based on Draft 2020-12 that works quite well when tested with gpt-4o-2024-08-06.

What do you think?

Example Output

User Query:
middle click with shift, fn, control and left command.

Validated JSON:
{
  "BTTPredefinedActionType": 119,
  "BTTCustomClickConfig": {
    "clickType": "010",
    "modifierBitmask": {
      "Control": "Left",
      "Option": "None",
      "Command": "Left",
      "Shift": "Either Right or Left",
      "Function": "Function"
    }
  },
  "BTTCustomClickUpDownConfig": "0"
}

BTT JSON:
{
  "BTTPredefinedActionType": 119,
  "BTTCustomClickConfig": "0108519689",
  "BTTCustomClickUpDownConfig": "0"
}

The JSON Schema

{
	"$schema": "https://json-schema.org/draft/2020-12/schema",
	"title": "Create_Customizable_Click_Action_In_BetterTouchTool",
	"description": "Defines the schema for creating a customizable click action in BetterTouchTool.",
	"type": "object",
	"properties": {
		"BTTPredefinedActionType": {
			"type": "integer",
			"description": "The predefined action type for a customizable mouse click in BetterTouchTool. Must always be 119.",
			"const": 119
		},
		"BTTCustomClickConfig": {
			"type": "object",
			"description": "Specifies the click type and optional modifier keys.",
			"properties": {
				"clickType": {
					"type": "string",
					"description": "Defines the type of mouse click with semantic labels for each valid value.",
					"anyOf": [
						{
							"const": "100",
							"description": "Left click"
						},
						{
							"const": "010",
							"description": "Middle click"
						},
						{
							"const": "001",
							"description": "Right click"
						},
						{
							"const": "200",
							"description": "Double-click left"
						},
						{
							"const": "300",
							"description": "Triple-click left"
						},
						{
							"const": "-300",
							"description": "Click with button 3 (special)"
						},
						{
							"const": "500",
							"description": "Click with button 4"
						},
						{
							"const": "600",
							"description": "Click with button 5"
						},
						{
							"const": "700",
							"description": "Click with button 6"
						},
						{
							"const": "800",
							"description": "Click with button 7"
						},
						{
							"const": "900",
							"description": "Click with button 8"
						}
					]
				},
				"modifierBitmask": {
					"type": "object",
					"description": "Specifies active modifier keys in full detail, including left and right distinctions.",
					"properties": {
						"Control": {
							"type": "string",
							"description": "Indicates if the Control key is active. If the user mentions the Control key but doesn't specify which one, it must always be 'Either Right or Left'.",
							"anyOf": [
								{
									"const": "Either Right or Left Control",
									"description": "If the user mentions the Control key but doesn't specify which one."
								},
								{
									"const": "Left",
									"description": "If the user specifies the left Control key."
								},
								{
									"const": "Right",
									"description": "If the user specifies the right Control key."
								},
								{
									"const": "Both Left and Right",
									"description": "If the user specifies both Left and Right Control keys."
								},
								{
									"const": "None",
									"description": "If the user specifies no Control key."
								}
							]
						},
						"Option": {
							"type": "string",
							"description": "Indicates if the Option key is active. If the user mentions the Option key but doesn't specify which one, it must always be 'Either Right or Left'.",
							"anyOf": [
								{
									"const": "Either Right or Left",
									"description": "If the user mentions the Option key but doesn't specify which one."
								},
								{
									"const": "Left",
									"description": "If the user specifies the left Option key."
								},
								{
									"const": "Right",
									"description": "If the user specifies the right Option key."
								},
								{
									"const": "Both Left and Right",
									"description": "If the user specifies both Left and Right Option keys."
								},
								{
									"const": "None",
									"description": "If the user specifies no Option key."
								}
							]
						},
						"Command": {
							"type": "string",
							"description": "Indicates if the Command key is active. If the user mentions the Command key but doesn't specify which one, it must always be 'Either Right or Left'.",
							"anyOf": [
								{
									"const": "Either Right or Left",
									"description": "If the user mentions the Command key but doesn't specify which one."
								},
								{
									"const": "Left",
									"description": "If the user specifies the left Command key."
								},
								{
									"const": "Right",
									"description": "If the user specifies the right Command key."
								},
								{
									"const": "Both Left and Right",
									"description": "If the user specifies both Left and Right Command keys."
								},
								{
									"const": "None",
									"description": "If the user specifies no Command key."
								}
							]
						},
						"Shift": {
							"type": "string",
							"description": "Indicates if the Shift key is active. Cannot distinguish between left and right Shift keys. Must always be 'Either Right or Left Shift' or 'None'.",
							"anyOf": [
								{
									"const": "Either Right or Left",
									"description": "If the user mentions the Shift key but doesn't specify which one."
								},
								{
									"const": "None",
									"description": "If the user specifies no Shift key."
								}
							]
						},
						"Function": {
							"type": "string",
							"description": "Indicates if the Function key is active. There is only one Function key, so it must always be 'Function' or 'None'.",
							"anyOf": [
								{
									"const": "Function",
									"description": "If the user specifies the Function key."
								},
								{
									"const": "None",
									"description": "If the user specifies no Function key."
								}
							]
						}
					},
					"additionalProperties": false,
					"required": [
						"Control",
						"Option",
						"Command",
						"Shift",
						"Function"
					]
				}
			},
			"required": ["clickType", "modifierBitmask"],
			"additionalProperties": false
		},
		"BTTCustomClickUpDownConfig": {
			"type": "string",
			"description": "Specifies the click behavior with semantic labels for each valid value.",
			"anyOf": [
				{
					"const": "0",
					"description": "Perform a complete mouse-down and mouse-up sequence (normal click)."
				},
				{
					"const": "1",
					"description": "Perform only the mouse-down event (mouse remains pressed)."
				},
				{
					"const": "2",
					"description": "Perform only the mouse-up event (release the mouse)."
				}
			]
		}
	},
	"required": [
		"BTTPredefinedActionType",
		"BTTCustomClickConfig",
		"BTTCustomClickUpDownConfig"
	],
	"additionalProperties": false
}

Python script to load the JSON Schema, make an API query to OpenAPI with a user input describing the customizable click Action in BTT, and a conversion from the outputted JSON to the structure that BTT expects.

from openai import OpenAI
import json
from rich import print, print_json, pretty
from rich.traceback import install
from rich.console import Console

# Install rich traceback and pretty print for better debugging and output formatting
install()
pretty.install()

# Initialize a console object for rich output
console = Console()


def print_d(input_dict):
    """
    Prints a dictionary in JSON format using rich's print_json function.

    Args:
        input_dict (dict): The dictionary to be printed.
    """
    return print_json(data=input_dict)


# Initialize the OpenAI client
client = OpenAI()

# Path to the JSON schema file for BTT keyboard shortcut actions
btt_keyboard_shortcut_action_json_schema_path = (
    "btt_action_schema/241121_19_50_01_btt_customizable_click_action.schema.json"
)

# Load the JSON schema from the file
with open(btt_keyboard_shortcut_action_json_schema_path, "r") as f:
    btt_keyboard_shortcut_action_json_schema = json.load(f)

# Extract schema details
title = btt_keyboard_shortcut_action_json_schema["title"]
description = btt_keyboard_shortcut_action_json_schema["description"]
properties = btt_keyboard_shortcut_action_json_schema["properties"]
required = btt_keyboard_shortcut_action_json_schema["required"]

# Define the OpenAI function definition using the loaded schema
openai_function_def = {
    "type": "json_schema",
    "json_schema": {
        "name": title,
        "schema": {
            "type": "object",
            "properties": properties,
            "additionalProperties": False,
            "required": required,
        },
        "strict": True,
    },
}


def query(user_message):
    """
    Sends a query to the OpenAI API and returns the response content.

    Args:
        user_message (str): The message to be sent to the AI.

    Returns:
        str: The content of the AI's response, or None if an error occurs.
    """
    try:
        response = client.chat.completions.create(
            model="gpt-4o-2024-08-06",
            messages=[
                {
                    "role": "system",
                    "content": "You are a BetterTouchTool AI assistant.",
                },
                {"role": "user", "content": user_message},
            ],
            response_format=openai_function_def,
        )
        return response.choices[0].message.content
    except Exception as e:
        console.print(f"[bold red]Error:[/bold red] {e}")
        return None


def generate_btt_json(validated_json):
    """
    Converts a validated JSON object into the original JSON format required by BTT,
    using the updated schema for `modifierBitmask`.

    Args:
        validated_json (dict): The validated JSON object to be converted.

    Returns:
        dict: The JSON object formatted for BTT.
    """
    predefined_action = validated_json["BTTPredefinedActionType"]
    click_config = validated_json["BTTCustomClickConfig"]
    click_type = click_config["clickType"]
    up_down_config = validated_json["BTTCustomClickUpDownConfig"]

    # Process modifier bitmask
    modifier_bitmask = 0
    if "modifierBitmask" in click_config:
        modifiers = click_config["modifierBitmask"]
        # Define bitmask mapping based on updated schema and BTT documentation
        bitmask_mapping = {
            "Control": {
                "Left": 1 << 0,
                "Right": 1 << 13,
                "Both Left and Right": (1 << 0) | (1 << 13),
                "Either Right or Left Control": (1 << 0) | (1 << 13),
                "None": 0,
            },
            "Option": {
                "Left": 1 << 5,
                "Right": 1 << 6,
                "Both Left and Right": (1 << 5) | (1 << 6),
                "Either Right or Left": (1 << 5) | (1 << 6),
                "None": 0,
            },
            "Command": {
                "Left": 1 << 3,
                "Right": 1 << 4,
                "Both Left and Right": (1 << 3) | (1 << 4),
                "Either Right or Left": (1 << 3) | (1 << 4),
                "None": 0,
            },
            "Shift": {"Either Right or Left": 1 << 17, "None": 0},
            "Function": {"Function": 1 << 23, "None": 0},
        }
        # Compute the combined bitmask
        for key, value in modifiers.items():
            modifier_bitmask |= bitmask_mapping[key][value]

    # Combine click type and bitmask into a single string
    if modifier_bitmask > 0:
        btt_click_config = f"{click_type}{modifier_bitmask}"
    else:
        btt_click_config = click_type

    # Build the final BTT JSON
    btt_json = {
        "BTTPredefinedActionType": predefined_action,
        "BTTCustomClickConfig": btt_click_config,
        "BTTCustomClickUpDownConfig": up_down_config,
    }

    return btt_json


# Example user query
user_query = "middle click with shift, fn, control and left command."

# Send the query to the AI and get the response
content = query(user_query)

# Print the user query
print("User Query:")
print(user_query)
print()

# Convert the response content to a dictionary
validated_json = json.loads(content)

print("Validated JSON:")
# Print the validated JSON
print_d(validated_json)
print()

# Generate the BTT JSON from the validated JSON
btt_json = generate_btt_json(validated_json)

print("BTT JSON:")
# Print the final BTT JSON
print_d(btt_json)

fortred2 · November 22, 2024, 3:19am

(post deleted by author)