Emojis turn into

Bugs or Unexpected Behavior
1

Using an apple script on a cron to update a .txt file based on json endpoint data output

set targetFile to "FILE_PATH/file.txt"
set webhookUrl to "JSON_ENDPOINT"

do shell script "curl -s '" & webhookUrl & "' | jq -r '.txt // .' | awk 'BEGIN {print \"async function retrieveJSON() {\\nlet items = \"} {print $0} END {print \"; \\n\\n return JSON.stringify(items);}\"}' > " & quoted form of targetFile

The JSON output has Emojis
That file is the source for a JSON Menubar Custom Context Menu

First update of the file using the script, I can see the Emojis, as soon as I connect it with BTT for that Context menu, the emojis stop rendering properly:

:green_circle: => "🟢"
⊙ => "‚äô"

Is there a Solution for this? thanks!

2

Try forcing it into UTF-8:

set targetFile to "FILE_PATH/file.txt"
set webhookUrl to "JSON_ENDPOINT"

do shell script "curl -s '" & webhookUrl & "' \
| jq -r '.txt // .' \
| awk 'BEGIN {print \"async function retrieveJSON() {\\nlet items = \"} {print $0} END {print \"; \\n\\n return JSON.stringify(items);}\"}' \
| iconv -f UTF-8 -t UTF-8 > " & quoted form of targetFile
1 Like
3

The issue isn't with the script. It only occurs with files linked to BTT. When I use a completely new TXT file in the Downloads folder, emojis display fine. However, once I link it with the BTT custom menu, the emojis stop rendering correctly.

4

you mean it renders incorrectly in BTT, or does the actual file change? (BTT doesn’t modify your files)

Have you tried changing the script as shown above? BTT requires UTF-8, other file encodings will render special characters incorrectly.

If you mean running the script outside of BTT works, but running inside of BTT - that is possible because BTT doesn’t read your environment variables by default. These include the default encoding

1 Like
5

I run the script inside BTT

The file contents have the emojis working using my original script

As soon as I select that file as a source for the custom meny inside BTT, it swicthes, I emojis no longer render properly

I tried the script your suggested, but that's not really the issue apparently, since it works fine until I connect it to BTT, then it doesn't

6

can you try this command on the (target) file (in your terminal) before and after?

file -I yourfile.txt

7

Tested on 2 files

Current file I've been using in BTT: [this file now also shows utf-8 for some reason..]
before connecting to BTT : text/plain; charset=utf-8 (Emojis working fine)
after connecting to BTT : text/plain; charset=us-ascii (Emojis not rendering)

New file I've created and linked to BTT:
before connecting to BTT : text/plain; charset=utf-8 (Emojis working fine)
after connecting to BTT : text/plain; charset=utf-8 (Emojis not rendering)

Note: When I open the txt file using sublime text, it shows the emojis no problem.

8

5.318 alpha forces everything to be utf8, that should workaround the issue (uploading now)

9

Yup, it fixed the problem for me

Thanks!