BTT web server doesn't allow query parameter spaces encoded with +

Describe the bug

It appears that the BTT web server only recognises spaces in query parameters when they're encoded as "%20", not when they're encoded as "+", which introduces problems when e.g. trying to trigger a named trigger via curl, particularly when the trigger name is parameterised (see examples below).

W3C recommends that query strings be encoded in the format defined for application/x-www-form-urlencoded, which says:

Space characters are replaced by +, and then reserved characters are escaped as described in [RFC1738], section 2.2

Looking around on Stack Overflow, etc. it seems that by convention many tools also accept %20 for spaces in query parameters — but if only one way is supported, The Right Thing is "+", as that's what that standard calls for.

Of course, BTT may not be seeking to adhere to that standard, which is fair enough — but it is probably what people expect when dealing with query strings, and in particular many tools (notably curl) do this, so it's probably worth supporting (both ways).

Affected input device

BTT web server

Device information

  • Type of Mac: MacBook Pro (13-inch, M1, 2020); Apple M1; 16GB
  • macOS version: Monterey, 12.3 (21E230)
  • BetterTouchTool version: 3.748 (1890)
  • curl version: curl 7.79.1 (x86_64-apple-darwin21.0) libcurl/7.79.1 (SecureTransport) LibreSSL/3.3.5 zlib/1.2.11 nghttp2/1.45.1

Examples

"Direct" embdedding

Suppose I have a named trigger called "A B", and that BTT's web server is listening on port 49326.

Then, the following curl invocation immediately gets a "200 OK" response and successfully runs the named trigger.

curl "http://127.0.0.1:49326/trigger_named/?trigger_name=A%20B"

... whereas the following takes ~5 seconds to get a "200 OK" response, but doesn't run the named trigger.

curl "http://127.0.0.1:49326/trigger_named/?trigger_name=A+B"

Parameterised trigger names and curl's --data-urlencode option

This makes life hard particularly when the trigger name is parameterised... curl tries to make this easy for us: its --data-urlencode option will do the encoding for us, so we don't have to.

Thus, this "should" work, and would be ideal:

name="A B"
curl -G "http://127.0.0.1:49326/trigger_named/" --data-urlencode "trigger_name=$name"

Alas it doesn't work, because curl encodes the query string following the application/x-www-form-urlencoded standard mentioned previously. In fact, it's functionally entirely equivalent to the second example above: it hits exactly the same URL.

As it happens curl did previously use "%20" here, and there are numerous blog posts, SO answers, etc. referencing this — but as of issue 3229 and curl v7.64.0, released Feb 2019, it uses "+".

Workaround for curl / BTT use case

As far as I can see there's no way to tell curl to encode spaces as "%20" instead of "+". So we need to encode the trigger name ourselves, forget about --data-urlencode entirely, and embed it directly in the URL.

E.g. this works:

arg=$(jq -rn --arg x '$name' '$x|@uri') && curl "http://127.0.0.1:49326/trigger_named/?trigger_name=$arg"

This works because jq / @uri only does %-encoding, including of spaces; if jq starts using "+" there, this will stop working too of course. :slight_smile:

As it happens integrating that "fix" into the rest of my toolchain — in this case goku (a Karabiner Elements preprocessor) is proving very awkward thanks to goku's import format and the familiar headaches of properly escaping things when you've got one language embedded in another in another, etc.... I haven't figured out a good way to do it yet, and I'll probably end up forgetting about curl entirely and just using python to bring goku and BTT together... :upside_down_face:

Suggested Fix

Allow the BTT web server to recognise both "%20" and "+" as space in query strings; it would then need to interpret "%2B" as a literal "+" to cover that case, but I suspect it already does...?

Thanks for reporting, I'll check how to fix this. I'm probably using the wrong encoding/decoding function :slight_smile:

1 Like