Replacing the Touch Bar with Floating Webviews

Overview
So I’ve started working on using Better Touch Tool's Floating Webviews to replace the Touch Bar, as my new computer does not have a Touch Bar. I know you could do a lot of this with something like the Stream Deck… but I like the idea of the Webviews being available on any computer I’m using, laptop or desktop, with no extra hardware required.

This also may end up allowing me to keep my hands on the keyboard more often, and who knows what other benefits in the future. Anyway, this is just the first experimental version, and I’m sharing this demo/prototype/unfinished preset before I go too far. That way it’ll be easier to incorporate any advice, feedback or suggestions you all might have, so please do share any and all thoughts or critiques etc.

With that said, here is a basic overview video of some of the prototype Webviews (emphasis on prototype/example/demonstration):

Controls/Workflow
The basic idea is very similar to how you’d use the Touch Bar, or Stream Deck. In this case when you press the designated key, a unique Webview shows up, per application. So the webview and its layout/buttons/functions/etc can change depending on what app you’re currently using. This is all with you only having to press and remember 1 button, and requires you to remember 0 keyboard shortcuts beyond that.

The basic workflow is:

  1. Hold down the designated key (caps lock in this case)
  2. Move the mouse and hover over desired button/box
  3. Release the designated key

So with this setup you release the keyboard key to ‘click’ the button… you don’t have to actually click with the mouse… which I find especially nice when using the trackpad. The webview starts centered under the cursor, so the direction you move the mouse stays consistent. I’ve also got basic keyboard control set up, so you can navigate the button grid with the arrow keys. The JKLI keys are also set up to work as arrow keys, although these work less consistently for me, I think they’re conflicting with my Karabiner setup.

Notes about the Preset and Setup
Caps Lock to F18 - caps lock is the main key used in this preset, I used Karabiner to map caps lock to F18 in order to get this to work without the caps lock light constantly turning on and off. So you need to set this up for this preset to work, alternatively you can change the Better Touch Tool to use caps lock directly, if you don’t care about the caps lock light turning on and off.

Visual Studio Code - to get all the buttons of the Visual Studio Code webview working, there are a couple keyboard shortcuts you need to add to Visual Studio Code itself, as they are unbound by default:

Blender - The Blender webview triggers only seem to work if I put them in the Blender App Specific settings of BTT, and also check “Shortcut Sending Compatibility”. Also, Blender has its own floating menus, so this Blender webview is mostly just for testing purposes right now. Having said that, the Blender webview needs a couple things to get set up:

  • Lock Camera to View -> F13 (add this to Blender using one of these methods)
  • For the Blender ‘Render View’ button to work, turn off the F11 -> Show Desktop shortcut in System Preferences.

Window Management and Global

  • These 2 webviews use different keys than the others, F3 and F4 in this case. The way it’s set up, you’re basically sacrificing F3 and F4 (mission control and spotlight) which I never use anyways. So if you’re not ok with that you can use other keys/shortcuts, or just turn these webviews off altogether.
  • There are 3 empty buttons in the Global webview (Blue, Orange, Green) that are set up to be easily customized using only Better Touch Tool. I would suggest these as a good place to start experimenting by adding your own actions to these buttons. (Also, the center date button should copy the current date to the clipboard)

Problems/Notes/Other

  • Focus - There seem to be some issues with focus, with multiple windows for example. If you have 2 Visual Studio Code windows open, the webview commands might be sent to the wrong window. The same thing can end up happening with tabs, multiple finder tabs for example. This can end up being pretty confusing.

  • Window Management - There can be some issues with windows not responding to the window moving/resizing commands. I’ve noticed this a lot with Safari recently, this is quite possibly related to the focus issues stated above.

  • Outside Buttons - I made the outer buttons bigger so they’d be easier mouse targets. This way I can move faster with the mouse, and not have to be so precise. The size of these buttons can of course be changed if desired. I also made it so you can ‘overshoot’ the outside buttons with the mouse, again in the interest of speed.

  • Trackpad Navigation - There are some disabled trackpad options in the BTT preset I’m sharing. Enabling these will turn 2/3 finger swipes into arrow keys, allowing you to use 2/3 finger swipes to navigate the grid. This is pretty cool, but enabling these will mess with the normal operation of the BTT UI though, so do so at your own risk.

  • Global Hot Corner - Also disabled in the preset is an option to show the Global webview when you move the mouse into the top left corner of the screen. If you hold command when moving the mouse away from the corner, the webview will stay visible, allowing you to select a button with the mouse. Alternatively you could use the arrow keys while the webview is visible here, and press enter on the keyboard to select a button

Here's the Preset:
Floating Webview.bttpresetzip (26.3 KB)

Final Thoughts

I’d just like to say thank you to @Andreas_Hegenberg for making Better Touch Tool. The fact that I can even attempt something like this with Better Touch Tool, says a lot about how capable it is. I wouldn’t want to use my Mac without it.

That's cool, thanks for sharing!

Since you're asking for feedback, here's what I can tell you.

  1. Andreas is working on floating menus, which should be very powerful. I assume everything you have done will work with these menus and will be easier to set up.

  2. I think a part (or everything?) can already be set up with the stream deck emulatur.

Basically I'm not sure if the idea is suitable as a replacement for the Touch Bar (or a Stream Deck). The concept there is "the finger goes to the icon/button". Not, the mouse goes somewhere.

If you have a Macbook without a Touch Bar, you can use the free app Touché. The app simulates the Touch Bar on the screen. And with BTT everything can be configured like with the real Touch Bar. If the Touché window is placed exactly above the F-Keys, it can be used to simulate the Touch Bar very well, even with physical keys like a Stream Deck. I do it this way and find it the best I have ever come up with.

So, if I were you, I'd wait for these new menus before you get into too much work. Just my 2 cents :slightly_smiling_face:

1 Like

Looks really nice!

@Frank1 If you know html & js the floating webviews will always be the most flexible & performant way to implement such stuff though. Many things will be possible with the floating menus, but the webview has the big advantage of being able to run Java Script directly. So you can basically program anything you want.

2 Likes

What is a "Webview"? I haven't encountered this term before, so it would be cool if your great article included a short sentence to introduce this term. (I know what "Floating" is, though.)

In case it is a BTT feature that has this name: I am new to BTT and use it in a quite a special way (for "Generic Devices"), so please bear with me.

It's basically a customizable window that contains a little web browser which has been augmented by BetterTouchTool specific features. For example you can easily trigger any trigger set up in BTT, run shell or Apple scripts and so on.

The UI is completely customizable using HTML and CSS

1 Like

Hey thanks for the feedback, and your thoughts in general. I’ll have to check out some of those suggestions you mentioned.

As for the upcoming floating menus, I was having very similar thoughts, wondering if I should continue working on this project or not….as it’s starting to become a decent amount of work. That’s part of my motivation for putting this preset out a little early/unfinished, I wanted to get a feel for whether or not this project is worth pursuing further.

Based on Andreas’s comment below, I think I’ll keep working on it for now, as performance is one of my main focuses here. But I imagine I’ll end up using both the floating webviews and menus at the end of the day.

This comment, if I understand it correctly, is kind of what I’m trying to address with the arrow key controls, and also the 2/3 finger swipe controls.

If you do end up trying the preset out, I’d suggest trying the window management webview with the keyboard controls. Using the webviews in this way, you don’t have to use the mouse at all, so no searching for buttons with the mouse cursor. Also the JKLI keys as arrow keys will allow you to keep your hands on the ‘home row’ of the keyboard more often, and additional keyboard controls can be added in the future.

Thanks again for the comments, and if you do try out the preset, I’d be curious to hear what you think of the ‘tactile experience’ of the keyboard and 2/3 finger trackpad swipe controls.

1 Like

Thanks! It’s interesting to get clarification on the performance aspect of this. I was actually thinking the opposite, that your upcoming floating menus might be more flexible & performant. With that in mind I was thinking maybe I should just wait for your upcoming floating menus. But I guess for now I’ll carry on with this project.

@ocean-success I think it's great if you keep working. This work can also be fun, right? :smiley:

I understand that with your system you can use the mouse or the keyboard. But it's not the same. It's not your fault, but if you don't have a touch screen on your Macbook (which you don't have because Apple doesn't produce them) it's not possible to implement the concept "the finger goes to the icon/button". This intuitive behavior (which every child understands immediately) only works with the Touch Bar or a Stream Deck if you're willing to work with external hardware, which I'm not. That's why I do it with Touché and the F-Keys. :slightly_smiling_face:

Hey, good call.
I added a link in the first paragraph to the Better Touch Tool documentation for the Floating WebViews. I also indicated that they are in fact Better Touch Tool’s Floating Webviews, and not any other type of floating thing.

1 Like

This is brilliant work .I've had a play around and the window-manager alone is fantastic.

@oceansuccess I'm wondering whether you could release a blank template where I could add my better touch tool trigger for each app I'm working in.

1 Like

Thanks, that’s good to hear!

Yes, that’s something I’m working on, a general preset that’s more easily customizable. I have a couple thoughts/questions regarding that:

  1. Are you familiar with JavaScript/HTML/CSS at all? If so it would be fairly straightforward to duplicate some files and edit them yourself, if so I can point you in the right direction.
  2. More long term, I’m looking into making a general preset that could be customized using BTT alone, no JavaScript needed. This will take some time though, and I don’t know for sure that it’s even possible/practical.

Anyway, maybe I’ll start looking into a more short term, basic blank template.

I’ve made some progress on this project and have an updated preset to share. It’s still a little rough around the edges, but this version should be easier to customize.

Changes:

  1. Modifier Support - You can now press Command or Option to show an alternate layout. The “S” and “A” keys also work as Command and Option.
  2. General Template - added a webview which is easier to customize, it is set to F5 in this preset. It doesn’t do anything by default, it has placeholder text, formatting and colors for demonstration purposes. (more info below)
  3. Window Management - updated this webview with an alternate layout to demonstrate modifier key support. The “4” and “5” keys can be used to show the alternate layouts, as well as Command and Option.
  4. Blender Webview - removed this webview as I was having problems getting it working consistently.

Notes/Problems:

  1. Key Repeat - It seems that in order to get the new modifier keys working consistently, key repeat has to be OFF. I’ve also noticed that, my key repeat settings are reverting back to ON sometimes. So if you’re having problems with the modifier keys, this might be the issue.
  2. Old Preset - A number of fundamental things were changed for this preset, so you probably should disable the original preset, or delete it entirely before using this new version.

How to customize the General Template
Here I’ll show how to make a new webview, for whatever app you want, using the General Template. In this example I’ll create a new webview for the default Notes app. Make sure to look at the images below for the full details.

  1. Set this preset (Floating Webview TouchBar Replacement) as the BTT master preset, this will make sure everything we add next gets added to the correct preset.
  2. Duplicate and Rename the General Template html file.
  3. Add a named trigger for this newly duplicated webview. You can copy and paste, and then rename the General Template trigger to accomplish this.
  4. Add an app specific keyboard shortcut for the Notes app, so we can trigger the Notes webview while using Notes. F18 should trigger a Named Trigger with the trigger name: “Notes_FloatingWebView”
  5. Verify that the Notes webview appears when you press caps lock while using the Notes app. It will look identical to the General Template, and it won’t do anything yet, in this next step we’ll make it do something.
  6. Add a trigger named “Notes_default_box04” and set it to command-option-s. Now box 4 of the Notes webview should be functional, when clicked it should “Show Folders” in the notes app.
  7. To change the text of box 4 we need to edit the “Notes_FloatingWebview.html” file that we created in step 2. At line 108 the text can be changed from “4” to “Show Folders”, color (blue, green, orange, red) can be changed here as well.





At this point you should be able to add new webviews for any app you want, by duplicating and customizing the General Template. When making new webviews, you need to use the following format when naming the files and triggers.

File Name Format: AppName_FloatingWebview.html
Trigger Name Format: AppName_layout_buttonID

For example, if you wanted to make a new webview for Final Cut Pro:

File Name:
FinalCutPro_FloatingWebview.html

Trigger Names:
FinalCutPro_default_box01
FinalCutPro_default_box02 ...etc
FinalCutPro_default_box19
FinalCutPro_command_box01 ...etc
FinalCutPro_command_box19
FinalCutPro_option_box01 ...etc
FinalCutPro_option_box19

@james948 This second preset should allow you to customize the webviews for yourself, as well as add new webviews per application. You will have to edit some basic JavaScript to change the text and colors of the webview buttons/boxes. Let me know if you have any questions.

Preset:
Floating Webview TouchBar Replacement.bttpresetzip (35.4 KB)

WOW I can't wait to give this a whirl! Shall report back!

1 Like

@james948 I’m looking forward to hearing your thoughts, let me know if the tutorial/example was clear enough.

As far as the next version goes, I’m thinking about adding icons:

@Andreas_Hegenberg I’m wondering if it’s possible to use SF Symbols in the webview, in the same way they’re being used elsewhere in Better Touch Tool?

I think if you have the sf symbol app installed you can just copy the symbols from there into your HTML. However they will then only show if the SF Symbols app is installed.

BTT also has an internal retrieve_sf_symbol function, which I could expose to the webview (that returns a image and works everywhere). I'll add that to the next alpha.

Awesome! Both methods sound good to me, I’ll try out the first option soon.

Does the second option with the retrieve_sf_symbol function require the SF Symbols app to be installed? Just wondering, as it might be ideal for preset sharing if the additional app wasn’t required.

Either way, thanks! I’m looking forward to the alpha.

no with that method the sf symbols app is not needed :slight_smile:

Nice! :+1:

Great job ocean! Love it. I have yet to play with it, but webviews are always delicious and it seems to me their potential is still a bit untapped. Thankfully we have folks that do like working with it and producing new things. Love the innovation. Cheers.

Will try this out tomorrow after setting up the preset and customizing it :facepunch:

Edit: also this doesn't have to replace the touchbar for those of us that still have a touchbar :sunglasses:.

This can just be an addition that extends the possibilities and can be integrated with different apps.

1 Like

Hey, Thanks!

That’s a good point about this not having to replace the Touch Bar, I’m sure they could work together nicely. I’d probably use both myself as I do like the Touch Bar, but sadly I don’t have one anymore :frowning:.

With both, you could keep using the Touch Bar normally, then add Floating Webviews where they make sense and/or have advantages over the Touch Bar.

A few possible advantages might be:

  • More Flexible UI - UI can be any size/shape/layout you want
  • Home Row Keyboard Control - using JKLI keys to navigate the grid keeps your hands on the "home row" of the keyboard more often
  • Additional Modifier Keys - basically any key on the keyboard can act as a modifier key, as seen in the window management webview, with the “4” and “5” keys showing alternate layouts

Anyway, let me know if you run into any problems getting set up, I’m sure I could improve the instructions.

1 Like