Replacing the Touch Bar with Floating Webviews

Hey thanks for the feedback, and your thoughts in general. I’ll have to check out some of those suggestions you mentioned.

As for the upcoming floating menus, I was having very similar thoughts, wondering if I should continue working on this project or not….as it’s starting to become a decent amount of work. That’s part of my motivation for putting this preset out a little early/unfinished, I wanted to get a feel for whether or not this project is worth pursuing further.

Based on Andreas’s comment below, I think I’ll keep working on it for now, as performance is one of my main focuses here. But I imagine I’ll end up using both the floating webviews and menus at the end of the day.

This comment, if I understand it correctly, is kind of what I’m trying to address with the arrow key controls, and also the 2/3 finger swipe controls.

If you do end up trying the preset out, I’d suggest trying the window management webview with the keyboard controls. Using the webviews in this way, you don’t have to use the mouse at all, so no searching for buttons with the mouse cursor. Also the JKLI keys as arrow keys will allow you to keep your hands on the ‘home row’ of the keyboard more often, and additional keyboard controls can be added in the future.

Thanks again for the comments, and if you do try out the preset, I’d be curious to hear what you think of the ‘tactile experience’ of the keyboard and 2/3 finger trackpad swipe controls.

1 Like

Thanks! It’s interesting to get clarification on the performance aspect of this. I was actually thinking the opposite, that your upcoming floating menus might be more flexible & performant. With that in mind I was thinking maybe I should just wait for your upcoming floating menus. But I guess for now I’ll carry on with this project.

@ocean-success I think it's great if you keep working. This work can also be fun, right? :smiley:

I understand that with your system you can use the mouse or the keyboard. But it's not the same. It's not your fault, but if you don't have a touch screen on your Macbook (which you don't have because Apple doesn't produce them) it's not possible to implement the concept "the finger goes to the icon/button". This intuitive behavior (which every child understands immediately) only works with the Touch Bar or a Stream Deck if you're willing to work with external hardware, which I'm not. That's why I do it with Touché and the F-Keys. :slightly_smiling_face:

Hey, good call.
I added a link in the first paragraph to the Better Touch Tool documentation for the Floating WebViews. I also indicated that they are in fact Better Touch Tool’s Floating Webviews, and not any other type of floating thing.

1 Like

This is brilliant work .I've had a play around and the window-manager alone is fantastic.

@oceansuccess I'm wondering whether you could release a blank template where I could add my better touch tool trigger for each app I'm working in.

1 Like

Thanks, that’s good to hear!

Yes, that’s something I’m working on, a general preset that’s more easily customizable. I have a couple thoughts/questions regarding that:

  1. Are you familiar with JavaScript/HTML/CSS at all? If so it would be fairly straightforward to duplicate some files and edit them yourself, if so I can point you in the right direction.
  2. More long term, I’m looking into making a general preset that could be customized using BTT alone, no JavaScript needed. This will take some time though, and I don’t know for sure that it’s even possible/practical.

Anyway, maybe I’ll start looking into a more short term, basic blank template.

I’ve made some progress on this project and have an updated preset to share. It’s still a little rough around the edges, but this version should be easier to customize.

Changes:

  1. Modifier Support - You can now press Command or Option to show an alternate layout. The “S” and “A” keys also work as Command and Option.
  2. General Template - added a webview which is easier to customize, it is set to F5 in this preset. It doesn’t do anything by default, it has placeholder text, formatting and colors for demonstration purposes. (more info below)
  3. Window Management - updated this webview with an alternate layout to demonstrate modifier key support. The “4” and “5” keys can be used to show the alternate layouts, as well as Command and Option.
  4. Blender Webview - removed this webview as I was having problems getting it working consistently.

Notes/Problems:

  1. Key Repeat - It seems that in order to get the new modifier keys working consistently, key repeat has to be OFF. I’ve also noticed that, my key repeat settings are reverting back to ON sometimes. So if you’re having problems with the modifier keys, this might be the issue.
  2. Old Preset - A number of fundamental things were changed for this preset, so you probably should disable the original preset, or delete it entirely before using this new version.

How to customize the General Template
Here I’ll show how to make a new webview, for whatever app you want, using the General Template. In this example I’ll create a new webview for the default Notes app. Make sure to look at the images below for the full details.

  1. Set this preset (Floating Webview TouchBar Replacement) as the BTT master preset, this will make sure everything we add next gets added to the correct preset.
  2. Duplicate and Rename the General Template html file.
  3. Add a named trigger for this newly duplicated webview. You can copy and paste, and then rename the General Template trigger to accomplish this.
  4. Add an app specific keyboard shortcut for the Notes app, so we can trigger the Notes webview while using Notes. F18 should trigger a Named Trigger with the trigger name: “Notes_FloatingWebView”
  5. Verify that the Notes webview appears when you press caps lock while using the Notes app. It will look identical to the General Template, and it won’t do anything yet, in this next step we’ll make it do something.
  6. Add a trigger named “Notes_default_box04” and set it to command-option-s. Now box 4 of the Notes webview should be functional, when clicked it should “Show Folders” in the notes app.
  7. To change the text of box 4 we need to edit the “Notes_FloatingWebview.html” file that we created in step 2. At line 108 the text can be changed from “4” to “Show Folders”, color (blue, green, orange, red) can be changed here as well.





At this point you should be able to add new webviews for any app you want, by duplicating and customizing the General Template. When making new webviews, you need to use the following format when naming the files and triggers.

File Name Format: AppName_FloatingWebview.html
Trigger Name Format: AppName_layout_buttonID

For example, if you wanted to make a new webview for Final Cut Pro:

File Name:
FinalCutPro_FloatingWebview.html

Trigger Names:
FinalCutPro_default_box01
FinalCutPro_default_box02 ...etc
FinalCutPro_default_box19
FinalCutPro_command_box01 ...etc
FinalCutPro_command_box19
FinalCutPro_option_box01 ...etc
FinalCutPro_option_box19

@james948 This second preset should allow you to customize the webviews for yourself, as well as add new webviews per application. You will have to edit some basic JavaScript to change the text and colors of the webview buttons/boxes. Let me know if you have any questions.

Preset:
Floating Webview TouchBar Replacement.bttpresetzip (35.4 KB)

WOW I can't wait to give this a whirl! Shall report back!

1 Like

@james948 I’m looking forward to hearing your thoughts, let me know if the tutorial/example was clear enough.

As far as the next version goes, I’m thinking about adding icons:

@Andreas_Hegenberg I’m wondering if it’s possible to use SF Symbols in the webview, in the same way they’re being used elsewhere in Better Touch Tool?

I think if you have the sf symbol app installed you can just copy the symbols from there into your HTML. However they will then only show if the SF Symbols app is installed.

BTT also has an internal retrieve_sf_symbol function, which I could expose to the webview (that returns a image and works everywhere). I'll add that to the next alpha.

Awesome! Both methods sound good to me, I’ll try out the first option soon.

Does the second option with the retrieve_sf_symbol function require the SF Symbols app to be installed? Just wondering, as it might be ideal for preset sharing if the additional app wasn’t required.

Either way, thanks! I’m looking forward to the alpha.

no with that method the sf symbols app is not needed :slight_smile:

Nice! :+1:

Great job ocean! Love it. I have yet to play with it, but webviews are always delicious and it seems to me their potential is still a bit untapped. Thankfully we have folks that do like working with it and producing new things. Love the innovation. Cheers.

Will try this out tomorrow after setting up the preset and customizing it :facepunch:

Edit: also this doesn't have to replace the touchbar for those of us that still have a touchbar :sunglasses:.

This can just be an addition that extends the possibilities and can be integrated with different apps.

1 Like

Hey, Thanks!

That’s a good point about this not having to replace the Touch Bar, I’m sure they could work together nicely. I’d probably use both myself as I do like the Touch Bar, but sadly I don’t have one anymore :frowning:.

With both, you could keep using the Touch Bar normally, then add Floating Webviews where they make sense and/or have advantages over the Touch Bar.

A few possible advantages might be:

  • More Flexible UI - UI can be any size/shape/layout you want
  • Home Row Keyboard Control - using JKLI keys to navigate the grid keeps your hands on the "home row" of the keyboard more often
  • Additional Modifier Keys - basically any key on the keyboard can act as a modifier key, as seen in the window management webview, with the “4” and “5” keys showing alternate layouts

Anyway, let me know if you run into any problems getting set up, I’m sure I could improve the instructions.

1 Like

I’ve been working on converting this preset from Floating Webviews to the newer Floating Menus. It’s been going pretty well, so I thought I’d share a mostly working first draft of this preset. It can’t do everything the Webview preset could, but it’s reasonably functional so far.

I think the main advantage of using Menus instead of Webviews, is the ease of customization. The Menus can be fully customized with BTT, without needing to edit html/css/javascript files. This should make it far easier/faster for you to customize these for yourself.

Notes

  • App Launcher - I added a new App Launcher Menu to F4, this can be used as an app launcher or switcher. An advantage of this is that the apps always stay in the same order. Combined with keyboard or trackpad control, this can be an interesting way to launch/switch apps.

  • Keyboard Control - Using this method, I’ve got basic keyboard control working. Arrow keys can be used to navigate the button grid. The I-J-K-L and 1-Tab-Q-W keys can also be used as arrow keys. (The 1-Tab-Q-W keys are a little weird as arrow keys, but I find them to be useful when showing a Menu via the trackpad or mouse)

  • Trackpad Control - 2/3 finger swipes can be used to navigate the button grid. 2 finger swipe = move 1 button, 3 finger swipe = move 2 buttons. Additionally, 2/3/4 finger Force Clicks will show various Menus, I’m using the Force Clicks with the 1-Tab-Q-W keys as arrow keys.

  • Mouse Control - I disabled these in the preset as they use right click to show the App Specific Menus, which is probably a dealbreaker for a lot of people. Having said that, if you have some other way to right click, using right click on a mouse to show the App Specific Menus is pretty cool when combined with the 1-Tab-Q-W arrow keys.

  • Copy/Paste Menu - The idea of these is to quickly copy some text you use frequently to the clipboard. Because of the keyboard controls, you can’t really type into these directly, so it’s easiest to copy text from somewhere else first, then right click to paste the text into the text box.

Problems

  • Modifier Keys - Menus are getting stuck in their alternate layout if the menu closes while on the alternate layout. I don’t know why this is happening, but the next time you open the menu, pressing the modifier key will get it back to its default state.

Setup

  • Caps Lock to Control - In order to get the App Specific Menus working, you need to change Caps Lock to Control in the macOS System Settings app.

Preset:

Floating Menu - TouchBar Replacement.bttpreset.zip (6.4 MB)

2 Likes

very cool, I’ll experiment with it later today!

1 Like

I need to reply here just to say how great this idea is, @ocean-success. Nice!!!

1 Like

@Andreas_Hegenberg Nice! If you have any thoughts after using it, I’d enjoy hearing them.

@mkozjak Thanks! These types of comments help encourage me to keep working on this.

1 Like

I’ve been playing around with the idea of using the Floating Menus as “virtual modifiers” … and combining that with the concept of keyboard chords. With those 2 ideas in mind, I have a little update here to the App Launcher Floating Menu.

I added the App Launcher Menu to Option + Tab, and it now has two modes.

First Mode
The first mode works the same as the App Launcher normally would on F4, and also works similarly to the macOS Command + Tab switcher.

First Mode Instructions:

  1. Option down
  2. Tab down (menu shows here)
  3. Tab up
  4. Select menu item (with arrow keys, trackpad or mouse controls)
  5. Option up (this triggers the menu item, and closes menu)

Second Mode
The second mode works more like a keyboard chord, where you show the Floating Menu first, release all keys, and while the menu is visible use keyboard shortcuts to launch apps. For example S to launch Safari or N to launch Notes.

Where it gets more interesting (and more complex / convoluted) is while in this second mode, you can use normal keys as if they are modifier keys. So holding S and then pressing E will launch System Settings for example, holding S then pressing C will open the Shortcuts app, hold S then press I to open the System Information app…etc.

Second Mode Instructions:

  1. Option down
  2. Tab down
  3. Option up (this is important, you need to release Option before Tab to enter the 2nd mode)
  4. Tab up
  5. Launch app with shortcut, such as: (S: Safari) (N: Notes) (C + D: Calendar) etc...

These shortcuts are only active when the Floating Menu is visible, in that sense the Floating Menu itself is the modifier. This is pretty nice, and I think this idea has some potential, so I’ll keep exploring it. For now, here’s the first working draft of this concept:

Preset: (this is an updated version of the whole preset by the way, not just the App Launcher)
Floating Menu - TouchBar Replacement - January 31, 2024.bttpreset.zip (9.2 MB)

2 Likes