Tap/doing things on non active window

One of my favorite parts of macOS is that you can scroll on a non active window. I would really love to be able to use my tip tap tab changing setup on any window I'm hovering over rather than needing to click the window first.

For example. I use single finger down and tap the finger next to it to change tabs left and right respectively. If I go and answer a message... then go back to my safari and scroll around THEN attempt to change tabs... it switches the tab in messages because technically its still active.

I guess this might be a feature request if its not something that possible now. Thanks!