Raise / Lift All fingers before "Two Finger Swipe" (Magic Mouse)

I've been having issues with false positives/accidentally triggering two finger swipes after two fingers have been sitting for a little while and then I raise them to click.

I'm using a two finger two swipe left and right to move between desktops/spaces

I think it would be helpful to have a "Raise / Lift All fingers before Two Finger Swipe" similar to the feature that already exists for clicks on the magic mouse.

Troubleshooting that I've already tried:
Decreasing the swipe sensitivity - makes it too difficult to trigger a two finger swipe, and I end up moving the whole mouse while I try to swipe.