How to disable trackpad drag recognition after swipe gestures

I'm using horizontal four-finger swipe gestures to move between spaces with yabai (replacing the native behavior). When I lift my hand, there's often a split second where I have two fingers on the trackpad and a handful of erroneous horizontal scroll events get fired off, which is SO annoying (e.g. accidentally seeking in a video player). I can't find a setting or action that allows me to either send a "fingers-up" event, or to disable the trackpad entirely for a brief moment. Is there a workaround?

It seems like "block scrolling" should work, but it doesn't seem to do anything.