Make slider widget respond to dragging only, not tapping

I have volume and brightness sliders on my touch bar and occasionally I will inadvertently touch them while pressing number keys. Depending on where I touched, this will suddenly set the volume or brightness to extreme levels.

A way to prevent this would be if the slider widgets would respond only to grabbing and dragging the indicator, not to tapping anywhere on the slider bar.

If there already is a way to configure them this way I apologize, I looked through the settings and didn't see anything.