Just had a bit of a further look, and it seems that you could potentially use the 'Toggle Presets Enabled/Disabled' action and apply it to the 'Caps Lock' key.
You could then create a new preset named something like CapsLockWASDArrowKeysLayer:
Configure the caps lock shortcut to toggle it on/off.
And then within that new preset, create standard keyboard shortcuts for W, A, S, D that assigns them to the arrow keys.
I haven't specifically tried it, but I believe that should then allow you to have that 'layer' toggled on/off when the caps lock key is pressed, and should get the functionality you're after.
I'm not sure if this is the best/most efficient/easiest way of achieving it, but at the very least I think it should work.