How fast do you play? When you're playing at one note every 2 seconds, your pinkie probably won't do that haha

Once you can comfortably play at that speed, play a tiny bit faster.. and then a tiny bit faster... BUT you need to make sure that your pinkie doesn't start making those motions, and if it does, then either SLOW down or, make some real-time adjustments with your fingers to cut that weirdness out. lol