That's a great idea slippin lizard! It's something you figure out over time, but this exercise makes it obvious.
I've found recently that when I'm using a lot of vibrato, I'll sometimes release my thumb from the back of the neck so to make it easier. I guess it allows me to use my arm and not just my fingers and wrist to get the desired effect. So yes, it's very doable.