Yeah, when I first got into sweeping I couldn't slow down without getting sloppy... That's why I tried to focus on sweeping with low bpm... It helped me a lot.
Now I have total controle over my sweeping technique, I can make it slower or faster at some parts, or play it all even at a given bpm .

Usualy , most techniques are easier when they're slow and get harder as you get faster.
With sweeping, it's easier with moderate speed, and gets harder if you slow down or speed up.

The trick is to practice.