I think you are describing a fundamental point. As someone once used to tell me, first you get good, then you get fast. In other words, it's easy to play fast, and hard to play well. ...relatively speaking that is.
I'm 46 with little practice time and less discipline. I despair of ever reaching the speed I would like, let alone playing as well as I would like. But that don;t stop me from trying. ;)
...so ever since then, I always hang on to the buckle.