I can recall having a lot of trouble with my pinky in the beginning. Once you hit barre chords, you have no choice but to start using it in chords. The problem comes with solos where you can choose to avoid using it.

I can't point to a particular moment where my pinky came into control, but it was sometime after learning the 5 pentatonic scale positions and practicing them up and down the fretboard.

Certainly it makes sense to "work with what you got", but I don't think you should give up on getting the pinky "under control". Getting it under control will give you more range in a given position without having to stretch really far and give you more "economy of motion", which leads to cleaner and faster playing. Speed isn't all about shredding, players like SRV, Gilmour, Clapton, Hendrix have plenty of fast runs in their solos. So I would say working on your pinky is a very worthwhile endeavor.

A good exercise I think would be to work on playing the major scale up and down in thirds. I suggest this because that scale uses the pinky on all but the high E string. That exercise will help you develop both coordination and strength in your pinky. Take it slow at first and work on getting it clean and precise. Then work with a metronome to help you speed it up.