Your fingers could be a mile away because they need to apply more force and that in turn could be solved by

(a) Lowering the action on your guitar
(b) Using Lower Gauge Strings
(c) Hammering on and Pulling off notes by placing your fingers to the rightmost possible point before the fret (assuming that you use your left hand on the fretboard)

By the way I think practising chromatic scales will help the most.