Your goal isn't to touch the fretboard. The FRET is what cuts the string length off and gives you different pitches. That is why you play as close to the fret as possible, not further away from it. The best way for a note to be played is to touch the string as lightly as you can, while still getting a good tone. That is, you have to press down enough for the note to sound, but no further pressure is needed.

Take a look at scalloped fretboards and try to touch the fretboard haha.