Low action is better for speed and ease of playing. You don't have to press them as far to reach the frets, right?
If you use heavy picking, you may get some strings buzzing against the frets.
I don't understand your comment about the string almost slipping under your finger. Once a string is fretted, it doesn't matter how high it was before you fretted it.