I'm an 11's guy myself. Its difficult at first just due to the thickness of the strings. But you play on standard tuning long enough, and you'l work your fingers up. You get so much more of a rich tone, I find it hard to believe you're gonna want to go back down to ultralights. Unless you're looking to shred, the heavier the better in my opinion. I used 9's on my uncle's LP and I couldn't even feel them under my fingers, which was kinda weird.

Whatever works for you though. All personal preference. A lot of the metal guys go heavier for nasty sounding pinches and such. Just gives a heavier tone all around.