If you push your elbow a little further from the body, you should be able to have a better perpendicular positioning of your fingers on the fretboard. Also, you'll have less stress on your joints if you play long period at a time.
Your shoulder might work more but at least it a muscle and that's what it's suppose to do lol Having your elbow too close to your body forces you to compensate with your wrist joints to try to get your finger perpendicular. That over time can cause you some pains.