That is a great question. You should definitely try to keep it straight up, but I also must admit that my thumb has wandered in both directions while I've been playing, depending on what kind of chords I'm making. It's best to try to keep the thumb as straight as you can for the time being as you get more comfortable with making cords. You may see some guitarists slant their hand to the side as well, based on their personal preference, but holding your fingers parallel with the frets will allow for an easier time while you're starting out, especially with barre chords.

Hope that helps!

Monique