IMV it's entirely normal at first, and in fact if you weren't doing it you'd be ignoring a valuable sense in learning, the visual assisting your aural and tactile. I think it would be accurate to say everyone does it initially, as it's instinctive. Gauging from my own experience, there's nothing to worry about doing so during this period.

After a while, you'll be forming the chords, changing chords and playing chord progressions automatically and instinctively at which time you should find yourself also doing so without looking at the chords nor needing to. If having progressed to this stage you find yourself still looking at the neck to affirm fret position for every chord other than an occasional altogether naturally instinctive glance, or worse, needing to, then it's developing into a habit you will need to consciously alter which should be fairly easy at that point.

Until then don't overly sweat it until you find yourself fluid and comfortable with playing songs with a typical four chord riff and a three or four chord chorus seamlessly and easily. Independence from visual confirmation is just an inevitible part of progressive learning and skills acquisition process IME. For me, it occurred without any conscious effort once I developed tactile proficiency and muscle memory.

Relating my continuing experience I'm finding the same process reoccurring with scales and solo licks, each just a sequence of individual notes like a word made from a number of given letters of the alphabet. I still need to visually affirm the fret position accuracy for the root note of any scale prior, although aurally I can immediately discern if I've hit a half tone higher or lower if I misplace my position by a fret which I'm still doing but expect time and exposure will correct. The process of developing technique for playing 'blind' as it were, is gradual, like adapting to playing standing up with a strap is to sitting without.