Remember that reading music is similar in many ways to reading language. Your eyes and mind are used for both, so let's look there for how it works.

Think about reading in general. When you first learned to read you had to concentrate on individual word sounds. Then you put those together phonically into words. After reading a few books you can easily remember and mimic a few first books. Children do this all the time. But they aren't really reading yet. They are building up the skills required, but aren't really reading per se.

Eventually, you (and all children that learn to read) learned a lot of words by sight through repetition. Now, when you read you don't typically have to concentrate or even consciously be aware of every single letter. You've seen enough words, or words that are similar or similar arrangements of letter to simply read what is written on the page.

But even after you can read well, after you are technically literate, you can still occassionally encounter things that might throw you off. For example if you see a weird or big word you've never seen before. You have to stop and sound it out. Or go to the dictionary to learn what it means or how to pronounce it. But when you see this word again, you might remember a little more about it. Eventually if you see that new word enough, your mind will store it's phonic sounds and meaning in your subconscious. Then when you see it again, your mind will access your subsconcious, bring it to your immediate attention awareness and you will "automatically know" how to deal with it and not have to stop and look at every letter.

Music is very similar to this. So I would suggest to continue trying to "test yourself" by reading as much sheet music as possible that is unfamilar to you. You can help yourself by reading scales, chords and arpeggios (and all various intervals, and chord progressions). Eventually, more and more of it becomes an automatic process.

Also keep in mind that you have to "stay in shape". If you haven't read in a long time you can get rusty. But the more you practice at it (like anything) the better you can do it.

Hope this helps.