I might be in the same boat as you. I've played guitar for 13 years (I'm 24 now) but I have only been learning theory for the past 2 years maybe. It does seem like I have to work a lot harder at it than a kid just starting out would. It's hard to, like you said, retain things and I find myself having to go back over and over again because I forget something I learned previously.

Everyone is different in that area too though. There are probably people who start really late but pick it up extremely fast.