to me this isn't really about remembering frets

it's about recalling information

to me you are just trying to recall more than you've actually internalized

this is the policy I go by for ANYTHING; not just guitar... ANY THING in life

IF I have trouble something

THEN I break it down into smaller parts; until I stop having trouble with it.

AND build back-up in complexity; until I can do the thing I'm trying to do.

that's it.

specifically for this situation...

If I was you

I'd reduce this to the first two notes

and just go back and forth with those two notes until you've got that down

C, D, C, D, C, D

and add one note

C, D, E, D, C, D, E, D, C

then add the next note

C, D, E, F, E, D, C, D, E, F, E...

etc

 until you can do the whole scale forward and backwards without even having to think about it.

and if at any point you start having trouble...

back-up until you get to the point where you aren't having trouble; and keep working from that point.