Another thing you can do (might take a little longer but I think it will engrave it into your mind better) is to start by memorizing the "dotted" frets. Frets 3, 5, 7, 9, 12 and the open string of course. Do this for just the top 2 strings for maybe a week. You know you can visually see the numbers of those dotted frets, so all you have to do now is memorize the letters instead. In a week, add another string. Another week, another string.

Once you have them all down you'll have a pretty good basic visual. Then it's just a matter of applying what hpcrazy suggested and fill in the missing frets. Since you'll have practically every other fret already memorized (dotted ones) you should be able to quickly thing what is the one to the left or right of it. Makes it easier than trying to just memorize from 1-12 on every string.

Make sure you don't rush it. When I say a week, it could even take longer. Don't fool yourself into thinking that it's easier than it is. Memorizing the fretboard takes time so just make sure you put the time into it.