I do the same as you. Cleaning the fretboard with the strings on, or some of them, would just be too much of a pain in the neck.

I've never had a problem with my neck warping or anything doing it this way.

I wouldn't leave the guitar sitting around with no strings on it, but having none on it for long enough to start putting the new ones on once the old ones are removed I think is fine.

I'm sure there will be a variety of opinions on this, but that's my take on it.