As per my experience, I have been using several different practice amps ( Fender GT40 with 6" speaker, Vox amp with 10" speaker and Peavey with 12" speaker, all 40W amps., Any speaker in the range 10" -12" makes a BIG differance in the sound ( other than the tone difference on each amp ). even if you play at lower volume.
In my openion it is better to go with a large speaker, and minimum of 40W ( or greater) amp.
Most amps have earphone output so can play silently as well if needed.