if the chord stays around Em, you can play the modes that have a minor third, or play the other ones to had tension
Otherwise, you have to be aware of the changes, some changes imply a set of notes (or scale), while others don't
There's a saying that goes "you can play anything anytime as long as it resolves on a chord tone" and you know what??? It's true