I think the damage comes because the ohms or signal from the headphone outputs (I'm guessing this is how your wiring it straight into the sound cards) is different to that used by mics. Thus the signal from the headphone jack could overload the soundcard.