I had a similar thing (although probably not as funny) happen to me when I was learning Spanish. I knew enough to make sentences, but couldn't distinguish between all of the sounds.
I think I was at a gas station and told the guy "Te voy a pegar" (I'll punch you) instead of "Te voy a pagar" (I'll pay ....). Fortunately, he didn't take offence and had a good laugh.
I guess there's a big difference between the "ehhh" and the "ahhh" sound.