Hmmm, I never had a Jam man but I once had a boss loop station and to have a clean sound looping, I had to put my settings on my amp and effects board to a clean sound, record myself, put the "dirt" back on, then jam away with the loop.
I hope this helps.