There are several pedals named 'Noise Supressor' , 'Noise Gate' , etc.
any one of them will solve your feedback problem and they have controlers of course to controle the noise removing level and puting it on/off ..
Actualy i recomend the Boss NS 'Noise Supressor" , i have Boss GT-3 multi-effect and it has that NS built in , and it works great so that if you use the Max. settings for that NS, there wont exist any feedback even at very high gain and volume, and sticking your guitar pickups into the amp's speaker !!
P.S.
====
NS= Noise Supressor
All the multi-effect processors that i've seen had NS or something similar, even the Zoom which is the ****tiest Guitar/Bass effect maker in history (that's my personal opinion)