Well, you got used to it right? so if you prefer it, why would a listener not?
I reckon that it will sound "wrong" which can become artistic...
It really depends on how you use it man...although if there is a consistant buzz, you probably should raise your action or speak to a tech about it. you can get that buzz with a normal guitar as well, if you just manipulate your fretting fingers appropriately...
So I just wrote a rather lengthy reply without actually helping you at all, sorry man, I guess you'll just have to try and see if people take to it or hate it...