With regards to your situation, Incidents, I'd say that you shouldn't be afraid to keep jamming with your current band. However, as your post suggested, don't pigeonhole yourself either: if you find a small trio or quartet you can jam in, take it up as something on the side. See which one you really prefer. Additionally, I'd say that you should analyze yourself as a player: Do you really need to play in a large band to have fun playing? Obviously, the next question would be to ask if you feel that a smaller band would make it any better?
With regards to the talent level: Remember that ultimately you are playing for you. Bands that work only do so because all the people get along together. I used to play with a bassist/guitarist who's now on his merry way to some music college with a free ride. Phenomenal player. Too bad he was a jerk. Anyway, my point is that: 1.) You shouldn't be afraid to experiment with different playing situations. 2.) You should have fun while playing. 3.) You should also be playing for you. If you're stuck with virtuosos, but the setup doesn't work for you, my best advice is to get out of there.
Hope my advice helped (especially considering this is my first post)