In any case, yeah, arps are where its at. To tell the truth, the chord changes in a lot of jazz songs are simply too fast to "nail" so to speak. I honesty dont even bother trying to keep up with chord changes. Also please try and get out of the pentatonics thing. I find its so bad for people to first learn teh pentatonics rather than the full minor and major scale. You just run into problems later cuz youa re so used to having a basic sound from the pents to the extent that when you do play the full major or minor scale, it will just sound wierd to you.
Also something you can do is to take time to listen to the backing track/ song or whtever. A lot of jazz uses the 2-5-1 progression, no mater how complex them may sound. What causes (IMO) a lot of the complexities in jazz is that notes and chords that are not diatonic to the major or minor scale are thrwon in. Despite all this, there is still an underlying harmony which you can try and listen out for.
Experiment with dufferent modes. If you do have a 2-5-1, dont just use the major scale, you can use the dorian which is the scale starting on the 2nd degree. Since you know your theory i wont even bother going into all that. Yo can even mess around with chromatic runs when you know there is a chord change coming soon.
Jazz is fun and challenging :).