I'd have to hear it on the record, but I guess he starts his upward sweeping motion by hitting the B string.

I-----------------------------
I-------5------------5--------
I-----4-^--4-------4-^--4-------
I---6---|----6---6---|----6---
I-7-----|------7-----|------7---
I-------|------------|-----------
down---up----down---up-----

What you do wrong is going through the arpeggio, then coming back down, so you hit the B sting twice... I hope I am clear.
That's the same technique in the opening section of Al di MEOLA's mediterrenean sundance, where he sweeps (something he nevers does except in that kind of arpeggios) and then hits the highest string in an upward motion, to sweep back to the start of the arpeggio.
You have to be careful of your timing, because you have to go AROUND the highest string, and it takes some time. Practice slow, and you'll get it.