The main differences between the XV-5080 and XV-5050 are:

Polyphony:
XV-5080: 128 voices
XV-5050: 64 voices

Patches:
XV-5080: 1200 patches
XV-5050: 1400 patches

FX processors:
XV-5080: 5 FX processors
XV-5050: 3 FX processors

Expansion boards:
XV-5080: 4 SRX boards and 4 SR-JV80
XV-5050: 2 SRX boards

Another advantage of the XV-5050 is that you can edit the patches on your pc using a USB cable and patch editing software.

Note:Some patches will take 8 voices of polyphony so you can run out of polyphony pretty fast on the XV-5050 if you layer sounds.

Greetz,
Marcel