I apologize in advance for this long-winded response but my thoughts regard the details of your comp and the construction of your patches; how many voices are being used in each patch and how big your pad chords are.
*If* your two pads are layered, comprised of two layers each and your chords consist of six notes, you've shot 24 voices and it leaves you with only 8 voies for the piano. Moreover, if the piano patch is two layers you're reduced to 4 voices, which means you have no room but for a two note chord. Sustain also needs to be taken into consideration.
*If that's the case,* there's a few things you could do, most require a bit of programming.
First, you can prioritize the piano by using voice allocation. Setting up the voice stealer will still result in drop out and the result will probably be noticeable. However, notes will be robbed from the pad(s), not from the piano.
You can take a more drastic measure and delete layers from the pad(s), and naturally, you can thin down your chords. No doubt, any of those three methods may take away from the overall feeling of your comp as it will thin down the background.
The best way, in my opinion, is not exactly the easiest because you'll have to do a lot of experimenting. While you're probably already using pan to offset your pads, you may want to try offsetting the chord structure as well.
Essentially, synth pads are the same as orchestral string sections, and you may find you can use your patches as they are and reduce polyphony by splitting the sections between tracks. Give some thought to pan as well as the role each section plays; a bass section is often divided by octave, so either a single layer patch playing two notes or a two layer patch playing one note can be used. Again, take your time and fart around (i.e. experiment) with the variables because there's many combinations that can add more color to your comp and leave you with a cleaner arrangement.
I'm done, but if you need some help with this feel free to drop me a line.