Difficult? No, not at all
I started playing keyboards about 2 years ago, it was quite hard for the first 3 months, but after that, a piece of cake.
Can't read notes and can't sing either (shame on me! ).
You say that playing the arranger is much harder then playing keyboard, Personally I find playing the arranger much easier.
You just have to play chords with the left hand and melody with right, and press the fill/variation button once in a while, With a little practise, It all goes automatic.
And I find playing keyboard/piano in general not that hard (It can be tricky sometimes).
I didn't find my Korg PA-60 hard to operate either, I'm not really a power user (I haven't gone through the manual yet), but I knew most stuff within a couple of days.
But then again I'm kinda into computers and stuff .

Greetz ,
Marcel