Well, there's nothing to prevent you from editing a commercial SMF into the arrangement you want, and of muting as many of the parts as possible to give yourself a musical challenge when you play it. Nor of using 'Mark/Jump' features to re-order the arrangement on the fly, if your dance floor needs extending (or shortening!) the piece.

I personally don't see a LOT of difference between 'noodling' over basic chords you input with the otherwise almost completely unused LH, and 'noodling' over an SMF. Either one of them can present as much challenge as you are willing to give yourself. Or as little.

The biggest challenge in doing more music from the last thirty or forty years, compared to standards, is popular music's move over to the electric guitar as the main sound used. Back in the 'standards' day, the guitar was just part of the rhythm section, and not a very big part even of THAT, and seldom featured compared to horns, etc.. If there's one thing arrangers in general are weak on, it's getting electric guitars RIGHT. As integral to modern pop and rock as they are, they tend to show up the weaknesses in our arrangers.

This might be the place to try and find good MP3 backing parts, with REAL guitarists playing. There's STILL nothing to stop you editing even THOSE into a structure you feel more comfortable with (although I'm not sure if any arranger has the 'Mark/Jump' feature for audio yet, dammit!), and the end result will go a long way towards packing the floor.

Another approach might be to work with whoever runs the break music, whether it is YOU, or someone else. You might be able to stick to your formula, if the break music concentrates exclusively on this 'younger' period. I've worked several clubs where the band tend to concentrate on say, classic rock, or R&B, and the DJ does the more top 20 stuff of the day. Worked pretty well when done right.
_________________________
An arranger is just a tool. What matters is what you build with it..!