I read somewhere (can't find it now) that this 20th Century phenomenon in Western popular music was linked to the emergence of certain classical pieces, such as Pachelbel's "Canon" and Bach's "Jesu, Joy of Man's Desiring" from historical obscurity to becoming some of the most listened to pieces out of a much larger body of classical music.

Supposedly when these classical pieces went mainstream, 20th Century composers picked up on their themes in a way that led to what we have today. Perhaps the radio and phonograph are partly to blame because they allowed these progressions to proliferate in a way not possible before broadcast and recorded music. And we all tend to emulate what we hear because it sounds "right."

Also in the 20th Century we've standardized on the equal-temperament scale tuning. Different progressions might sound better in some of the 17th- and 18th-Century intonations.

Disclaimer: I'm not trained or well-versed in classical music. This is just me paraphrasing something I read a while ago. FWIW.