Only my opinion:
There are many parts of being a musician. I go to the gym to improve my health – in the same way arpeggios and scales help my performance.
Practicing pitch recognition can also help; I have never done that.
I think of the different scales as families of notes and chords; I am comfortable knowing most of them. Try to avoid the transposing button is not a bad idea. I have some friends who play everything in the same key. There are some good points about that idea also. Confession: Lately I find myself playing everything in C major. (Getting lazy)
Everything in moderation, I do not wish to become a machine that I have created; what comes naturally should not be lost.

Improvising:
Studying the Major and Minor scales – and scales modes are helpful. When overdone they become you and you become boring. They become a very destructive to your natural ability.

What I found valuable is to listen to a musician that you would like. I listened to Johnny Smith (guitar player) for years; I never studied any of the books he wrote. After a bit of time I found myself playing some of his ideas.
PLAY YOUR FAVORITE MUSIC EVRYDAY AND KEEP SEARCHING FOR NEW IDEAS FOR WITHIN YOUSELF.

Good subject, John C.