This doesn't surprise me at all. Just as boosting gain increases distortion on a guitar amp by driving the preamp harder don't boost the gain unless you are actually trying to add distortion as you may want to do to dirty up an organ or electric guitar patch to add some grit to the sound.