I have noticed that if there is much sound at all from the speakers/p.a. reaching the vocal mic, the harmonizer is quick to "reharmonize" these sounds and distort. Also, it is easy to over-e.q. the mic. In this case, even though the mic sounds good, the harmonizer will break up if certain frequencies are too dominant.
Having said all this, I suspect that some units do have problems that mine doesn't exhibit.
DonM