OK... as a european I have to say something in defense of our friend from Frankfurt, because here in Italy we have a similar saying: when we are faced with something that is very difficult to accept we say that we have to "swallow the toad", just like I think that in the United States you say that you have "to swallow the crow", without implying with this a direct offense to the person who might have said the thing you have to swallow.
So.... "the toad" is not Jim, but instead what he said or proposed.