I think you are making a good decision Gary. Glad you made it back o.k.
I sold my 19 foot, 150 hp, bass boat and bought a much smaller one, with stick steering. I'm able to get in and out of it much easier. There are some compromises, but at least I'm still fishing!
Yesterday my friend and I were fishing when a storm popped up. We were about two miles from the dock, but managed to get back just in time to not get blown away.
Maybe you should trade your boat for a powered cruiser and still be able to get out on the water.