In its sometimes misguided attempts to do everything automatically, XP holds a driver cache somewhere and will re-install whatever files it thinks are appropraite upon system reboot without you even getting a look in.

Therefore it can be necessary to find all instances of the driver files that you thought you'd removed, and then get rid of those as well! Also there may be cached ".inf" files relating to the driver, and it's best to zap those as well.