Jump drives normally come with an installation disc which is required to run them on Win 98, (Theoretically 98se solved the problem, but in practice anything before the Me Edition, requires the installation disc)
The same goes for an external hard disc, but I would recommend against this, as it will alter your drive letters, and as yours is an old USB connection it will be slow as well.
If you decide you want another computer, then make a list of what you want the computer to do, then get the best you can afford, if you go for a cheap one it will be out of date so fast, you might as well not have changed, whereas if you go for the best you can afford, you will have quite a few years use out of it.
Hope this helps.

Bill