You sure you are ejecting the stick from the Mac properly..? Remember, unlike a PC, you can't just pull the thing out willy-nilly. You have to unmount it first (and wait a few seconds until any LED flashing stops) before you pull it out of the Mac.

Next time you see those files, try to remember what you just did. That's always step one of troubleshooting. As it doesn't happen every time, something unusual is going on.

Most of the usual 'invisible' files (that less sophisticated PC based systems are not clever enough to not display) you see are .plist files and things like that. What you are describing is not part of the usual files on a stick or HD...