I use FFP off a USB drive at work and have for quite awhile. Work is the only place I use it. When FF wants to update, it's done at work.
I was in the Add/Remove Programs and was surprised to see FF 2.0.0.3 listed there. I verified that I did not have a locally installed copy of FF, shut down FFP, removed my flash drive and then started the uninstall. Sure enough I got an error that it couldn't find the uninstall and gave a path to the location on my (removed) flash drive. I let XP remove the uninstall entry and all is well.
Has anyone else seen this? The only way I can think of that it was "installed" was when FFP updated itself to the latest version.