Two things are going wrong here:
1) When the USB-stick is unplugged prematurely, portableapps.exe does not stop itself. Instead it crashes (but remains active).
2) After that, when reinstering the USB-stick and running start.exe, it doesn't start because a ghost portableapps.exe is still running (and consuming 100% CPU!!).
For reasons of pure awesomeness (meaning NOT), a USB-stick on my pc may "unplug" all of the sudden without ever having been told. Don't ask. This is, of course, a horrific screwup from whatever outside PA.com is causing this, but the whole nature of hotpluggable devices like USB-sticks is to be, well, hotpluggable.
This may also happen when unplugging the USB-stick after going to hibernation while portableapps.exe was kept running.
Unplugging is just fine for data. But unplugging a drive underneath an application that is running from it is absolutely not fine.
For now, you will need to manually kill the PA.c Platform in that situation and will lose any settings or changes from that session. There is a bug preventing it from detecting an unsafe eject and just closing. Some apps may leave data behind, lose data or wind up in an unstartable state from an unsafe eject.
Sometimes, the impossible can become possible, if you're awesome!