In the taskbar, the "Safely Remove Hardware" icon has 2 ways of removing the thumb drive. 1st is to right-click and on the popup you have to click stop, another popup is displayed, and you have to select OK. The 2nd way is to left-click, and on the menu, select the drive you want to eject.
Currently the eject feature of Portable apps behaves like the right click, where you have to wade through 2 popups in order to remove your drive. I'm suggesting to operate in the much simpler fashion, if at all possible.