When I right click Skype's icon and choose to exit it, the icon indeed disappears.
Alas, both Skype Portable and Skype itself still secretly run in the background and take a lot of memory in vain (see screenshot). They only get a closed after a minute or two (or five).
Is this a bug in Skype itself? If so, can you solve it by, say, detecting the exit and force close Skype?
Tested version 184.108.40.206 (currently the latest) in XP SP3.