Many times, when I run VLC Portable, after watching something in full screen (e.g. episodes of Lost, most recently), there will be another program running on the taskbar beside VLC. It won't have a name and its icon will be the default Windows .exe (white window with solid blue title bar). Clicking the taskbar entry twice makes it disappear.
I can't find anything out of the ordinary in the Task Manager.