Can't remove Facebook when starting Firefox Portable

This is driving me nuts. Since recently I have Facebook appearing along with my other tabs no matter what I do.
What I did :
- delete facebook.com in history
- delete the whole history
- disable every extensions
- make sure default page should be a blank page
- delete profile
- delete any Mozilla folder from AppData/Local or Roaming
- reinstall Firefox Portable entirely
- restart the computer

Even with no recent tabs the only page appearing is still Facebook.com
I noticed the last update was yesterday, that's the last thing I can think of, I'm out of ideas.

I have used Firefox Portable for YEARS, never seen this before. This issue doesn't happen with "normal" Firefox only the portable one. I can't find any info about it, nor any info on a virus that could do that.

Please help

(PS: I just created this account and made a mistake on the name, how to delete this account so I can recreate it with the correct name ?)