As noted, LOCALAPPDATA\stellarium is left behind on your system when run.
I have tested on a clean Win7 system.
Fortunately, there's a simple fix that can be implemented for the next revision. See here.
New: StressMyPC (Dec 10, 2025), Platform 30.1.3 (Nov 17, 2025)
1,400+ portable packages, 1.2 billion downloads
Happy New Year! Please donate to support us in 2026!
This is fixed in Stellarium Portable 0.13.0.
Sometimes, the impossible can become possible, if you're awesome!