As noted, LOCALAPPDATA\stellarium is left behind on your system when run.
I have tested on a clean Win7 system.
Fortunately, there's a simple fix that can be implemented for the next revision. See here.
New: Pinta (Apr 21, 2026), Platform 30.4 (Apr 28, 2026)
1,400+ portable packages, 1.2 billion downloads
PortableApps.com is operating at a loss. Please donate if you can!
This is fixed in Stellarium Portable 0.13.0.
Sometimes, the impossible can become possible, if you're awesome!