Krita is NOT completely portable and needs to be fixed within the next update. I could fix this myself but there's no point if the next update overrides it without knowing, thus the report here.
The following folder and files are left within %LOCALAPPDATA%:
Folder 1: "krita"
-> Consists of caches that are not cleaned after exiting the application.
File 1: "kritadisplayrc"
-> Some kind of display settings that are saved after changing the display settings of krita.
File 2: "kritashortcutsrc"
-> Custom keyboard shortcuts that are saved after changing keyborad shortcuts into custom keybinds. This is where I discover the incomplete krita portability.
File 3: "krita-sysinfo.log"
File 4: "krita.log"
Thank you wholeheartedly for creating portable apps for the community.
This is fixed in today's release of Krita Portable 4.3.0 Rev 2. Existing platform users will get a small 482KB patch file.
Sometimes, the impossible can become possible, if you're awesome!