Hey,
I ran into this post by the traveling librarian who used a batch file to run a Second Life Viewer, then delete the files on the local disk after the program closes (Link: http://travelinlibrarian.info/2007/01/second-life-portable/). I used that script as a template for another program (Robert Lang's Origami TreeMaker Link: http://www.langorigami.com/article/treemaker). It's open-source and relatively simple compared to GIMP. I just modified the script, so that it deleted registry files using the command REG delete. I tested it on a Windows 10 laptop and a Windows 7 desktop. I didn't find any unwanted files in %appdata%, %userprofile% or %localappdata%. It also successfully delete all the registry keys, and it didn't show up on the "Programs and Features" list.
Would this app be "truly" portable? What are the limitations of using this method?