NETworkManager (for Windows) is Open Source and has a portable build. Settings and Profiles are saved in the app folder. Some temp files are saved on the PC (AppData, Temp, Registry).
https://github.com/BornToBeRoot/NETworkManager/
https://github.com/BornToBeRoot/NETworkManager/releases/latest
Unfortunately, their portable package is not portable. It requires .NET 6 to be installed locally, which requires admin rights. You can bundle .NET runtimes with your app to make it portable (Paint .NET does this) but it required compiling in a specific way which would be up to the publisher.
Sometimes, the impossible can become possible, if you're awesome!