It seems Telegram Desktop Portable is unaware it should remove HKCU\Software\TelegramDesktop, can you add this?
Also, TelegramDesktopPortable\Other\Source\AppNamePortable.ini should be renamed to TelegramDesktopPortable.ini
Thanks!
[EDIT] Marked as bug, title clarified - mod GC
The Registry cleaning would be a bug. I'll mark it as such, and we'll fix it in a future update.
As for AppNamePortable.ini, that is the correct name for the file - see Source\Readme.txt for details.
This is fixed in Telegram Portable 1.8.8
Sometimes, the impossible can become possible, if you're awesome!
Thanks!