Seems we're missing portablization of a file within the Thunderbird Portable profile. Specifically, folderCache.json which looks like it's for recent folders. I've put together a test fix for people to try before the next Thunderbird Portable release. Please give it a try and let me know if it's working as expected for you.
Download the zip file and replace your existing ThunderbirdPortable.exe launcher with the one included in the ZIP. The GPL source is included in the Other\Source directory as usual.