Any way to stop Thunderbird to check for new version?

You should always update Thunderbird to the latest version to keep up with security updates. And the best way to update is via the PA.c Platform or by downloading the new Thunderbird Portable and installing over the current release.

If you want to stay on an old release and prevent the notification, create a directory called ThunderbirdPortable\App\Thunderbird\distribution and create a blank text file called policies.json within it. In that file, enter:

{

 "policies": {
   "DisableAppUpdate": true
 }

}

Create a distribution directory in the ThunderbirdPortable\App\Thunderbird64 directory and copy that new file to there as well. This will prevent the notice.