Many OpenSource applications are actually portable. You just need to copy their installation to your USB key and take care to SUBST you USB drive letter to some fixed letter e.g. Z:/ before launching the Portable Apps shell. If some trouble remains most of the time you can cope with them adding the USB key the missing dlls and launching the app thru a .bat command file that takes care to assign the DOS variables TMP HOME PATH etc to some folder under Z:/
In the years I used to solve my problems that way and built nice USB keys that assisted me when changing classroom (you do this very often in some countries!). Portable apps were the first cornerstone building this. Another cornerstone in making your teaching support classroom-independent is the adoption of a cloud storage.
If you are interested in all of this you can follow a set of webinars (here) we conceived to summarize approaches in this direction. If you prefer videos (here) is a playlist with the same lessons. In particular (this) lesson deals with PortableApp onto USB (this video).