I deleted my bitcoin-qt thread when I was reminded it downloads the entire blockchain, but how about Electrum?
It has the blockchain stored on the internet, while it does not store your actual wallet there.
New: fb2kPortable / foobar2000 (Apr 18, 2024), Platform 29.4 (Apr 10, 2024)
1,000+ portable packages, 1.1 billion downloads
Please donate today
Electrum seems like a decent fit, so I packaged up both of them. Let me know if there are any issues.