I have tried to run Open Office 2.21 with four different XP Prof based laptop and in two desktop system (XP Pro and Vista). I think Portable Open Office is still quite unstable and can't be used for serious work. In my system Portable Open Office is installed to default directory to new fast Cosair Voyager GT 8GB USB stix. The most frustrating experience was a lost spreadsheet after a couple hours of work when PC freezed when I was saving the work. There is also some features not working at all in Open Office menus - Extension manager is a example which I like to use for customized thesaurus and hypenation.
When rough edges are away I think PortableApps system and Portable Open Office is a killer system I shall use every day. Today I'm waiting for better days...