I'm pretty sure this is a simple packaging error, but while I was getting the main OpenOffice.org Icon to show up on the menu, I found that the appicon.ico for OpenOffice.org 3.1.1 Portable is outdated. It's the icon from OpenOffice.org 2, not 3. It's not that big of a deal though since the icon doesn't show up on the menu by default.
[Topic move to correct forum by mod Tim]