Exclude any Windows relations/user folders

I'm still using Windows XP on my 2 machines, so my user folder location is:
C:\Documents and Settings\USER-NAME\Application Data

Correct me if I am wrong, but with my Portable-Apps Platform installed and being
ran from a USB Flash Drive, each time I plug it in and run it, doesn't it make and
use a folder inside my Windows/USER directory, such as I stated above ?

If it does not, then disregard this entire post, with apologies.

BUT, if so, I want to know if there is a way to make it STOP using any part of
Windows, and keep everything self-contained on the USB Drive itself, instead
of using any Windows-User directories.

How can it be done ?

.... thanks.