We have no control over crap program leave behind on our disk. Some leave cookies, other logs and some even bigger and scarier files. Some change registry settings that may affect other applications. The worst kind, are those that delete files - even those not belonging to them.
One way to deal with this problem is to run your app in a VM (such as VMWare's Browser Appliance). Another, is to use an application called Sandboxie.
Sandboxie will allow you to add applications to a 'sandbox' - anything they try to write to the hard disk or the registry, will remain in the sandbox. Sandboxie achieves this by installing a driver in your system, that captures all the write commands and redirecting them to a temporary storage locations. Read commands go through normally (although they are logged), allowing applications to access their data.
You can assign any application you'd like to run in Sandboxie. By default, it allows you to run a web browser or an email client - with no configuration. You can add programs by selecting them from the Windows start menu, or by pointing at their window (so you can actually sandbox an application that's already running - let's say, if you point your browser at a site you don't trust - and then release it).
Sandboxie is free and can be downloaded here.
A new version (3.21) relaxes the domain licensing restrictions to allow unrestricted use on portable computers. Get it at the same link above.