>I'm trying to fully remove an application and I have found a bunch of keys under various branches of the CLSID tree. Do I need to worry about these?

If the app came with an uninstaller, usually that's all you have to run. In some cases an uninstaller doesn't remove absolutely everything, sometimes it'll leave things you may have entered such as your name, company name, hashed registration key etc. so if you ever re-install that information is already there for you as a convenience.

In some extreme cases, such as various Symantec AV products, you have to run a tool (e.g. "Norton Removal Tool") after running the uninstaller, in order to get all the crud out.

If it's malware you're trying to get rid of then yes, clean out everything you can.