>
>I have considered many ways to do this like populating a list (saveList) with all the filenames in the directory after checking each one against a deleteList first. Then processing the files based on the list. Once saved moving them to the delete list. After completeing the save list, I could process the delete list to delete them. I am not convinced this is the most efficient way to do this so looking for better ideas or proven ways to deal with a file based queue. You can see my basic code below; for simplification I removed the catch code and some of the event firing code, and other stuff that didn't matter.

You'll want to double-check to make sure you're releasing/closing the file after converting it back to an object (this might be done by just ensuring you call Dispose() on the object which actually handles opening the class, ex. StringReader). I'll buck public opinion on this one - MSMQ can be nice if you need all the bells and whistles of a full blown message queue but usually I think it's overkill and one more dependency where stuff can just break and it's a bit opaque as to why. Remember how much fun it is debugging WCF issues? ;-)

I would keep this simple and just wrap the delete file code in a try/catch. If it fails, add it to an internal collection that gets processed before transmitting any new data. Check this list before processing any items in the queue. It will be lost if you shutdown and still have undeleted files, but I'd bet this would still clean up a lot of the errors you're seeing on the server. If it really bothers you (and assuming you have a good candidate key available) you could just run a startup process that scans the files and checks to see if they've already been processed. If so, remove them. That should really cover most of the points of failure.