>If you build an EXE file and place app.config file in the same directory your application should pull the connection string from the app.config text file.

Setup and deployment actually copies the app.config into your deployment project. If you delete the app.config from your applications directory, .NET reinstalls it the next time you run the EXE.