I totally agree. I can't stand wearing ties. I have a client in NYC and so I have to wear a suit when I visit them, but beyond that I don't wear them.
I'm a firm believer that people who feel comfortable in their attire (reasonable attire) will generally be more productive (and yes, some are more comfortable in suits).
Kevin