Actually most of it will be your broadband connection, the router in your house, your internal network, and everything along its path. Following that, take a look at:

A good little review on troubleshooting slow internet speeds:

http://www.sprintbroadband.com/speed_guide.html

What ever happened to Intel's internet chip? Supposedly you could access the internet and run applications twice as fast as was previously possible:

http://news.com.com/Intel+to+unveil+its+Internet+on+a+chip/2100-1040_3-257786.html

and Lucent's internet DSP chip, and connect Ones Wireless internet chip, etc. I remember reading about a bunch of supposed technology jumps 5 years ago...