I don't see it as too many sports being removed. I think Winter and Summer Olympics would be improved by being much smaller and restricted to the "classic" track and field events. Also removing sports where the quality of the sprotspersons equipment (ie money) becomes one of the deciding issues.
You could start with a level playing field. All the javelin throwers use a standard supplied javelin. All the bobsleigh teams use a standard supplied bobsleigh. Then you wouldn't be measuring the performance of euipment but of the person which is what the whole thing is supposed to be about anyway.

Nick Mason