Remote views are not Rushmore optimizable. The whole IDEA behind C/S is to get small set of records at the time (i.e. Select a phone number from a list of area codes). So dont get all phone numbers load only area codes. Then using an area code as a parameter for your remote (parameterized!!) view get records a user is looking for.
Hope I helped. Mark