Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Static web pages
Message
General information
Forum:
ASP.NET
Category:
Other
Miscellaneous
Thread ID:
01308638
Message ID:
01309655
Views:
10
>>You can create a page with all the names, then make sure your robots.txt file encourages indexing. Google robots.txt and you'll see the settings.
>>John
>
>Hi,
>I suppose you could create a page with all the names where each name was a link to the relevant dyanamic page but with 100,000 names you'd probably want to spread it over several pages. Is this what you meant?
>
>Also, I thought Robots.txt was to list files that crawlers should *not* access?
>Regards,
>Viv
>

Yes, you would put the names in the pages for the robots to crawl. The robots.txt file can be used to point robots to certain folders as well as to deny.
John Harvey
Shelbynet.com

"I'm addicted to placebos. I could quit, but it wouldn't matter." Stephen Wright
Previous
Reply
Map
View

Click here to load this message in the networking platform