Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Scraper Site Clearout Collateral Damage?


Seo1 - 3:57 pm on Aug 15, 2005 (gmt 0)


Hi Dataguy

It is found at the link below:

[google.com...]

I clipped a few of the important issues from the page below:

# Make use of the robots.txt file on your web server. This file tells crawlers which directories can or cannot be crawled. Make sure it's current for your site so that you don't accidentally block the Googlebot crawler. Visit [robotstxt.org...] to learn how to instruct robots when they visit your site.
# If your company buys a content management system, make sure that the system can export your content so that search engine spiders can crawl your site.
# Don't use "&id=" as a parameter in your URLs, as we don't include these pages in our index.

Hope this helps.


Thread source:: http://www.webmasterworld.com/google/30571.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com