Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Hiding robots.txt

Bloggerheads - 9:27 am on Oct 30, 2003 (gmt 0)


Oops. My bad.

OK, here's a generic example for you:

# /robots.txt file for http://www.example.com

User-agent: *
Allow: /games
Allow: /forum
Allow: /tutorials
Disallow: /

[edited by: engine at 5:19 pm (utc) on Oct. 30, 2003]
[edit reason] examplified & de-linked [/edit]

Thread source:: http://www.webmasterworld.com/robots_txt/74.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com