Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Doorway / Hallway pages Vs. Robots.txt


Brett_Tabke - 12:54 pm on Feb 28, 2001 (gmt 0)


Hi Gary (apoligies for the overlooked post - it happens rarely)

Well, doorways and hallways work best with cloaking. You can control who gets what version of the page.

If you are not cloaking, then you can do the now you see me - now you don't tricks with robots.txt.

Build one set of directories for what ever engines you are targeting. Mask off the Alta directories in your robits for Excite, and mask off the google from everyone else. It can lead to a pretty complicated robots.txt, but it is worth it if you have enough content to warrent.


Thread source:: http://www.webmasterworld.com/robots_txt/73.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com