Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Doorway / Hallway pages Vs. Robots.txt

Does one need to build "many" sites under one roof?

5:43 pm on Jan 13, 2001 (gmt 0)

New User

joined:Feb 27, 2013
votes: 0

A bit new to the game and have several thoughts (mostly confusion) I need comments on if anyone has the time. I would appreciate it.

If you optimize a page, say for AV, and want "other" spiders to not index it you would build a robots.txt file. I read, "This can work IF there are NO other links to that page from the rest of your site".

My literal interpretation is that one needs to build multiple web sites under one roof. The robots.txt file would keep AV from looking at optimized pages for Exite and so forth. It is my understanding you need hyperlinks in the "home" page for better ranking among other considerations.

If the optimized page has a link to the index.html and that home page has links or a site map to other pages set up for Google / Yahoo and AV (as an example), you are going to have quite a mess. Obviously, I don't fully understand the concept of doorway and hallway pages and how to implement them.

12:54 pm on Feb 28, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
votes: 47

Hi Gary (apoligies for the overlooked post - it happens rarely)

Well, doorways and hallways work best with cloaking. You can control who gets what version of the page.

If you are not cloaking, then you can do the now you see me - now you don't tricks with robots.txt.

Build one set of directories for what ever engines you are targeting. Mask off the Alta directories in your robits for Excite, and mask off the google from everyone else. It can lead to a pretty complicated robots.txt, but it is worth it if you have enough content to warrent.

2:38 pm on Feb 28, 2001 (gmt 0)

New User

joined:Feb 27, 2013
votes: 0

Thanks for the reply. I need to reflect on your comments for a day or so. (Also busy as a one armed paper hanger) :)


3:35 am on Mar 20, 2001 (gmt 0)

New User

joined:Feb 27, 2013
votes: 0

Where can I find a decent document on cloaking.


10:45 pm on Mar 20, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 12, 2000
votes: 0

Hi Gary, lots of GREAT info. in the cloaking forum, there you will find mentions of different cloaking softwares, styles of cloakings, comments on the ethics of cloaking, how SE's perceive cloaking, etc. If you are interested in this, I'd reccommend you start by checking out that forum to find comprehensive info.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members