homepage Welcome to WebmasterWorld Guest from 54.161.236.92
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
302 code and robots.txt
will google spider check the robots.txt when the server code is 302?
coco




msg:1528674
 9:16 am on Sep 10, 2002 (gmt 0)

Basically, I've got a few domain names. and for technical reasons beyond my control, I can't point all these domain names to the same ip. some of them have 302 codes.
I don't want google to follow these, because it seems to be unable to resolve all these domain names to the same website.
so i want to put a robots.txt banning google from those domain names that serve a 302.

Is this possible? or will google just follow the 302 straight away?

Thanks for any help!

 

korkus2000




msg:1528675
 11:39 am on Sep 10, 2002 (gmt 0)

It will get the 302 in the header when it requests the robots.txt. It will never get to read the content. The robots.txt will not work on any header redirect. With a 302 redirect your telling the spider that the object it is looking for has temporarily moved to the redirect page.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved