Welcome to WebmasterWorld Guest from 54.159.50.111

Forum Moderators: goodroi

Message Too Old, No Replies

302 code and robots.txt

will google spider check the robots.txt when the server code is 302?

     
9:16 am on Sep 10, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 25, 2002
posts:74
votes: 0


Basically, I've got a few domain names. and for technical reasons beyond my control, I can't point all these domain names to the same ip. some of them have 302 codes.
I don't want google to follow these, because it seems to be unable to resolve all these domain names to the same website.
so i want to put a robots.txt banning google from those domain names that serve a 302.

Is this possible? or will google just follow the 302 straight away?

Thanks for any help!

11:39 am on Sept 10, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 20, 2002
posts:3732
votes: 0


It will get the 302 in the header when it requests the robots.txt. It will never get to read the content. The robots.txt will not work on any header redirect. With a 302 redirect your telling the spider that the object it is looking for has temporarily moved to the redirect page.