Welcome to WebmasterWorld Guest from 54.144.243.34

Forum Moderators: goodroi

302 code and robots.txt

will google spider check the robots.txt when the server code is 302?

   
9:16 am on Sep 10, 2002 (gmt 0)

10+ Year Member



Basically, I've got a few domain names. and for technical reasons beyond my control, I can't point all these domain names to the same ip. some of them have 302 codes.
I don't want google to follow these, because it seems to be unable to resolve all these domain names to the same website.
so i want to put a robots.txt banning google from those domain names that serve a 302.

Is this possible? or will google just follow the 302 straight away?

Thanks for any help!

11:39 am on Sep 10, 2002 (gmt 0)

WebmasterWorld Senior Member korkus2000 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



It will get the 302 in the header when it requests the robots.txt. It will never get to read the content. The robots.txt will not work on any header redirect. With a 302 redirect your telling the spider that the object it is looking for has temporarily moved to the redirect page.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month