joined:Jan 15, 2004
Robots.txt is very important from the SEO perspective. By default, the robot looks for the robots.txt file in the root of the original domain. The question is, what happens exactly when the spiders face a 301 redirect?
Does the robot skip it on not finding the robots.txt at the default location or goes on to the redirected landing domain as well to pick up indexing instrucions?