to redirect or remove? robots.txt on IP version of site

robots.txt, IP, mod_rewrite,

[martinmartin]

We have redirected IP requests to www using mod_rewrite in httpd conf Apache. In Webmaster Tools we see the IP/robots.txt there. Question is; should we remove the robots.txt? Should we 404 it? What to do? It is a duplicate of the robots.txt on our www.

Thanks,

DM

[g1smd]

If you are redirecting IP requests to www for all URLs, I would add an exclusion to the redirect so that requests for robots.txt are not redirected, and add a rewrite so that the IP requests for robots.txt are served a different robots.txt file. This extra robots.txt file would block nothing so that all redirects are seen.

[martinmartin]

Thank you!