Forum Moderators: phranque
Unusual questions in my mind...
I have a friend who has had Google bot HEAVILY crawling his site for over a month. It keeps going through his pages again and again throughout the day, it then leaves and then comes back the next day. We want to see if we can redirect the bot to another site as an expierement. Would there be an entry I could have him add to his robots.txt as it always request it first when entering his site? I know you can get creative with .htaccess but he is on a Windows server.
Once the bot is on my site we want to try the same on my Apache server hosted site. How would I go about this with .htaccess?
Any thoughts or is this just plain silly to try?
Thank you,
Brian
I take it you aren't too worried about confusing googlebot and possibly losing being listed?