Forum Moderators: phranque

Message Too Old, No Replies

Passing a bot from one site to the next

Windows OS

         

Reflect

1:58 pm on Jul 1, 2005 (gmt 0)

10+ Year Member



Hello,

Unusual questions in my mind...

I have a friend who has had Google bot HEAVILY crawling his site for over a month. It keeps going through his pages again and again throughout the day, it then leaves and then comes back the next day. We want to see if we can redirect the bot to another site as an expierement. Would there be an entry I could have him add to his robots.txt as it always request it first when entering his site? I know you can get creative with .htaccess but he is on a Windows server.

Once the bot is on my site we want to try the same on my Apache server hosted site. How would I go about this with .htaccess?

Any thoughts or is this just plain silly to try?

Thank you,

Brian

Dijkgraaf

2:22 am on Jul 2, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There is nothing you can do in robots.txt except to disallow google to crawl certain URL's.
If he has full access to IIS you can easily redirect the whole website or a virtual subdirectory to anther URL, but I take it you want to be a bit more selective?
Is he allowed to run .asp pages, if so you can do a redirect in there.
There are two different redirects, 301 Moved Permanently and 302 Found. Which do you want to try?

I take it you aren't too worried about confusing googlebot and possibly losing being listed?

Reflect

4:11 pm on Jul 5, 2005 (gmt 0)

10+ Year Member



I assumed on robots.txt but had to ask.

Yes he can run ASP pages. Please keep in mind though I want to bounce it back to another site.

Either 301 or 302. My main concern is tripping an alarm, so to speak. This is just a test so I don't want to get banned for anything.

Take care,

Brian

Easy_Coder

10:52 am on Jul 7, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You could tell the crawler to 'stay out' and build a google sitemap and distribute that file to google? just an idea.