Welcome to WebmasterWorld Guest from 54.196.238.210

Forum Moderators: goodroi

Message Too Old, No Replies

Changing your Mind with robots.txt

Will googlebot come back to the party?

     

Spine

5:53 am on Nov 26, 2004 (gmt 0)

10+ Year Member



A couple of weeks ago I banned googlebot from a underused domain using robots.txt , now I'm thinking I want to change the content on that domain and it's pages, and let G-bot back in.

I think G-bot has come around 3-4 times and found itself excluded with robots.txt. If I remove the r line referring to gbot in my robots.txt, will it come back at some point, or does it only take the 'insult' so many times before it gives up? As I said, it's only been about 14 days.

I looked around and didn't find any info on this.

Cheers

Spine

4:41 pm on Nov 26, 2004 (gmt 0)

10+ Year Member



Update:

The bot came back today, about 4 days after it's last visit (when it wasn't welcome).

 

Featured Threads

Hot Threads This Week

Hot Threads This Month