Welcome to WebmasterWorld Guest from 54.146.221.231

Forum Moderators: goodroi

Message Too Old, No Replies

Google - robots.txt - indexing

     
10:01 am on Aug 21, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Oops, hmm - Google indexed our robots.txt [google.com]. That's the first time I've ever seen that.
10:56 am on Aug 21, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 17, 2001
posts:409
votes: 0


Well, you put a link to it some time ago in one of the forums, didn't you?

(www.webmasterworld.com/forum27/239.htm)

With the pagerank this site has anything that is linked from here gets indexed easily :)

11:17 am on Aug 21, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


Brett_Tabke, did you add the HTML or did Google?
11:52 am on Aug 21, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:June 19, 2001
posts:183
votes: 0


'Please, no more that 1 hit per minute and use off peak hours.' .... lol, must remember to add it to our robots.txt along with 'please rank us very highly'
3:01 pm on Jan 25, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


Nah, it's a pure text file KM.
3:37 pm on Jan 25, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 8, 2001
posts:926
votes: 0


a few links [google.com].

Your robots.txt has got a PR of 4 ;)

9:24 am on June 24, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38047
votes: 11


It still perplexes me that a robots.txt would be indexed. hmmm

Anyone else got a robots.txt index?

9:38 am on June 24, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 20, 2001
posts:478
votes: 0


Google, CNN, W3.org, your in good company

Woz

9:40 am on June 24, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member woz is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 13, 2000
posts:4823
votes: 0


funny, first time I clicked on the link in the first post it came up showing zero responses. Then, on second attempt, it came up. Perhaps still quivering after the aborted update.

Onya
Woz