Welcome to WebmasterWorld Guest from 23.20.162.200

Forum Moderators: goodroi

Message Too Old, No Replies

Google - robots.txt - indexing

     
10:01 am on Aug 21, 2001 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


Oops, hmm - Google indexed our robots.txt [google.com]. That's the first time I've ever seen that.
10:56 am on Aug 21, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 17, 2001
posts:409
votes: 0


Well, you put a link to it some time ago in one of the forums, didn't you?

(www.webmasterworld.com/forum27/239.htm)

With the pagerank this site has anything that is linked from here gets indexed easily :)

11:17 am on Aug 21, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


Brett_Tabke, did you add the HTML or did Google?
11:52 am on Aug 21, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:June 19, 2001
posts:183
votes: 0


'Please, no more that 1 hit per minute and use off peak hours.' .... lol, must remember to add it to our robots.txt along with 'please rank us very highly'
3:01 pm on Jan 25, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


Nah, it's a pure text file KM.
3:37 pm on Jan 25, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 8, 2001
posts:926
votes: 0


a few links [google.com].

Your robots.txt has got a PR of 4 ;)

9:24 am on June 24, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38079
votes: 28


It still perplexes me that a robots.txt would be indexed. hmmm

Anyone else got a robots.txt index?

9:38 am on June 24, 2002 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 20, 2001
posts:478
votes: 0


Google, CNN, W3.org, your in good company

Woz

9:40 am on June 24, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member woz is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 13, 2000
posts:4823
votes: 0


funny, first time I clicked on the link in the first post it came up showing zero responses. Then, on second attempt, it came up. Perhaps still quivering after the aborted update.

Onya
Woz

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members