Welcome to WebmasterWorld Guest from 54.204.74.171

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt missing?

   
3:26 pm on Apr 27, 2004 (gmt 0)

10+ Year Member



hi,
I have a question about the robots.txt file.
It seems that the file is missing on my server.

07:09:55 64.68.82.58 GET /robots.txt 404
00:17:38 69.3.78.160 GET /robots.txt 404
03:17:43 209.113.172.21 GET /robots.txt 404
03:50:32 67.101.186.237 GET /robots.txt 404
.........

I checked some recent log files. Some of these were from google. All of them only "get" the robots.txt and index.html
How can I add a robots.txt to my account? and is there any sample I can follow?
thx for any help.

5:19 pm on Apr 27, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Here is a tutorial about robots.txt.

[searchengineworld.com...]