Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

robots.txt missing?

3:26 pm on Apr 27, 2004 (gmt 0)

New User

10+ Year Member

joined:Apr 13, 2004
votes: 0

I have a question about the robots.txt file.
It seems that the file is missing on my server.

07:09:55 GET /robots.txt 404
00:17:38 GET /robots.txt 404
03:17:43 GET /robots.txt 404
03:50:32 GET /robots.txt 404

I checked some recent log files. Some of these were from google. All of them only "get" the robots.txt and index.html
How can I add a robots.txt to my account? and is there any sample I can follow?
thx for any help.

5:19 pm on Apr 27, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:July 18, 2002
votes: 37

Here is a tutorial about robots.txt.