homepage Welcome to WebmasterWorld Guest from 54.196.168.78
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
robots.txt missing?
zyshen




msg:1528783
 3:26 pm on Apr 27, 2004 (gmt 0)

hi,
I have a question about the robots.txt file.
It seems that the file is missing on my server.

07:09:55 64.68.82.58 GET /robots.txt 404
00:17:38 69.3.78.160 GET /robots.txt 404
03:17:43 209.113.172.21 GET /robots.txt 404
03:50:32 67.101.186.237 GET /robots.txt 404
.........

I checked some recent log files. Some of these were from google. All of them only "get" the robots.txt and index.html
How can I add a robots.txt to my account? and is there any sample I can follow?
thx for any help.

 

hannamyluv




msg:1528784
 5:19 pm on Apr 27, 2004 (gmt 0)

Here is a tutorial about robots.txt.

[searchengineworld.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved