Forum Moderators: open

Message Too Old, No Replies

Robot.txt - Please help me to make one

Robot.txt . I need to make one

         

kteoj

2:44 am on Oct 3, 2003 (gmt 0)

10+ Year Member



Hi

I am getting a lot error messages in my stats about no robot.txt was found.

I read a some about robot but I do not fully understand what it is....

Do I need one?
Can It help me in any way?
Can someone help me make one ... my site is www.g-emarketing.com

Thanks,
Trond Erik jacobsen

jdMorgan

2:54 am on Oct 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



kteoj,

Welcome to WebmasterWorld [webmasterworld.com]!

Here's an authoritative site for robots.txt: [robotstxt.org...]

In simple terms, it is a text file named robots.txt (all lowercase) located in your web root directory (usually the same directory as your site's "home page").

If you would like to have a simple one that will just get rid of the 404 errors, use this:


User-agent: *
Disallow:

This will disallow nothing, so all robots will feel welcome to all pages of your site. However, they won't cause 404 errors any more, since robots.txt will exist.

Just a note: Please be aware that posting your site's URL is against the terms of service [webmasterworld.com] here at WebmasterWorld. Thanks.

Jim