homepage Welcome to WebmasterWorld Guest from 54.167.138.53
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Fav icon and robots.txt
Do I need these?
spikedo55




msg:1528114
 2:45 pm on Aug 12, 2003 (gmt 0)

I was looking thru my error log file and found lots of messages referencing these two files? Do I really need these? If I do, where can I go to learn more about them? Thanks.

 

bcolflesh




msg:1528115
 2:47 pm on Aug 12, 2003 (gmt 0)

favicon.com/

robotstxt.org/wc/robots.html

jbinbpt




msg:1528116
 2:56 pm on Aug 12, 2003 (gmt 0)

Robots.txt for sure....

Favicon is optional but cool

spikedo55




msg:1528117
 3:01 pm on Aug 12, 2003 (gmt 0)

Why do I need robots.txt? What does it do? Do you have a sample code of a typical file?

jbinbpt




msg:1528118
 3:09 pm on Aug 12, 2003 (gmt 0)

Check this out also. All the info you need is here
from Search Engine World [searchengineworld.com]

spikedo55




msg:1528119
 3:34 pm on Aug 12, 2003 (gmt 0)

Thanks. Has it hurt my site not having the robots file?

jbinbpt




msg:1528120
 3:49 pm on Aug 12, 2003 (gmt 0)

Not a big problem, but good practice. Check out other threads on this board for robots to disallow. Lots of great stuff here.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved