Welcome to WebmasterWorld Guest from 54.162.138.175

Forum Moderators: goodroi

Message Too Old, No Replies

Really necessary to add to robots.txt

     
9:06 pm on Jul 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 19, 2002
posts:61
votes: 0


Is it really necessary to add the two lines to robots.txt like the AdSense FAQ says? As long as you are not explicitly banning their bot, why is this necessary?
9:54 pm on July 4, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 5, 2003
posts:636
votes: 0


It's not necessary. If you're not banning their bots (or all bots) you don't need to worry with adding anything to robots.txt.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members