Welcome to WebmasterWorld Guest from 54.163.168.15

Forum Moderators: goodroi

Really necessary to add to robots.txt

   
9:06 pm on Jul 4, 2003 (gmt 0)

10+ Year Member



Is it really necessary to add the two lines to robots.txt like the AdSense FAQ says? As long as you are not explicitly banning their bot, why is this necessary?
9:54 pm on Jul 4, 2003 (gmt 0)

10+ Year Member



It's not necessary. If you're not banning their bots (or all bots) you don't need to worry with adding anything to robots.txt.
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month