---- What would make you block Google in robots.txt?
pageoneresults - 5:19 pm on May 9, 2011 (gmt 0)
What's that old cliché?
Don't cut off your nose to spite your face.
At what point would you say enough is enough, and block the Google (scraper) bot from your site altogether.
Probably never based on past, current, and predictable future trends.
I think ALL of us here at WebmasterWorld rely on Google to some extent. Many like to state that they don't but, they do! If only 10% of your traffic is from Google, that's 10% you didn't have. For most of us, we're probably at 30%+ with some as high as 90%+, we are reliant on Google for organic and paid traffic and that isn't going to change anytime soon. Not during this generation anyway. ;)