Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: open
How the hell do I block it?
Sorry guys , ust a little peeved, bandwidth is so expensive were I live that every little bit counts!
Any help is greatly appreciated.
The whole point of the crawler is to index web sites that can later be searchable by AV users. If your complaining about a legitamite search engine spider then you really cant be planning for any real traffic. You aint going to get real traffic without search engine listings and you aint going to get those without being crawled.