goodroi
msg:3682325 | 12:46 pm on Jun 24, 2008 (gmt 0) |
Welcome to WebmasterWorld Senthil! Are you sure it is Google? Some people change their browser's user agent to googlebot. This is also a common practice for data miners. Be careful not to just look at user agents but to confirm the different ips are actually owned by Google. Also Google has multiple bots. They have one for search, adsense, etc. If you are participating in multiple google programs you should expect these bots to be visiting your pages. This will drive up the amount of bot traffic on your site.
|
Senthil
msg:3682334 | 1:08 pm on Jun 24, 2008 (gmt 0) |
Hi Goodroj, I am very sure, it is GoogleBot spider only, and All the IPs is owned by Google Inc. I have blocked GoogleBot IP then swiftly a different GoogleBot IP is crawl my site, and if I removed from the blocking list after that very immediately Google Spider happening on my site, I know the reason because it’s well designed Link Structure (Not Spam), is there any other way to control?
|
goodroi
msg:3683064 | 12:56 pm on Jun 25, 2008 (gmt 0) |
You are already doing the proper way - using google webmaster tools. This can take some time to see results. An unorthodox approach which I do no recommend is to return a 503 for every 4th or 5th request by googlebot. It would lower the amount of bandwidth but it might also create new problems.
|
Senthil
msg:3683729 | 6:07 am on Jun 26, 2008 (gmt 0) |
Yes you are right… But my prospect is (I hope its valid one) Google should offer BOT crawling for specific time or day, such a great option will help lot for millions of webpage sites and Webmasters/SEO… Why can’t we recommend to MattCutts/Google? And how can we recommend?
|
shaiguitar
msg:3735348 | 7:54 am on Aug 31, 2008 (gmt 0) |
Is there any update on this? ?
|
shaiguitar
msg:3735349 | 7:54 am on Aug 31, 2008 (gmt 0) |
Is there any update on this? ?
|
|
