joined:Nov 2, 2004
My traffic reports show a lot of bandwidth going to various spiders claiming to be googlebot in their user-agent field. I'm pretty confident that the actual Googlebot wouldn't forget to close the parenthesis on the URL. Does anyone know definitively what Googlebot will call itself?
And is there any point in trying to ban the fake googlebots, or will those spiders just learn to spoof better?