Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: open
I'm being hit very hard by google's freshbot at the moment, and going deep too. At first glance at what is currently going on with the little guys, I had to check and double check that the IP's were 64.... (they are).
It's behaviour, in terms of hard hitting and depth of crawl (it's going through the entire site) is more like the character of the old deepbot.
In fact, it's identical behaviour to deepbot the last time it crawled this site back in April.
I'm interested in hearing from others who are seeing the same.
You may be right and that is good info. My point is that it is not useful to say that the deepcrawl is happening because new pages are being picked up. It certainly is a strong possibility that the 64 range is now deepbot.
I can't think of any reason why GoogleGuy can't confirm or deny this.
What is a fact, and has been shown by several webmasters logs (including matching of SID's where applicable) is that freshbot has crawled the original deepcrawl index from April.
It may be that has been the case for you also. In doing so, inevitably, the behaviour looks like deepbot because it is, in effect, following deepbots footsteps.
i mean it is normal behaviour for fresh pages to be listed and dropped again. but while lately almost everyday new pages were added (with fresh tag) for the last 4-5 nothing was added.
i am probably just spoilt expecting google adding everyday new pages of mine ;)
Also, I have access to my visitor IP Addresses(through my webstat program). Is this the correct place to look for freshbot/deepbot addresses? I do not have access to my raw server logs.
On the other hand, deepbot is currently busy, since Sunday and up to this morning it has been to my site 678 times already. So we'll see if this is the new deep crawl.
If I could stray off topic...
My hosting company had my site down for 19 hours, yesterday to today. I'm in the process of changing companies because of it having happened too many times. Does anyone know if this will deter freshie from coming back if the site was down when it visited? (Freshbot has been dropping by regularly the last week or so).