Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: open
I'm being hit very hard by google's freshbot at the moment, and going deep too. At first glance at what is currently going on with the little guys, I had to check and double check that the IP's were 64.... (they are).
It's behaviour, in terms of hard hitting and depth of crawl (it's going through the entire site) is more like the character of the old deepbot.
In fact, it's identical behaviour to deepbot the last time it crawled this site back in April.
I'm interested in hearing from others who are seeing the same.
You may be right and that is good info. My point is that it is not useful to say that the deepcrawl is happening because new pages are being picked up. It certainly is a strong possibility that the 64 range is now deepbot.
I can't think of any reason why GoogleGuy can't confirm or deny this.
What is a fact, and has been shown by several webmasters logs (including matching of SID's where applicable) is that freshbot has crawled the original deepcrawl index from April.
It may be that has been the case for you also. In doing so, inevitably, the behaviour looks like deepbot because it is, in effect, following deepbots footsteps.
Must be that new algo again
Normal surfers can come in on 64 or 216 etc and often do as you have stated.
Following Fridays crawl my pages were live Saturday/Sunday.
Following todays I would expect them to be live tomorrow/wendesday..
Looks like normal freshbot activity to me :)
While freshbot adds pages, it does not take them away and this badly affects searches.
i mean it is normal behaviour for fresh pages to be listed and dropped again. but while lately almost everyday new pages were added (with fresh tag) for the last 4-5 nothing was added.
i am probably just spoilt expecting google adding everyday new pages of mine ;)