Welcome to WebmasterWorld Guest from 22.214.171.124
Forum Moderators: open
I'm being hit very hard by google's freshbot at the moment, and going deep too. At first glance at what is currently going on with the little guys, I had to check and double check that the IP's were 64.... (they are).
It's behaviour, in terms of hard hitting and depth of crawl (it's going through the entire site) is more like the character of the old deepbot.
In fact, it's identical behaviour to deepbot the last time it crawled this site back in April.
I'm interested in hearing from others who are seeing the same.
You may be right and that is good info. My point is that it is not useful to say that the deepcrawl is happening because new pages are being picked up. It certainly is a strong possibility that the 64 range is now deepbot.
I can't think of any reason why GoogleGuy can't confirm or deny this.
What is a fact, and has been shown by several webmasters logs (including matching of SID's where applicable) is that freshbot has crawled the original deepcrawl index from April.
It may be that has been the case for you also. In doing so, inevitably, the behaviour looks like deepbot because it is, in effect, following deepbots footsteps.
Must be that new algo again
Normal surfers can come in on 64 or 216 etc and often do as you have stated.
Following Fridays crawl my pages were live Saturday/Sunday.
Following todays I would expect them to be live tomorrow/wendesday..
Looks like normal freshbot activity to me :)
While freshbot adds pages, it does not take them away and this badly affects searches.
i mean it is normal behaviour for fresh pages to be listed and dropped again. but while lately almost everyday new pages were added (with fresh tag) for the last 4-5 nothing was added.
i am probably just spoilt expecting google adding everyday new pages of mine ;)
Also, I have access to my visitor IP Addresses(through my webstat program). Is this the correct place to look for freshbot/deepbot addresses? I do not have access to my raw server logs.
joined:May 28, 2002
On the other hand, deepbot is currently busy, since Sunday and up to this morning it has been to my site 678 times already. So we'll see if this is the new deep crawl.
Just one of those things.
If I could stray off topic...
My hosting company had my site down for 19 hours, yesterday to today. I'm in the process of changing companies because of it having happened too many times. Does anyone know if this will deter freshie from coming back if the site was down when it visited? (Freshbot has been dropping by regularly the last week or so).
joined:Oct 23, 2002
I believe Freshbot is usually 64.68.82.* . Deepbot is the one that usually starts with 216.239. And, lately, there has been speculation that freshbot is doing the job of deepbot.
Fresh is/was 64.68.x.x and and there's a new one that starts with 64 (can't remember the rest, saw it in the logs...)
Deep is/was 216.239.x.x
Scooter is 216.39.x.x