Is Freshbot now Deepbot?

The line is getting drawn ever thinner

[trillianjedi]

I've seen several postings about this now in the last few days, although this is my first actual experience of it.

I'm being hit very hard by google's freshbot at the moment, and going deep too. At first glance at what is currently going on with the little guys, I had to check and double check that the IP's were 64.... (they are).

It's behaviour, in terms of hard hitting and depth of crawl (it's going through the entire site) is more like the character of the old deepbot.

In fact, it's identical behaviour to deepbot the last time it crawled this site back in April.

I'm interested in hearing from others who are seeing the same.

TJ

[GoogleGuy]

Some fresh data might be incorporated next-day, but I wouldn't expect everything freshbot found yesterday to make it in a day. There's a lot of data that needs to be fetched and cross-checked--I would expect the full data to show up more on the timeframe of what you would expect from a crawl/index. Step 1 is done (index is at all data centers). I know that one subtle spam filter is going in soon, but Napoleon, I would start counting on the weeks-but-not-months comment beginning from the time that the index switchover finished. Again, just trying to give webmasters information so they have the right expectations: more than weeks, less than months.

But I'm glad that people have noticed that freshie has been crawling deeply in addition to normal freshie duties.

[ogletree]

I don't think there is any consistancy. I have seen sites with new stuff and some with old stuff. Right now there is just no way to know what is going on. I have 51 links today and I had 40 yesterday. Of course they are all from the March Crawl as I stated earlier. I have a friend that had 1 link a month ago and now has 21. We both made the same change to our CMS system so that Google would crawl it at the same time. He has new links I have old ones. I guess we all just have to wait.

[parabola]

So, as I anticipated, we will have to wait for a "normal" period of time to see new backlinks (ie month from now)

[Napoleon]

Thanks GG. The major point I think everyone is choking on though is when will the 'new' data (including April deepcrawl) embed itself into the proper CALCULATIONS for ranking.

I'm not talking the fresh stuff here, which is (or at least was) relatively superficial and short term in the ranking system (come and go). I'm talking stable ranking.

It's this data that has surely caused most of those more recent, but high quality, sites to go/fall. I assume that this will NOT be tomorrow, but "more than weeks, less than months".

That timescale actually sounds like the next mid-June cycle to me (assuming there is one anymore), with the new algo implementation replacing the mid-May cycle.

[GoogleGuy]

I think we're on the same wavelength, Napoleon.

P.S. About the questions you raised about country language/tld/redirection, I think we've got some changes scheduled soon (next week) that should bring it in line with what many users expect again. Just wanted to let you know that that was coming too.

[eflouret]

Sorry, but I don't get it.

GoogleGuy says that the index is in all servers.

What does it means?

I've been continuosly checking google by searching for 'homeandoutdoors' which is the name of one of my sites, and since it only returns results from that site, it is easier for me to check the number of pages indexed.

When I perform that search in different datacenters I get different results:
www -> 2440 pages
www2 -> 272 pages
www3 -> 272 pages
www-cw -> 2360 pages
www-fi -> 2200 pages
www-sj -> 272 pages
www-ex -> 316 pages

So it seems that "the index is the same in all servers" is not what I understand. There is something that I don't know about. Perhaps these are not the servers googleguy says, or perhaps the meaning of 'index' is different from what I supposed.

Thanks,

Enrique

[Napoleon]

Top man.... appreciated.

I think someone mentioned the other day that predictability is always helpful for business. This at least gives us some general direction.

[crobb305]

GG,

You said before that backlinks would be factored in "gradually" and "overtime". That implied continual updating of backlinks over a long period of time. Now it sounds like you are saying that the new data (backlinks) will be factored in all at once, in a few weeks (more than weeks, less than months). Thus, we are stuck with the current backlink count (and missing anchor text) until around mid-June. Is this a correct interpretation?

[parabola]

Not quite sure where everyone got the idea that we were going to have some sort of continual update process. Seems the one they have in place makes more sense, technology wise.

Either way, we can look for a recent index mid June.

[kevinpate]

I've read it differently. It's hardly gradual (in my opinion) if weeks go by and then badabing, badaboom, a new switch is flipped and a whole lot happens in 24 hours.
Maybe I'm misreading GG, but my take is that acknowledgements of watch the calendar, not the clock is that a little happens every few days and it will take a month or more for everything that planned to be fully online, including old and new filters, backlinks since Feb/March era, expanded freshy, etc.

Again, just my opinion. right or wrong, a pizza place isn't gonna swap me a 12 inch pepperoni for it.

[crobb305]

Not quite sure where everyone got the idea that we were going to have some sort of continual update process. Seems the one they have in place makes more sense, technology wise

No one knows what Google is doing. Everyone has had different interpretations and opinions. Hence, 9 update threads with thousands of posts. Regardless, Googleguy's use of the words "gradually" and "overtime" implied to a number of people a slow, continual update of backlinks. Not just one dump of new backlinks 6 weeks later (i.e., a regular monthly update). Besides, if Inktomi and Alltheweb are able to do continual updates, who is to say Google can't (technologically-speaking)?

C

[GoogleGuy]

crobb305, some things will be filtering in sooner (I know a few more autospam filters will happen earlier), and I wouldn't be surprised to see fluctuations in backlinks and pages, but I would hold on to the idea of an update that brings in more data for a little while longer. In time, I do think things will be more gradual. However, we're still in the transition period for this system, so I wouldn't be surprised to see a traditional update for a little while longer. Hope that helps.

[Critter]

Boy is that just my luck or what :) I had all my pages optimized and ready to go for the April crawl/May update, and then Google basically skips the May update.

Reminds me of the old school saw: "The day you decide to turn over a new leaf is the day the teacher decides to make an example out of you."

Peter

[parabola]

Crobb,

You are right that Google is surely capable of doing many things.

IMO, if anything, they have shown how difficult it is to bring in backlinks rather than a shift towards more fluidity. If not, why else would we be seeing Feb data?

Thanks GoogleGuy for clearing everything up and letting us know to expect a regular update at some point with recent backlinks. I think this is the info we have been waiting for.

[GoogleGuy]

Happy to help. I'll try to post if something changes so people know what to expect.