homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 95 message thread spans 4 pages: < < 95 ( 1 [2] 3 4 > >     
Next Google Deep Crawl?
What Can We Expect.

 4:14 pm on Jun 7, 2003 (gmt 0)

Can anyone give me an educated guess as to when we will see the next Google Deep Crawl for June and what can we expect to see?



 12:07 pm on Jun 9, 2003 (gmt 0)

olwen, and resolve to crawler11 and crawler12 -> freshbot
The freshbot always resolves to crawlerXX... - the deepbot always resolves to crawlXX...



 6:19 pm on Jun 9, 2003 (gmt 0)

Nope...just freshie.


 7:36 pm on Jun 9, 2003 (gmt 0)

Very brief visit last night - 1/2 dozen pages


 7:23 pm on Jun 12, 2003 (gmt 0)

I just saw an error server message on www. However when I refreshed the page it was back to normal. Can this be the start of a new update?


 7:28 pm on Jun 12, 2003 (gmt 0)

No, it was just an error.


 3:21 am on Jun 13, 2003 (gmt 0)

Perhaps things are indeed getting back to normal:

Upon checking my logs, imagine how suprised I was to see about 1600 page fetches from:

Unless I've been working too many hours ( and I have ) and am seeing things ( which I don't think so ), it looks the deep crawler is diving in!

- SP

[ edit ] added the .46 to the IP's [ /edit ]

[edited by: SilentPartner at 3:49 am (utc) on June 13, 2003]


 3:32 am on Jun 13, 2003 (gmt 0)

Deepbot is 216.239.46.*.


 3:48 am on Jun 13, 2003 (gmt 0)

My error, I forgot to include the 46...

- Thanks


 3:54 am on Jun 13, 2003 (gmt 0)

There has been speculation (which was kind of confirmed by Googleguy in the cryptic way that he does) - that the freshbot is performing the deep crawl this month.

It may already have started (or finished)


 4:24 am on Jun 13, 2003 (gmt 0)

One of my sites was visited by the bot-like program from earlier this morning. The requests were strange. Two pages were accessed 8 times each. At 5:24:27 am page 1 was accessed 4 times in sequence from the same IP within a two-second window. At 5:24:30 am page 2 was accessed 4 times within 2 seconds.

Then starting at 6:26:08 am the bot-like creature accessed page 1, then page 2 four times in a row over a span of about 30 seconds. (For a total of 4 page accesses each.)

I think that in my case the bot was attempting to figure out how similar the pages are and if they contain the same content. How have I reached this conclusion? The content of the pages in question are pricing comparisons for "Movie Title" and "Movie Title 2". (i.e. the original and the sequel).

I'm not sure if this helps anyone but I thought it might be significant.


 5:12 am on Jun 13, 2003 (gmt 0)

I knew I'd been working too many hours - no deep crawl, I was looking at the wrong data -- wrong time frame.

- SP


 5:13 am on Jun 13, 2003 (gmt 0)

Just had some nice freshbot rankings today :)


 5:36 am on Jun 13, 2003 (gmt 0)

Just to echo what I mentioned in another thread, I think that a lot of people have noticed that freshbot is crawling deeper into websites where the deepbot used to explore. It's getting deeper into not just fresh content these days. Maybe we should rename it to deepfreshbot (freshdeepbot?).


 5:40 am on Jun 13, 2003 (gmt 0)

- I'm going for Fredbot


 5:49 am on Jun 13, 2003 (gmt 0)

I'm sure in a few months we'll be abbreviating it as the dfbot. ;) Then again, who knows? Maybe Fredbot will catch on. I hear that >90% of SEOs believe that Googlebot is a "she" though. :)


 7:05 am on Jun 13, 2003 (gmt 0)

GoogleGuy, without saying it specifically, I think you're trying to tell us that Google search will move to more of an "old school style" continuously updated index like Altavista. Googlebot will process your pages when it has time to do so, and new pages will be added as processing allows. Would I be correct in saying that?

I've always felt that this was a better way for a search engine to do things, but maybe it was never considered possible with Google until on-the-fly PageRank calculations were perfected.


 7:33 am on Jun 13, 2003 (gmt 0)

So is that to say, my friend GoogleGuy, that the "deep crawl" which precedes the update is currently occuring?

I also wonder how this bot now differentiates between content which is immediately published to the live index, and content that is stored for the next update?

Thanks a lot.


 7:45 am on Jun 13, 2003 (gmt 0)

GG, of COURSE she's a "SHE". All Powerful. All knowing. fickle, finicky, difficult to please but delightful when achieved. And we have to wait forever for her when she's getting ready.


 11:21 am on Jun 13, 2003 (gmt 0)

what are these?

9148 0.05% Googlebot/2.1 (+http://www.googlebot.com/bot.html)
10 0.00% Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html)


 11:25 am on Jun 13, 2003 (gmt 0)

10 0.00% Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html)

Never seen that before...


 11:27 am on Jun 13, 2003 (gmt 0)



 1:19 pm on Jun 13, 2003 (gmt 0)

we've seen deepbot this week on one of our client sites. about 1200 pages so far.

bizarre behavior really. we usually see deepbot hit most of our sites and client sites within the (relatively) same timeperiod. but i've only seen deepbot hit the one client site.


 1:30 pm on Jun 13, 2003 (gmt 0)

Can we get an "official" change of the statement; "more than weeks, less than months" to "more than days, less than weeks"? :)

Seriously, though, knowing that freshbot is now doing deepbot's job put a lot of us at ease. Thanks for the info GG!


 2:44 pm on Jun 13, 2003 (gmt 0)

Freshbot does seem to be highly active on my sites at the moment - Fresh Crawl on 10th, these results seem to be in the Serps at the moment - another crawl on 11th - a day of yesterday and another crawl today...

Makes you wonder if all the fresh data does make it into the index as fresh serps - or if it is a deepcrawl in disguise :)

Fredabot :) (Female Fresh/Deep bot - with the added advantage that we can call on Google to Fee the bot!)


 3:32 pm on Jun 13, 2003 (gmt 0)

I have been lurking for better than a year now and have learned alot from the sharing of information. I felt I had to join so I could get the GoogleGuy messages as Did others I am sure. I hope to add some theory and insite if possible to this board. There are many knowledgable SEO's here and there are not many items to be covered that have not been covered extensively in these forums.

I think the posting of the GoogleGuy Q&A was great and feel a new indexing will start soon, probably within a couple days. The reasoning behind this is that GoogleGuy seems to have tried to add some closure to the last update, thus making it possible to put to rest some fears about the ranking methods many felt were affecting them whether perminent or temporary.

With closure of the Dominic update, it now seems to me that a new update is soon to follow. It only makes sense to calm the masses before relighting the fire. I do believe the upcoming index will not be the spam ridden index a lot feel it has been experiencing lately, but, rather it will give much more insite to what the final Algo will be. Not the final Algo, but certainly closer the end result we may see in the near future.

I hope the worried have not changed their SEO thinking out of fear that they have been penalized or the practices they have been using have no weight anymore unless they know they do have something to worry about.(Not following true SEO practices)I don't think there should be such a great change as to affect sites that have been on top and deserve to stay with good relevant content, proper backlinks, and offering what the searcher is looking for.

When trying to make a better mouse trap you sometimes dismantle the one you know works, you then figure out through tried and true testing what can I do to make this mouse trap better. You then start to dismantle and manipulate inner parts piece by piece and test each one to see if the performance is better. Once the performance is better and meets your need you begin to rebuild. You rebuild by adding interior pieces first(you don't build from the outside in, you build from the interior working out)Now at this point, you have a solid base to work with and any enhancements can be made to the final product.

The only thing that seems to be stable since May 5th is the amount of Prozac some claim to be taking....LOL

All in all, I think we can come to a conclusion that Google was not broke, rather Google was undergoing an extreme makeover which in the long run will benifit searchers and honest SEO's and keep us on our toes as should be.

Wait it out and see the wonderful changes ahead. :)


 3:46 pm on Jun 13, 2003 (gmt 0)

I am getting the fresh bot 64***

But I have also got a few the 216.** Deep bots lately:

* User Session - IPASS-1BLK (CA,US)
2003-06-03 18:25:20 - user created
2003-06-03 18:25:20 /zinline/inlinewifiareas1.htm

* User Session - Google Inc. (CA,US)
2003-06-04 22:24:43 - user created
2003-06-04 22:24:43 /zinline/inlinehistory.htm

* User Session - Google Inc. (CA,US)
2003-06-07 07:39:54 - user created
2003-06-07 07:39:54 /zthemes/party.htm

Just an FYI


 4:11 pm on Jun 13, 2003 (gmt 0)

Does anyone know which bot is?


 4:47 pm on Jun 13, 2003 (gmt 0)

Welcome to candleman!

As for new unknown IP's / robots, please use the Search Engine Spider Identification [webmasterworld.com] forum.

As for reports of freshbot and deepbot, please note that freshbot runs from 64.68.82.* and deepbot from 216.239.46.*.

Google has several other uses for 216.239.*.* i.e. Froogle, language tools etc.

The way I read Google Guy's postings (thanks GG!) is:

1. an update is comming soon, no time specifics.
2. freshbot is taking over the duties of deepbot.

Further there are indications that this is going to be the last update (dance session) and a more continous update will take over - a logic implication of freshbot taking over deepbot duties.


 5:03 pm on Jun 13, 2003 (gmt 0)

Actually, ulstrup, I got GoogleGuy to give us a hint on when the next update will occur [webmasterworld.com] (start with msg #33).

Essentially, they're trying for June.


 5:12 pm on Jun 13, 2003 (gmt 0)


I did see that, but I don't consider "trying for June" specific, no more than "more than weeks less than months" ;)

So comming soon does it for me.


 5:48 pm on Jun 13, 2003 (gmt 0)

Will the June Dance have the backlinks from the April and May crawl. I can't seem to find my backlinks anywhere! Are they lost or what?

This 95 message thread spans 4 pages: < < 95 ( 1 [2] 3 4 > >
Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved