Welcome to WebmasterWorld Guest from 54.198.118.102

Forum Moderators: open

Message Too Old, No Replies

Next Google Deep Crawl?

What Can We Expect.

     
4:14 pm on Jun 7, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:June 6, 2003
posts:520
votes: 0


Can anyone give me an educated guess as to when we will see the next Google Deep Crawl for June and what can we expect to see?
12:07 pm on June 9, 2003 (gmt 0)

New User

10+ Year Member

joined:June 7, 2003
posts:20
votes: 0


olwen,

64.68.82.54 and 64.68.82.65 resolve to crawler11 and crawler12 -> freshbot
The freshbot always resolves to crawlerXX... - the deepbot always resolves to crawlXX...

Regards,
captain.

6:19 pm on June 9, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 9, 2003
posts:546
votes: 0


Nope...just freshie.
7:36 pm on June 9, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 12, 2002
posts:198
votes: 0


Very brief visit last night - 1/2 dozen pages
7:23 pm on June 12, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:June 6, 2003
posts:520
votes: 0


I just saw an error server message on www. However when I refreshed the page it was back to normal. Can this be the start of a new update?
7:28 pm on June 12, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 26, 2000
posts:2176
votes: 0



No, it was just an error.
3:21 am on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 8, 2003
posts:7
votes: 0


Perhaps things are indeed getting back to normal:

Upon checking my logs, imagine how suprised I was to see about 1600 page fetches from:

216.239.46.171
216.239.46.226
etc...

Unless I've been working too many hours ( and I have ) and am seeing things ( which I don't think so ), it looks the deep crawler is diving in!

- SP

[ edit ] added the .46 to the IP's [ /edit ]

[edited by: SilentPartner at 3:49 am (utc) on June 13, 2003]

3:32 am on June 13, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 20, 2003
posts:390
votes: 0


Deepbot is 216.239.46.*.
3:48 am on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 8, 2003
posts:7
votes: 0


My error, I forgot to include the 46...

- Thanks

3:54 am on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 4, 2002
posts:666
votes: 0


There has been speculation (which was kind of confirmed by Googleguy in the cryptic way that he does) - that the freshbot is performing the deep crawl this month.

It may already have started (or finished)

4:24 am on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 4, 2003
posts:61
votes: 0


One of my sites was visited by the bot-like program from 216.239.45.4 earlier this morning. The requests were strange. Two pages were accessed 8 times each. At 5:24:27 am page 1 was accessed 4 times in sequence from the same IP within a two-second window. At 5:24:30 am page 2 was accessed 4 times within 2 seconds.

Then starting at 6:26:08 am the bot-like creature accessed page 1, then page 2 four times in a row over a span of about 30 seconds. (For a total of 4 page accesses each.)

I think that in my case the bot was attempting to figure out how similar the pages are and if they contain the same content. How have I reached this conclusion? The content of the pages in question are pricing comparisons for "Movie Title" and "Movie Title 2". (i.e. the original and the sequel).

I'm not sure if this helps anyone but I thought it might be significant.

5:12 am on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 8, 2003
posts:7
votes: 0


I knew I'd been working too many hours - no deep crawl, I was looking at the wrong data -- wrong time frame.

- SP

5:13 am on June 13, 2003 (gmt 0)

Preferred Member

joined:Oct 23, 2002
posts:449
votes: 0


Just had some nice freshbot rankings today :)
5:36 am on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 8, 2001
posts:2882
votes: 0


Just to echo what I mentioned in another thread, I think that a lot of people have noticed that freshbot is crawling deeper into websites where the deepbot used to explore. It's getting deeper into not just fresh content these days. Maybe we should rename it to deepfreshbot (freshdeepbot?).
5:40 am on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 21, 2002
posts:154
votes: 0


- I'm going for Fredbot
5:49 am on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 8, 2001
posts:2882
votes: 0


I'm sure in a few months we'll be abbreviating it as the dfbot. ;) Then again, who knows? Maybe Fredbot will catch on. I hear that >90% of SEOs believe that Googlebot is a "she" though. :)
7:05 am on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:Feb 25, 2003
posts:27
votes: 0


GoogleGuy, without saying it specifically, I think you're trying to tell us that Google search will move to more of an "old school style" continuously updated index like Altavista. Googlebot will process your pages when it has time to do so, and new pages will be added as processing allows. Would I be correct in saying that?

I've always felt that this was a better way for a search engine to do things, but maybe it was never considered possible with Google until on-the-fly PageRank calculations were perfected.

7:33 am on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 23, 2003
posts:163
votes: 0


So is that to say, my friend GoogleGuy, that the "deep crawl" which precedes the update is currently occuring?

I also wonder how this bot now differentiates between content which is immediately published to the live index, and content that is stored for the next update?

Thanks a lot.

7:45 am on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 8, 2002
posts:2335
votes: 0


GG, of COURSE she's a "SHE". All Powerful. All knowing. fickle, finicky, difficult to please but delightful when achieved. And we have to wait forever for her when she's getting ready.
ME/ducks.
11:21 am on June 13, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2003
posts:435
votes: 0


what are these?

9148 0.05% Googlebot/2.1 (+http://www.googlebot.com/bot.html)
10 0.00% Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html)

11:25 am on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 21, 2003
posts:176
votes: 0


10 0.00% Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html)

Never seen that before...

11:27 am on June 13, 2003 (gmt 0)

Full Member

10+ Year Member

joined:May 24, 2003
posts:242
votes: 0

1:19 pm on June 13, 2003 (gmt 0)

Administrator from CA 

WebmasterWorld Administrator bakedjake is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 8, 2003
posts:3878
votes: 57


we've seen deepbot this week on one of our client sites. about 1200 pages so far.

bizarre behavior really. we usually see deepbot hit most of our sites and client sites within the (relatively) same timeperiod. but i've only seen deepbot hit the one client site.

1:30 pm on June 13, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 19, 2002
posts:2139
votes: 0


Can we get an "official" change of the statement; "more than weeks, less than months" to "more than days, less than weeks"? :)

Seriously, though, knowing that freshbot is now doing deepbot's job put a lot of us at ease. Thanks for the info GG!

Dayo_UK

2:44 pm on June 13, 2003 (gmt 0)

Inactive Member
Account Expired

 
 


Freshbot does seem to be highly active on my sites at the moment - Fresh Crawl on 10th, these results seem to be in the Serps at the moment - another crawl on 11th - a day of yesterday and another crawl today...

Makes you wonder if all the fresh data does make it into the index as fresh serps - or if it is a deepcrawl in disguise :)

Fredabot :) (Female Fresh/Deep bot - with the added advantage that we can call on Google to Fee the bot!)
:)

3:32 pm on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:June 12, 2003
posts:32
votes: 0


I have been lurking for better than a year now and have learned alot from the sharing of information. I felt I had to join so I could get the GoogleGuy messages as Did others I am sure. I hope to add some theory and insite if possible to this board. There are many knowledgable SEO's here and there are not many items to be covered that have not been covered extensively in these forums.

I think the posting of the GoogleGuy Q&A was great and feel a new indexing will start soon, probably within a couple days. The reasoning behind this is that GoogleGuy seems to have tried to add some closure to the last update, thus making it possible to put to rest some fears about the ranking methods many felt were affecting them whether perminent or temporary.

With closure of the Dominic update, it now seems to me that a new update is soon to follow. It only makes sense to calm the masses before relighting the fire. I do believe the upcoming index will not be the spam ridden index a lot feel it has been experiencing lately, but, rather it will give much more insite to what the final Algo will be. Not the final Algo, but certainly closer the end result we may see in the near future.

I hope the worried have not changed their SEO thinking out of fear that they have been penalized or the practices they have been using have no weight anymore unless they know they do have something to worry about.(Not following true SEO practices)I don't think there should be such a great change as to affect sites that have been on top and deserve to stay with good relevant content, proper backlinks, and offering what the searcher is looking for.

When trying to make a better mouse trap you sometimes dismantle the one you know works, you then figure out through tried and true testing what can I do to make this mouse trap better. You then start to dismantle and manipulate inner parts piece by piece and test each one to see if the performance is better. Once the performance is better and meets your need you begin to rebuild. You rebuild by adding interior pieces first(you don't build from the outside in, you build from the interior working out)Now at this point, you have a solid base to work with and any enhancements can be made to the final product.

The only thing that seems to be stable since May 5th is the amount of Prozac some claim to be taking....LOL

All in all, I think we can come to a conclusion that Google was not broke, rather Google was undergoing an extreme makeover which in the long run will benifit searchers and honest SEO's and keep us on our toes as should be.

Wait it out and see the wonderful changes ahead. :)

3:46 pm on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 7, 2002
posts:75
votes: 0


I am getting the fresh bot 64***

But I have also got a few the 216.** Deep bots lately:

* User Session 216.239.97.227 - IPASS-1BLK (CA,US)
2003-06-03 18:25:20 - user created
2003-06-03 18:25:20 /zinline/inlinewifiareas1.htm

* User Session 216.239.37.5 - Google Inc. (CA,US)
2003-06-04 22:24:43 - user created
2003-06-04 22:24:43 /zinline/inlinehistory.htm

* User Session 216.239.33.5 - Google Inc. (CA,US)
2003-06-07 07:39:54 - user created
2003-06-07 07:39:54 /zthemes/party.htm

Just an FYI

4:11 pm on June 13, 2003 (gmt 0)

New User

10+ Year Member

joined:May 29, 2003
posts:5
votes: 0


Does anyone know which bot 200.82.183.145 is?
4:47 pm on June 13, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 25, 1999
posts:405
votes: 0


Welcome to candleman!

As for new unknown IP's / robots, please use the Search Engine Spider Identification [webmasterworld.com] forum.

As for reports of freshbot and deepbot, please note that freshbot runs from 64.68.82.* and deepbot from 216.239.46.*.

Google has several other uses for 216.239.*.* i.e. Froogle, language tools etc.

The way I read Google Guy's postings (thanks GG!) is:

1. an update is comming soon, no time specifics.
2. freshbot is taking over the duties of deepbot.

Further there are indications that this is going to be the last update (dance session) and a more continous update will take over - a logic implication of freshbot taking over deepbot duties.

5:03 pm on June 13, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 23, 2003
posts:163
votes: 0


Actually, ulstrup, I got GoogleGuy to give us a hint on when the next update will occur [webmasterworld.com] (start with msg #33).

Essentially, they're trying for June.

5:12 pm on June 13, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 25, 1999
posts:405
votes: 0


Seattle_SEM

I did see that, but I don't consider "trying for June" specific, no more than "more than weeks less than months" ;)

So comming soon does it for me.

This 95 message thread spans 4 pages: 95