Welcome to WebmasterWorld Guest from 54.167.22.37

Forum Moderators: open

Message Too Old, No Replies

Google Deep Crawling My site Today.

     

Chief

3:30 pm on May 1, 2003 (gmt 0)

10+ Year Member



Crawling again. About 20+ bots on my site. Crawl1....(but is the crawl or crawler the deepbot? I forget.

Does this mean the update is farther away than we would like?

John_Caius

3:34 pm on May 1, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



216 is the deepbot, 64 is the freshbot.

PeterHo

2:16 pm on May 2, 2003 (gmt 0)

10+ Year Member



How do you know that bot's have visited your page?
By the IP-numbers of you visitors?

ncsuk

2:22 pm on May 2, 2003 (gmt 0)

10+ Year Member



My site has been deep crawled as well and some pages 4 levels deep have been picked up

speda1

5:42 pm on May 2, 2003 (gmt 0)

10+ Year Member



You cannot tell if its deepbot by the pages that are grabbed. You have to look at the IP of the bot. See John's post above.

tomapple

6:31 pm on May 2, 2003 (gmt 0)

10+ Year Member



havn't seen the deepbot today, just "freshie" hitting a bunch of pages

taxpod

6:31 pm on May 2, 2003 (gmt 0)

10+ Year Member



I'm seeing only freshy.

kmtell

8:22 pm on May 2, 2003 (gmt 0)

10+ Year Member



I have a ton of 216.* hits on my site today.

speda1

8:59 pm on May 2, 2003 (gmt 0)

10+ Year Member



With a user agent of 'Googlebot'?

markusf

9:11 pm on May 2, 2003 (gmt 0)

10+ Year Member



I get a lot of these on my site..

216.239.33.5
Nokia6610/1.0 (4.18) Profile/MIDP-1.0 Configuration/CLDC-1.0 (Google WAP Proxy/1.0)

speda1

1:27 am on May 3, 2003 (gmt 0)

10+ Year Member



That looks human to me.

matrix_neo

4:18 am on May 3, 2003 (gmt 0)

10+ Year Member



What is the difference between up date and google dance (I believe both are the same). Do you guys mean update is PR calculation after the google Or PR calculation is also happening simultaneously?

speda1

5:24 pm on May 3, 2003 (gmt 0)

10+ Year Member



The update and the dance are the same thing. Each of the google data centers are updated with the new index. Because Google load balances with DNS, and because it takes some time to update all of the data centers, search results fluctuate during this time, hence the 'dance'.

PR is most likely calculated during the period between the deep crawl and the start of the dance. We are in this period now.

Jesse_Smith

11:54 pm on May 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



freshBot: 64.68.82.* bah, listed for only a few days, short dinner date then it dumps you.
deepcrawler: 216.239.46.* Good bot, it likes you, listed until death do you depart. So don't make him mad or it will divorce your site. You can divorce the Googlebot by using your robot.txt file. It's much cheaper and faster than going to the courts.

If it's the Googlebot it will say

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

 

Featured Threads

Hot Threads This Week

Hot Threads This Month