Forum Moderators: open

Message Too Old, No Replies

Is there any hope of more deep crawling this month?

Our page rank up, number of pages deep crawled down.

         

latimer

3:02 pm on Jul 9, 2002 (gmt 0)

10+ Year Member



So far this month our number of pages crawled is way down from last month. We have noticed the last several months that google usually comes in and grabs links off the index and then returns some days later to do the deep crawl. We are hoping that googlebot is doing same this month and waiting for the critters return. What are thoughts regarding this months crawl? Please don't say it's over!

Abrexa_UK

3:09 pm on Jul 9, 2002 (gmt 0)

10+ Year Member



I think that the last few pages were taken on Saturday. Most of our pages were taken a few days before that, when the crawl started.

Sorry :(

ciml

3:35 pm on Jul 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I see some deep crawling in my logs from yesterday. Just deeper, low PageRank pages and re-fetching of ODP/Yahoo! listed pages.

taxpod

3:56 pm on Jul 9, 2002 (gmt 0)

10+ Year Member



Last month I had my deepest crawl ever. I mean it went from 50,000 pages to 100,000 pages. This month I'm down to 97,000 pages and sitting here crying my eyes out. I think I've become as spoiled as my two small children!

But on the eve of, what's the name, AOLoogles, I'm a little anxious. But seriously, I'm curious if others had less deep and meaningful crawls also.

WebOpt

3:58 pm on Jul 9, 2002 (gmt 0)



Latimer,
It seems like googlebot has slowed down a bit. But don't worry, she will come back..She always does.

JamesR

5:45 pm on Jul 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Let Google know you update your site often, the crawler loves it.

latimer

6:30 pm on Jul 9, 2002 (gmt 0)

10+ Year Member



JamesR Could you be more specific as to your suggestions?

JamesR

7:10 pm on Jul 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'll leave it up to you but how would a search bot figure out which sites have updated and which haven't?

haxorize

7:43 pm on Jul 9, 2002 (gmt 0)

10+ Year Member



It looks like Googlebot finished crawling our site yesterday after getting 5000+ pages over the course of 8 or 9 days.