Forum Moderators: open

Message Too Old, No Replies

Deep Crawl question

keeps crawling what does it cache

         

mfishy

8:32 pm on Feb 10, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi,

GG deepcrawler is still hitting my sites today. My question is, if GG deepcrawls a page multiple times, what visit does it cache? If it's the last visit, then we can really make changes still for next months update.

lazerzubb

11:42 am on Feb 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm not sure which one it caches, normally it doesn't hit the same page more than once, unless it experienced some problem the first time, i would guess that it takes the latest cache.
Are you sure it wasn't the freshbot who hit your site?

More information about the difference can be found at:
Googlebot: Deepbot and Freshbot FAQ and Information [webmasterworld.com]
(as google says "Shameless Self promotion" :) )

mfishy

2:09 pm on Feb 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks,

I'm sure it was the deep crawler. And, every month it hits my high ranked pages multiple times.

I am just curious as to what it does with the information on the second time around

rfgdxm1

7:47 pm on Feb 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My guess is if the deep crawler hits pages more than once, the first time is to get the links, and subsequent hits would be to see if the page had been updated, and use the most recent content if so.

wellnesscafe

2:04 am on Feb 13, 2003 (gmt 0)

10+ Year Member



Hi, I'm new to the deepcrawler info. I assume that it is a googlebot that crawls deep into the site.

I'm going to submit a site with 200 pages to google. It is a site about health and each page contain different information. Will google only index the homepage or will it index the entire site? Is there something I should do to get google to index all 200 pages?

Also, when other quality websites link to my site, will that improve the ranking or motivate google to index all my pages, especially if other sites link to specific files on my site? Does it make any difference at all for google?