Forum Moderators: open

Message Too Old, No Replies

Crawl vs. Crawler

         

pardo

8:22 pm on Sep 5, 2003 (gmt 0)

10+ Year Member



Been hitting good last days by Google and notice today that new content (couple of hours...?) been hitting by crawl.xxx spider and older/existing content by the crawler.xxx spider (64.68.88.xxx vs. 64.68.82.xxx).

Did I miss a previous post or do anyone see familiar actions?

Might be that different spiders collecting existing and completely new data (not pages with adjustments), fresh data for example will be used to add linking information towards the 'ongoing update' cycle database from where PR and inbound links will be determined and serps constructed.

Just my 2 eurocents...

Trisha

12:34 am on Sep 7, 2003 (gmt 0)

10+ Year Member



I was trying to figure out what the differences between crawl and crawler might be a while back too. I'm not sure now if they do different things or not. See this thread:
[webmasterworld.com...]