homepage Welcome to WebmasterWorld Guest from 54.227.20.250
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Datedate
do spiders look at that?
fashezee

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 1597 posted 8:26 pm on Jan 14, 2003 (gmt 0)

Is how fresh you content a criteria when spidering occurs?

 

skirril

10+ Year Member



 
Msg#: 1597 posted 8:49 pm on Jan 14, 2003 (gmt 0)

I think that depends,

On a site I co manage (that changes content pretty frequently, and has high traffic volume), I see spiders more often than on a lower-trafficked site (where content does not change that often).

I think the age of the content may well influence the ranking of the site, at least with the 'top5' search engines.

Skirril

jatar_k

WebmasterWorld Administrator jatar_k us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 1597 posted 8:56 pm on Jan 14, 2003 (gmt 0)

this thread dealt with that very issue, specific to googlebot but more than likely applies to other bots as well.

Are you using If Modified Since? [webmasterworld.com]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved