homepage Welcome to WebmasterWorld Guest from 54.145.183.169
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
HEAD checks
wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4443211 posted 1:13 am on Apr 20, 2012 (gmt 0)

Absurd as this may sound!

If an IP and/or entity does a HEAD check, wouldn't it be a safe assumption that they have a cache file on hand for comparison?

205.188.116.79 - - [20/Apr/2012:01:06:43 +0100] "HEAD

I know it's long been a practice of AOL, however their membership taint what it used to be.

 

MxAngel



 
Msg#: 4443211 posted 3:38 am on Apr 20, 2012 (gmt 0)

Not sure, I've seen many HEAD requests from the usual Twitter swarm. For example the twitter or bit.ly bot uses HEAD to check whether the URL is valid or not.

HEAD does return the full headers without the body / content so it could be used indeed to check if a ressource has been modified or not based upon LastModified/ContentLength.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4443211 posted 3:56 am on Apr 20, 2012 (gmt 0)

But, but. . .why on earth would they desire to check if the page changed unless they held a cache copy for comparison?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved