Welcome to WebmasterWorld Guest from 54.163.115.193

Forum Moderators: Ocean10000 & incrediBILL & phranque

If Modified Since - Non Parsed Headers?

   
8:32 pm on Jan 24, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Anyone have some example if-modified-since code managed from a non-parsed header script? In Perl?
1:22 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



No one is doing dynamic If-Modified-Since on the fly from code?

Wow...this maybe an opportunity for and intrepid developer.

2:30 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Probably lots of this going on in the PHP world, but things are getting lonely in PERL-land... :(

Jim

3:17 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The usage of IMS is supported by GoogleBot. If you run a dynamic website (oh say - a forum), you could cut your spidering dramtically by supporting IMS.
8:45 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Presto chango....

For your listening pleasure....

Dynamic if-modified-since support in perl...


#!/usr/bin/perl
# HTTP_IF_MODIFIED_SINCE - Sat, 26 Jan 2008 13:26:06 GMT
# pass $file as full path to file to check modified time.
# minor code used from: #http://vancouver-webpages.com/proxy/log-tail.pl

$imsd = $ENV{'HTTP_IF_MODIFIED_SINCE'};
$ims = $imsd;
$ims =~ s/;.*//;# for Netscape 4
$ims = &get_gmtime($ims);
@stat = stat($file);

if ($ims && $stat[9]<=$ims) {
print "Status: 304 Not Modified\n\n";
exit;
}

else your code continues here.

#-------------------------------

# library routine called from above. ripped from wwwdates.pl found freely on web.
sub get_gmtime {
use Time::Local;
$Mstr = 'JanFebMarAprMayJunJulAugSepOctNovDec';
@DoW = ('Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday');
@MoY = ('Jan','Feb','Mar','Apr','May','Jun',
'Jul','Aug','Sep','Oct','Nov','Dec');
local($_) = @_;
local($[) = 0;
local($day, $mn, $yr, $hr, $min, $sec, $adate, $atime, $mon, $midx);
local($offset) = 0;

# Split date string
local(@w) = split;

# Remove useless weekday, if it exists
if ($w[0] =~ /^\D/) {
shift(@w);
}

if (!$w[0]) {
return 0;
}

if ($w[0] =~ /^\D/) {
$mn = shift(@w);
$day = shift(@w);
$atime = shift(@w);
shift(@w);
$yr = shift(@w);
}
elsif ($w[0] =~ m#/#) {
($adate, $atime) = split(/:/, $w[0], 2);
($day, $mn, $yr) = split(/\//, $adate);
shift(@w);
if ( $w[0] =~ m#^([+-])(\d\d)(\d\d)$# ) {
$offset = (3600 * $2) + (60 * $3);
if ($1 eq '+') {
$offset *= -1;
}
}
}
elsif ($w[0] =~ /-/) {
($day, $mn, $yr) = split(/-/, $w[0]);
shift(@w);
$atime = $w[0];
}
else {
$day = shift(@w);
$mn = shift(@w);
$yr = shift(@w);
$atime = shift(@w);
}
if ($atime) {
($hr, $min, $sec) = split(/:/, $atime);
}
else {
$hr = $min = $sec = 0;
}

if (!$mn ($yr!~ /\d+/)) {
return 0;
}
if (($yr > 99) && ($yr < 1970)) {
return 0;
} # Epoch started in 1970

if ($yr < 70) {
$yr += 100;
}
if ($yr >= 1900) {
$yr -= 1900;
}
if ($yr >= 138) {
return 0;
}# Epoch counter maxes out in year 2038

# Translate month name to number
$midx = index($Mstr, substr($mn,0,3));

if ($midx < 0) {
return 0;
}
else {
$mon = $midx / 3;
}

return (timegm($sec, $min, $hr, $day, $mon, $yr) + $offset);
}

9:25 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Don't forget to run each of your included libraries as well as any database table timestamp entries through your date routines to grab and store the latest date for comparison. For example, if you have a content system that is including a header, a footer, another included content file from the filesystem or perhaps even an entry from a database table (hopefully your content entry in the database tables(s) are being given last modified timestamps!) you will want to check the latest date entry and use that in your If-Last_Modified comparison.

I find it easiest to push the dates into an array as unix timestamps and grab the max from the array.

10:56 pm on Jan 29, 2008 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Ya, I hear ya. My main issue was with one db part (msgs here), so a simple file stat check (last modified) was all that was needed.

Just since I posted the above, we have served not-modified to Google bot 22,000 times. The resource savings is going to be huge.

$logims= 1;
$EnableIMS= 1;

if ($EnableIMS) {
$imsd = $ENV{'HTTP_IF_MODIFIED_SINCE'};
$ims = $imsd;
$ims =~ s/;.*//;# for Netscape
$ims = &get_gmtime($ims);
@stat = stat($threaddat);
if ($ims && $stat[9]<=$ims) {
if ($logims) {
$imz= &rawdateformat($ims);
$ftz= &rawdateformat($stat[9]);
open (HEADERLOG,">>$data/headerlog-ims.dat");
print HEADERLOG qq¦--------------------: $threaddat \n$ENV{HTTP_USER_AGENT}\n¦;
print HEADERLOG qq¦imsd: $imsd\n¦;
print HEADERLOG qq¦ims : $ims - $imz\n¦;
print HEADERLOG qq¦ft : $stat[9] - $ftz\n\n¦;
close HEADERLOG;
}
print "Status: 304 Not Modified\n\n";
exit;
}
}

3:02 am on Jan 30, 2008 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



here is what lincoln stein thinks about if-modified-since [oreilly.com] including some mod_perl tips.
(he's Mr. Perl CGI, to you...)
9:18 pm on Feb 6, 2008 (gmt 0)

10+ Year Member



When testing a while ago, I found that the bots are much more likely to send an If-Modified-Since if we serve them a Last-Modified header.

What I find troublesome in handling IMS is that one should not only check if the data has changed, but all templates involved in generating the layout should also be checked. (at least those that could make a significant difference for what content is displayed)

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month