HTTP Header

Last modified date - is it used by bots?

[PhraSEOlogy]

If you cloak by using a script and serve up an optimized page no last modified date is sent in the header.

Will this cause a red flag for the bots?

If so, is there a method for sending an artificial last modified date with the page?

[PhraSEOlogy]

I did some searching and apparently there is the XBitHack I can use. Anybody have any other ideas or perls of wisdom?

[glengara]

Not my area of expertise, but in the thread where GG suggests using one, the reasons given were to save bandwidth more than anything else.

I suspect many sites aren't using one yet.....

[PhraSEOlogy]

glengara,

I saw the GG post - thats why I think it may be important to send the "last modified date".

My main concern is that the pages served up by the perl/cgi script look identical to static pages, otherwise the bots may see the pages as "cloaked".

<added>The pages WILL be cloaked - I just dont want the bots to know that</added>

[volatilegx]

The problem with the xbithack method is that the last modified date sent is the same date as the request that is being executed, meaning that the last modified date will change every time the page is accessed.

Keep in mind that you can simply create your own last modified header and send it before the Content-type header.

[PhraSEOlogy]

volatilegx,

Do I really need to worry about sending the "last modified date" in the header? Will the search engines smell a rat if they see a page "widget.htm" - which is not really a static page but output from a script?

[PhraSEOlogy]

Will this work in the script?

#!usr/bin/perl

## processing stuff here

print "Content-type: text/plain\n\n";
print "Last-Modified: $modified_date\n";
print "Content-Length: $length\n";

print "Content-type: text/html\n\n";
print "$HTML";

[volatilegx]

Leave out this:

print "Content-type: text/plain\n\n";

[volatilegx]

Do I really need to worry about sending the "last modified date" in the header? Will the search engines smell a rat if they see a page "widget.htm" - which is not really a static page but output from a script?

No, probably not. Worrying about the last modified header is probably paranoid :)

[PhraSEOlogy]

volatilegx,

Paranoid - LOL - oh dear, it appears you have me sussed...

Thanks for the feedback.