Forum Moderators: open

Message Too Old, No Replies

How does the bot tell a page is updated

Does the bot use 'last modified' attribute or compare page content

         

trimmer80

12:15 am on Jan 16, 2004 (gmt 0)

10+ Year Member



Continuing from another post, I am interested in finding out how a bot tells a page has been modified.

I have a large site and content is updated regularly through the database.
If the bot uses the file last modified attribute of the pages then it will alway appear not updated.
If the bot compares the last crawled page with the new it will appear modified.
People have mentioned adding a date to the page to make it appear modified every crawl but this will only help if it compares pages and does you the file last modified.
Any ideas?

pmac

7:56 pm on Jan 16, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Welcome to WebmasterWorld trimmer80.

[webmasterworld.com...]