Forum Moderators: open

Message Too Old, No Replies

Robot html

need meta name information

         

fmmk

3:45 pm on Feb 20, 2004 (gmt 0)

10+ Year Member



Can anyone give me the correct html code lines to include robots, revisit-after verbage, etc.?
Do I need to state the resource type first?
Thank you

tedster

4:50 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



First thing, an important comment.

You should know that today's spidering search engines will not pay any attention to this meta tag. They crawl on their own schedule and that's that. The creation of this tag was a questionable idea at best, probably based on a lack of vision or a poor estimate of how big the web would become.

That said, here's the syntax:

<meta name="revisit-after" content="30 Days">

fmmk

7:05 pm on Feb 20, 2004 (gmt 0)

10+ Year Member



thank you

drbrain

9:13 pm on Feb 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You may have luck with setting Last-Modified and Expires headers for your pages. Setting them appropriately may also reduce load on your site from ordinary users.

fmmk

3:12 am on Feb 22, 2004 (gmt 0)

10+ Year Member



DrBrain-I do not know what this means

tedster

3:53 am on Feb 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Those are server HTTP headers - not really HTML and BRowsers topics, but certainly worht mentioning. In fact, if your server is using HTTP 1.1, send the If-Modified-Since header.

Modern crawlers really appreciate it, and they may be able to spend the allotted time for crawling your domain to find more pages, instead of gathering the same ones they already have.

Straight from GoogleGuy:
[webmasterworld.com...]