Google Plans a New Meta Tag - "unavailable_after"

[tedster]

Jill Whalen reports from the third SEMNE event (Search Engine Marketing New England), that Dan Crow, director of crawl systems at Google, gave feedback on a number of issues - including plans for a new unavailable_after meta tag that will identify time-sensitive content.

Google is coming out with a new tag called “unavailable_after” which will allow people to tell Google when a particular page will no longer be available for crawling. For instance, if you have a special offer on your site that expires on a particular date, you might want to use the unavailable_after tag to let Google know when to stop indexing it. Or perhaps you write articles that are free for a particular amount of time, but then get moved to a paid-subscription area of your site.

[highrankings.com...]

[g1smd]

That had better be a meta tag. I see some people already speculating on the syntax and dreaming up stuff that would fail validation. I'll guess that it will need an ISO8601 [webmasterworld.com] / RFC 3339 [webmasterworld.com] compliant [webmasterworld.com] date [webmasterworld.com] now [webmasterworld.com].

.

>> Google is starting to crawl the supplemental index more often... <<

Hmm. Googleguy already said that - about a year to 18 months ago.

[Marcia]

>>Hmm. Googleguy already said that - about a year to 18 months ago.

Yes, and "unavailable after" may just indicate to Google not to bother re-visiting the page to re-index, but does it really, really mean that it'll disappear from the index? Or will it simply languish in the Supplemental index for months to a year?

Added:
Oh yeah, the candle burning brightly on that cake is how to get your site into Google:

1) Get links.
2) Do up and submit a site map.

Adding to that making sure that all pages within a site can be reached by a crawlable link, am I mistaken, or aren't there still people who do all those things saying that they still can't get all their pages indexed?

[edited by: Marcia at 8:27 pm (utc) on July 13, 2007]

[pageoneresults]

Google is coming out with a new tag called "unavailable_after".

Well, I guess they've reached the point where they can now do that and many will follow.

It sure would be nice if ALL the crawlers would just follow one protocol, possibly something already in existence with a slight modification? All these proprietary tags are going to cause issues. I can see it now, This Site Best Viewed in Google.

410 Gone

[Marcia]

>>410 Gone

I've just started using that this past month, with even a nice little custom 410 page.

Redirect gone name-of-page-removed-permanently.html
ErrorDocument 410 that-page-is-gone.html

[g1smd]

I am guessing that the new meta tag is going to be a way to tell Google in advance of when a page goes away, or is no longer relevant, exactly when it is going to be gone.

[Marcia]

a way to tell Google in advance of when a page goes away, or is no longer relevant, exactly when it is going to be gone.

So?

It's also a good way to tell Google that whoever is responsible for the site attends SEO events or frequents SEO forums, because the average Charlie won't know and probably won't care.

[Marcia]

>>the new meta tag
...is good PR amongst a certain segment of webmasters, I'd say. We also have to admit that the article with the "news" makes darn good link bait. :D

[edited by: tedster at 2:19 am (utc) on July 16, 2007]

[cabowabo]

Wasn't it last year Whalen reported that adding HTML errors to your site would improve your rankings in Google? That turned out to be bogus info and essentially "link bait". Unless there is mention of this from Google directly, I'm not buying it.

[reseller]

Unless there is mention of this from Google directly, I'm not buying it.

Me too :-)

[tedster]

Well, some skepticism is certainly healthy -- it prevents rumors and misinformation, and that's a good thing. And until there's a formal announcement, we won't know the exact syntax for this meta-tag anyway and cannot implement it even if want to.

But Dan Crow really is Google's director of crawl systems (see wikipedia), he really did speak to the marketing group in Rhode Island (there are even photos) and this information is now reported by at least one other first hand attendee (Eric Lander).

I'm more skeptical of some of the spam-control spin directed to the SEO community from various Google vehicles than I am of this purely technical "sneak preview".

[pontifex]

If they continue to spill out new meta tags, the previous ones seemed to help them to make their job easier!

P!

[europeforvisitors]

Well, I guess they've reached the point where they can now do that and many will follow.

Let's hope so. Someone has to be a leader.

The "unavailable_after" tag sounds like an excellent idea, because it's good for both the search engine and site owners. If other search engines want to take advantage of it, great; if not, no harm done.

[pageoneresults]

If other search engines want to take advantage of it, great; if not, no harm done.

I'll disagree with that. There is a lot of harm done. Google ends up indexing all that unavailable_after stuff after its been regurgitated and stripped of that metadata. Same with the nofollow attribute. They are at most, band-aid fixes to a much deeper issue.

[Marcia]

But Dan Crow really is Google's director of crawl systems

Granted, and while it's perfectly lovely of him and other Googlers to communicate with webmasters, IMHO that isn't at all the issue.

some skepticism is certainly healthy -- it prevents rumors and misinformation, and that's a good thing.

My skepticism is very healthy. It's about the wisdom of eating the dinner that this little biscuit is being served with. This tidbit called "unavailable_after" is a side dish that's being served on a cafeteria menu called

"how to get your site into Google".

And until there's a formal announcement, we won't know the exact syntax for this meta-tag anyway and cannot implement it even if want to.

Who would be keeping up with such an announcement and who would be using it anyway? Certainly not the people who are concerned about

"getting their site into Google."

It gives me mental indigestion to see a Google rep's good intentions being "served up" this way.

[sun818]

I hope eBay will use this to exclude auction listings that have expired.

[reseller]

But what would be the difference between the meta tag "unavailable_after" and the one we already know:

The Expires Meta Tag

<meta name="expires" CONTENT="Wed, 08 Aug 2007 08:21:57 GMT">

[igorberger]

Hey, I just use noindex, follow when a product or a topic is deleted...

I have a better meta tag.

meta help="Google"

This can be used for -30 and -950

This will be a hotty--->

[Eric_Lander]

Being at the even I can confirm that Dan went into much detail over this tag. While he covered it on a few slides of his powerpoint, there were more groundbreaking elements to what he shared with the audience.

For example, Crow discussed the ability to add in meta tags for non-traditional web media such as Flash, videos, etc. Still, he certainly added caution in that these things were months (or more) off -- but were being developed.

[Marcia]

Eric, personally it's not an issue of whether the speaker was the "real deal." Not that anyone gives a hoot what I think, but my personal issue is the nefarious way of capitalizing on what as of this point in time is a twinkle in Google's eye. It's kind of like sending out huge announcements and invitations for an elaborate baby shower based on nothing more than a decision to go off the pill.

[edited by: Robert_Charlton at 3:19 am (utc) on July 16, 2007]
[edit reason] edited out response to moved post [/edit]

[RonPK]

@reseller: the Expires-header tells user agents when they should no longer fetch the page from cache:

The Expires entity-header field gives the date/time after which the response is considered stale. A stale cache entry may not normally be returned by a cache (either a proxy cache or a user agent cache) unless it is first validated with the origin server (or with an intermediate cache that has a fresh copy of the entity). [..]

The presence of an Expires field does not imply that the original resource will change or cease to exist at, before, or after that time.

[w3.org...]

[Go2]

I am glad that Google is now realizing the powerful use of meta tags for controlling search engine and web directory listings. There are potentially many applications where a meta tag can be used to provide such directives.

As an example, the unavailable_after or expire tag has been supported by NetInsert since 2000, along with other kinds of useful tags such as revisit, news, and hover.

[PatomaS]

Hi

But is kind of ridiculous that Google "start" a new tag since we have the expires meta and the expires header for at least 10 years now.

The worst part, from my point of view is that a lot of people will start using that tag instead of the expires.

Well ...

Bye

[Sharper]

Can someone tell me why a webmaster would want to use this "unavailable_after" tag instead of just taking the page down and replacing it with a 301 to a different page they want related traffic for?

I mean, what's the actual incentive for a webmaster to add an extra tag to a page in order to tell Google to decrease traffic to their site?

Is achieving lower amounts of free traffic a high-demand area for SEO experts these days?

This "proposed" tag doesn't seem to pass the common sense and stupidity tests.

[igorberger]

Okay I figured out the problem. I had to go back to my first post that asked for email notification and uncheck it.

[edited by: igorberger at 5:51 am (utc) on July 17, 2007]

[plumsauce]

If anyone is foolish enough to bother with a tag that is irrelevant to anyone or anything else, then be sure to also use the meta robots no-cache to prevent caching by google.

Otherwise, the content will become available only from the google cache, surrounded by ads of course, and the excuse is going to be that you said it would not be available from your site after date yyyy-mm-dd. But as a public service, we have cached it in perpetuity.

After all, "not available after" is not the same as "not cached after".

no-cache + 410 is the perfect combination.

It exists today, and is recognised universally.

So why foist off an oval wheel that only fits fords?

"google where serp quality is job 1"

[europeforvisitors]

Can someone tell me why a webmaster would want to use this "unavailable_after" tag instead of just taking the page down and replacing it with a 301 to a different page they want related traffic for?

Maybe because they genuinely don't want a page to be available after a certain date, and they don't want to wait for the old URL to be crawled and processed?

Example: A manufacturer's rebate page with a July 30 expiration date, or an e-commerce site with a sale that ends on August 10.

Just because everyone may not need or use an "unavailable_after" meta statement doesn't mean nobody would find it useful.

[batgirl13uk]

I work at Monster and our jobs are only advertised for a certain amount of time so it is important to us to ensure that they are taken down once the job is filled or expired. Using a tag like this will help us automate the process so expired jobs aren't popping up in the SERPS creating back clicks and unhappy jobseekers.

[benjibones]

They are at most, band-aid fixes to a much deeper issue.

Hey pageoneresults, I'm interested in knowing what that deeper issue may be. New around here, although I have been lurking for a while.

Tnx

[tedster]

Dan Crow announced today that this new tag is now operational at Google:

[googleblog.blogspot.com...]