Check your diagnostics report, folks

Adsense MediaBot checking MSN's cache

[Bluesplinter]

My Adsense diagnostic report has an odd item in it:

Blocked URL
http:/ / cc. msnscache. com/ cache. aspx? q= [number removed]&lang=
en-US&mkt= en-US&FORM= CVRE10

Reason Blocked: Robots.txt

Last Crawl Attempt: Sep 9, 2006

Failed Attempts: 1

Why on earth should MediaBot be trying to find my pages on MSN's cache? I posted about this on the Adsense help group, and noticed someone else has the same error, so you may want to see if it's in your Adsense diagnostics report as well.

Weird.
--
On further reflection, I guess this might simply mean that someone viewed a page on my site in MSN's cache, and Google hasn't yet filtered that cache out of MediaBot's travels.

[edited by: Bluesplinter at 2:29 am (utc) on Sep. 12, 2006]

[RonS]

Because one of your visitors found your site through MSN and requested the cached version.

The cached version contains your AdSense code, and so their browser immediately sent a request for an ad placement to AdSense, giving the MSN cached page URL as it's source. Not being familiar with that URL, AdSense immediately dispatched a MediaPartners bot to scan the page, but MSN blocked access to it.

Edit: Well looks like you figured it out without my help. I should probably read the whole message before hitting the reply button, eh? ;)

[edited by: RonS at 4:44 am (utc) on Sep. 12, 2006]

[incrediBILL]

Now you've learned why you should specify NOARCHIVE in all your pages as their is no valid reason to allow these pages to be displayed as cache, ever.

[Bluesplinter]

They say it's a good day when ya learn something :) I'm in the midst of redesigning my site's page layout, so now's a good time to stick that in there.

[plasma]

Now you've learned why you should specify NOARCHIVE in all your pages as their is no valid reason to allow these pages to be displayed as cache, ever.

Does this have an impact on rankings?
Does this have an impact on visits?

[AlexPAlex]

Saw that entry in my report too...

[Hobbs]

Does this have an impact on rankings?
Does this have an impact on visits?

No, I did it many months back with no effect on serps or traffic.

[fredw]

So is there any possible drawback to doing this? What is lost (if anything) by the absence of a "Cached" link in your SERPs? Does anyone here think they benefit from their "Cached" links in SERPs?

One thing I do know, more often than not, Adense ads are totally off topic when a page is viewed in Google's cache as opposed to the live version.

Also, anyone know, how long would it take for the "Cached" link in Google SERPs to go away after you included the metatags in your pages?

[jomaxx]

Sure there are valid reasons for it to exist.

There's no logic to anyone using the cache most of the time, but if a site is down, or if the content has been changed and I can't find what I'm looking for on the current page, it comes in quite handy.

[theRealairness]

Does anybody know the syntax of NOARCHIVE?

[Mohamed]

<meta name="ROBOTS" content="NOARCHIVE">

[edited by: Mohamed at 11:12 am (utc) on Sep. 13, 2006]

[plasma]

I always use the cached version, if it exists, because it highlights the search terms, the content is unchanged and it loads faster. But I guess that I belong to a minority doing that. That's why I asked.

[Hobbs]

I lost no traffic with nocache
As for the highlighting, Google toolbar has a highlighting button on non cached pages

[WallyWorld]

Using NOARCHIVE can raise a red flag with search engines looking for those who use cloaking on their sites. If you use cloaking it will help the SE's find you so they can ban your site.

[Hobbs]

If you are using cloaking, no archive is the least of your problems.

[incrediBILL]

Using NOARCHIVE can raise a red flag with search engines looking for those who use cloaking on their sites.

Red flag?

Why don't they just not index any page with NOARCHIVE then and save themselves the trouble but that isn't the case as NOARCHIVE simply means don't show my CACHE'd page to searchers.

I'm sure the SE's still have a copy internally, but they don't show the rest of the world.

Besides, if you want to control your content tightly you should always use NOARCHIVE because scrapers can scrape a search engine cache regardless of how good your anti-scaper bot blocking efforts may work.

[ken_b]

Wouldn't the NOARCHIVE tag prevent archive.org (the WayBackMachine) from making a copy? That could be less than helpful if you were trying to prove you published some page/content first.

[WallyWorld]

incrediBILL,

You don't understand. A cached page from a cloaked site shows the visitor the view of a page the search engine sees which is NOT the view the cloaker is wanting a visitor to see. To avoid this problem a cloaker will use the NOARCHIVE setting so visitors must visit the site and see the view the cloaker wants them to see.

Knowing this, the search engines realize that using the NOARCHIVE feature is one of the flags they can use to locate possible cloaked sites for closer review.

[jatar_k]

I am sure incrediBILL understands

yes, the NOARCHIVE tag is a flag, not sure that it is as much anymore as it used to be.

this dicscussion is not about cloaking though, if you can't handle a manual review then you have problems if you get one, cloaking is not for the faint of heart.

As was mentioned the NOARCHIVE tag has many advantages but, depending on someone's specific situation, may have some drawbacks.

It is a decision to be made for each individual, I have used it everywhere on some sites, on specific pages for some sites and not at all on some others.