Welcome to WebmasterWorld Guest from 54.166.181.58

Forum Moderators: mack

Message Too Old, No Replies

Does Bing read "noarchive" as "noindex"?

There may be a bug in Bing ...

     
6:29 pm on Oct 14, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts:841
votes: 0


Hi all,

I am trying to investigate a bad case of a disappearing Bing site index. Unfortunately, Bing only keeps 6 months of index data, and it was not a very important source of traffic for me, so I did not investigate while it was hot. Now some of the data is from memory and printscreens I made almost 2 years ago.

So, long story short - many, many years ago I had a site hacked and the investigation, though inconclusively, pointed to the attack vector being thru Google cache. This, and also not seeing a reason for there to be a cache of my pages, I just slapped
<meta name="robots" content="noarchive">
onto all pages of my site. Or at least I thought that it was on all, but now I have discovered that one particular taxonomy of pages (only about 60 pages on a 250K page site) did not have its view updated, and the robots meta was just missing.

That was at least 5-6 years ago, if memory serves. I have never reconsidered the robots meta tag since.

Fast forward to December 15-ish, 2015. The amount of indexed pages in Bing started dropping like flies. In a matter of days it dropped from about 150K indexed to something like 15K, then slowly continued to drop until in a few months there were only 100 or so left. Traffic dried up completely.

So now, two years after the fact, I have started looking into it and discovered two things :

  1. ONLY those pages that accidentally had no meta robots tags remain in the index
  2. Bing Webmaster Portal labels pages with robots "noarchive" as "robot tag needs review", and most suspiciously, Severity High


I have removed the robots tags, but it is too early to tell if there are any changes.

So, the question to this esteemed group of people: has anyone come across issues with noarchive meta robots in Bing?

P.S. during this time Google index of that site had its ups and downs but never so drastic and in general seems unperturbed by the noarchive meta
8:36 pm on Oct 14, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


Does Bing read "noarchive" as "noindex"?
It's not supposed to, but who really knows with Bing. It ignores many standards. For the #2 SE in the Western World, it is about as stupid as it gets.

it dropped from about 150K indexed to something like 15K, then slowly continued to drop until in a few months there were only 100 or so left.
Is this metric coming from Bing Webmaster Tools, of is it your personal observation?

Can't offer any explanation why your pages were de-indexed, but hopefully by changing that meta tag, they will recover.

Just a FYI - All my pages and the pages I have built for others (which are thousands) use the noarchive meta tag and I've seen no issues.
8:48 pm on Oct 14, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts:841
votes: 0


Thanks, keyplyr+ yes, the number of indexed pages is from Bing Webmaster "Total Pages Indexed" graph. I guess, who knows if that's accurate. But if your traffic from Bind stops dead, it's kind of easy to correlate to the index diminishing by 99%, so I believe it could be trusted.

I am thinking there may perhaps be a combination of factors, but "noarchive" seems to play at least some role.

I will try to update this thread as time goes on. Not sure if simply removing the tag would change anything, but I have really no other ideas at this point.
9:37 pm on Oct 14, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


This is serious. Bing doesn't bring in the level of traffic that Google does for most sites, but it plays a major influence in other areas that may affect site standing.

I would resubmit your sitemap.xml to help get Bingbot in motion. I would also pay diligent attention to your site access logs and keep traffic of the pages Bingbot crawls, confirming if they are getting reindexed.
2:26 am on Oct 15, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:8718
votes: 699


This is serious. Bing doesn't bring in the level of traffic that Google does for most sites, but it plays a major influence in other areas that may affect site standing.


How?

Meanwhile, Bing runs lean and mean. If you don't want it archived it might not be worth indexing. One of those cake and eat it, too kind of things.

Not saying that is what is happening, but let's not kid each other, either.
4:00 am on Oct 15, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 17, 2006
posts:841
votes: 0


@tangor my take on "noarchive" was simply to have the cached version shown to a random visitor or not, i.e. to have that extra feature made available to a potential attacker or rather not. Never really occurred to me that it would be taken as a measure of its worth. TBH, until today I never really heard this take on the meaning of "noarchive" to search engines. Perhaps it makes sense, but never heard a search engine's spokesperson sound this opinion. Maybe just missed a huge billboard-sized memo :) .
4:34 am on Oct 15, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


Never really occurred to me that it would be taken as a measure of its worth
I've never read any authoritative statement aligning the noarchive tag with the value of a site in any way.

IMO your issue is something else however I still hope that removing the tag will resolve your issue.

As I said, I've used the noarchive tag for years without issue and I do well in Bing search.
6:08 pm on Oct 15, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 19, 2002
posts:3419
votes: 56


As I said, I've used the noarchive tag for years without issue and I do well in Bing search.


i concur with keyplyr, i use noarchive on every page on every site and i'm well indexed by bing and get traffic from them.
9:27 pm on Oct 15, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Feb 12, 2006
posts:2706
votes: 116


I use noarchive on every single page as well, and bing has indexed the whole lot okay.

My guess would be that you’ve somehow managed to block bingbot in your firewall, or htaccess file.
10:45 pm on Oct 15, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:8718
votes: 699


How many believe that NOARCHIVE means don't keep a copy?

How many believe it means DON'T DISPLAY it as an archive?

Either way it is indexed at least once to find the NOARCHIVE directive.

As to whether it has a weighting signal or not is open to speculation. I suspect NOT....
10:47 pm on Oct 15, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


Either way it is indexed at least once to find the NOARCHIVE directive.
Indexing and crawling are two different things.
12:41 am on Oct 16, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:8718
votes: 699


Yeah, right. Can't have one without the other.
1:20 am on Oct 16, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


Point being, just because a document is crawled does not mean it gets indexed.

The link is *crawled* to determine that it has a noarchive meta tag. This has nothing to do with whether that link will or will not be *indexed.*
2:36 am on Oct 16, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 890


My guess would be that you’ve somehow managed to block bingbot in your firewall, or htaccess file.
Agreed londrum, that's the most likely scenario.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members