"Some results have been removed" - Site dropped from Bing SERPs

[robzilla]

I was investigating a ~10% traffic drop on one of my sites (10+ years old), and it turns out it's no longer appearing in the search results over at Bing and all Bing-powered sites like Yahoo, AOL and DDG. A site: search returns only "Some results have been removed". In Web search, that is; an Image search still returns thousands of images from the site.

No guideline, malware or other problems reported in Webmaster Tools, no major changes to the site in the past months. I've submitted a support request, but may have to wait up to 10 days for a response, if I get one. Content is original, no scheming or spamming; at most, a bit heavier on the ads lately. The site previously ranked at the top of its niche for most keywords, as it still does on Google, and is one of the most complete and authorative, so I'm a bit stumped by what appears to be a heavy penalty.

Has anyone here had this happen to a site? A quick Google search (heh) indicates it's fairly common on Bing for sites to just disappear like that, and I have yet to find a report where the site eventually recovers.

Here's a visual from Webmaster Tools: [i.imgur.com...] (green is indexed pages, purple is clicks)

[robzilla]

Yep, ads.txt and robots.txt

This would be funny if it wasn't the #2 search engine we're talking about.

[Andem]

This would be funny if it wasn't the #2 search engine we're talking about.

Here's what "Inspect URL" comes back with: "The inspected URL is known to Bing but has some issues which are preventing indexation. We recommend you to follow Bing Webmaster Guidelines to increase your chances of indexation."

[imgur.com...]

Not to mention robots.txt has been crawled for years and it's showing as discovered on 08 Feb 2022.

[robzilla]

I get the same thing when I run robots.txt through the Inspect URL tool. But of course it shouldn't be indexed, so that's fine. It's just weird that for you it appears in the "URLs with guideline issues" report.

I only have one URL in that report right now, and that's one with the words "not-found" in it (and "X not found" in the page title), so Bing seems to think it's a soft 404 (which is apparently a "guideline issue"). Google will actually report these as Soft 404s in Search Console, but that report is empty for me, so they don't have trouble seeing that the mentioned "not-found" URL is in fact just a normal page.

Previously the majority of my URLs were also shown in the "URLs with malware" report. I think I finally pinpointed what file caused this. It was a false positive, I regenerated the executable with a newer version of the program used to create it, and by now the same report in Site Explorer is empty. It's possible that this was what eventually got my penalty lifted. The report is unfortunately useless in fixing the issue.

If you host any executables, ZIP files, macros or any type of document that could possibly contain malware, run them through VirusTotal to see if any positives come up, particularly with the Microsoft engine.

[xnokia5200]

Okay, I tried my luck few weeks ago and submitted another support ticket and got the response (image here) [paste.pics...]

Today i saw one link pop up on the bing search results. I cannot tell you how excited I am (Hopefully it stays there,lol). Will update moving further. If you are facing this issue, here's what I can suggest.
- Links- any broken links, anything out of the ordinary (fixit it) few of my articles are rewritten so i had 302 in place, but there were some URLs that were not - fixed that.
- 404 Errors, make sure you have a 404 page. Unfortunately Bing bots get confused with a genuine 404 and a real error?

Finally,
- Try raising a ticket once every month and wait for their response(don't spam) and if they response with the same generic lame excuse, wait a few weeks see if any changes can be made to the site and request again.

All the best folks!

[robzilla]

Unfortunately Bing bots get confused with a genuine 404 and a real error?

Not sure what you mean by "a genuine 404 and a real error" (they both sound real?), but Bing definitely has trouble distinguishing between soft and hard 404s. But ultimately it's our responsibility to serve the correct error, of course.

Congrats on getting the penalty lifted!

I'm happy to report that my Bing traffic is pretty much back to what it was before this all started.

[xnokia5200]

Oh yeah, thats what I mean lol! Thank you for correcting that, roughly how many weeks it took you to get back to same traffic levels?

[robzilla]

About 4 weeks. Impressions are still a bit lower but CTR has increased, so the number of clicks is about the same as it was.

[levforever]

Here the situation is always the same, zero indexed pages.
We have contacted the assistance, we are waiting.

However as far as our case is concerned, I doubt that the cause is incorrect or miscellaneous links. There may be some kind of blacklist. They cannot say this for legal reasons, otherwise everyone would ask for the reasons. Blocks remain on all of our domains and all of our customers have stopped advertising on Bing.

[MrOnTheRopes]

Our site has suddenly vanished from SERPs (from the data in Webmaster Tools seems it went on 7th April)
Everything looks normal in WT. Site is fully indexed / no problems reported during INSPECT URL
Had to raise a support ticket with Bing...

[robzilla]

Everything looks normal in WT.

Does that include all the filters in the Site Explorer? E.g. "URLs with malware", "URLs with guideline issues", etc.

[MrOnTheRopes]

"No Data Available" reported in those (and most others except Indexed URLs and a few redirects. Normal stuff)

[Ethan_Rob]

Has anyone sent a support email to Bing recently? It's been 15 days since my last email, but they haven't responded.

[MrOnTheRopes]

I had a couple of (pointless) responses to my query above but since then silence (last emailed on 20th April).
As it happens my problem just went away (our site is back in the SERP). I don't know if they did something without telling me. My last email was to ask them...

[Frankengolfer]

Hello all,

Our site has suddenly vanished from SERPs (from the data in Webmaster Tools seems it went on 7th April)
Everything looks normal in WT. Site is fully indexed / no problems reported during INSPECT URL

I had the same issue, on April 9th my site dropped out of the index, all the stats dropped to 0 immediately (got approx. 1k views per day before).

Took a while until I noticed it, then I found this thread and sent a support request to Bing 1 week ago.Got the automated ticket number right away, a member of the support team contacted me 4 days later stating that he is working on my issue.

I have to admit I had little hope, but this morning I got a reply telling me that everything is fine, that my site has been indexed (which was never the problem, I just wasn't listed any longer) and he advised that my drop out may have been caused by not enough traffic:

More so, this is a case that some of the site’s pages have not accrued sufficient ranking signals (such as links, traffic) to be selected into the index

and he recommended re-submitting it.

Anyway, I checked the webmaster tools immediately after his reply and voilà - I'm getting traffic and hits again since three days at the previous levels. So it just took a couple of days from submitting the support request to being back, never expected that.

Thank you for all the information posted here, I hope you will have the same luck regarding getting back listed than I had.

Best regards,

Thomas

[MrOnTheRopes]

Hi Thomas

That's interesting, thanks.

After a while Bing got mine to display again but wouldn't say why it had vanished. It was fine for a week or so, then on 11th May it was gone again. Then on 13th it re-appeared! Then on the 14th May it vanished again (and is still gone today)
I've had to contact them again but it takes sooooo long to get a response.

Maybe I should try re-submitting like you did.

(note WMT looks fine anyway, but it's worth a shot!)

Cheers

[samwest]

May account for DDG's loss of sites too.

[Frankengolfer]

Hi MrOnTheRopes,

the same strange phenomenon here - site kicked out of the index and reappeared after I opened a ticket (as described in my post). After a while, klicks and impressions went down to zero again, however the site was in the index and I was able to find it.

A couple of days later, back to normal. Today again everything dropped to zero again, Webmaster tools states site as indexed, however when searching for it no results are returned (however I don't get the message "not in index" either). Very strange. Luckily at least Google numbers are soaring.

Very confusing.

Cheers,

Thomas

[MrOnTheRopes]

I've opened another support ticket (on the 16th May). The automatic response says "we'll get back to you in 10 days". Such fantastic service! lol

[Amrita]

I thought I would share my quick 2 cents here, I have about 30 small websites none of which are spam, and they vary a lot in content from e-stores to small informational pages with no ads, I have a very random collection but I strive to follow the guidelines of the search engines.

I had every single one of my sites removed simultaneously a few months ago now, everything from my stores to my personal blog to pages to purely informational sites I had not yet put ads on like an htaccess file tool and things like that.

Here is what I found out... So my normal routine if I had a malicious client hitting my websites doing things like trying to guess passwords, trying to find exploitable scripts and other clearly obvious hacking attempts would be to block them globally using the Cloudflare IP list, which I kept as a "blocked IPs" list applied in my firewall. It turns out that a large percentage, maybe 40-50% of all the clearly malicious traffic was originating from Microsoft's networks, which are located all over the world. Some of the traffic was so malicious and persistent that I assumed these had to be customer networks that somebody was using to spam or hack with, and that it should be ok to block those random hosts because I was not blocking the networks under the main Bing ASN where search would live, but I was wrong and it turns out Bing search operates from those same scattered networks with WHOIS showing Microsoft as the owner.

I finally checked one of the sites' live inspection tool and it reported that it could not access the URL, and figured out that I had blocked some of the Microsoft web bots trying to block the malicious traffic. My solution in the end was to purge every IP from my firewall list, and just allow all traffic malicious or not to hit my sites, knowing that I am generally careful about securing things and would just have to hope nothing would actually get hacked in to. As soon as I let all of the malicious hosts hit my servers again I appeared back on Bing with slightly more traffic than I have had before... they are all very small sites but it was a noticeable increase.

After doing all of the digging I could regarding these "rogue" servers I found that many were actually internet security firms running exploit scans against my sites, or were originating from government networks that were using Microsoft systems to perform exploit scanning and hacking attempts on my sites, from various government entities in Virginia which was the biggest offender of any geographic location. Also, these were so very clearly hacking attempts that there was no question left to the viewer of the logs, many of them attempting brute-force scans for very particular exploits or guessing hundreds or thousands of common passwords in a row, but originating from federal property using Microsoft servers.

So the long TLDR; is: If you want to remain in search, do not use or rely on IP firewalls to guard your web server, and avoid automated firewall solutions like Wordfence, or configure them to be more lenient and not fully block the traffic and throttle or rate-limit it instead, and in lieu of a firewall I am transitioning to static HTML websites where feasible and Cloudflare as a proxy for things that must be firewall secured, although I no longer manually create firewall rules I only use the general "Medium security" WAF setting and avoid flagging any particular host, and doing so restored everything to search and in some cases dramatically improved my SERP rankings.

I hope this helps someone, good luck everyone <3

[rdscx]

Moving out from lurk mode and started posting here, I thought I might share with you my experience with this type of Bing issue. Please note this is still ongoing, but seems to solve at least my kind of problem with Bing.

Problem: suddenly, "site:domain.com" returned zero results, with the message "some results were removed".

Solution (my case): My site is quite popular in Google and since the start of 2022 I had thousands of bad links pointing to my site from hacked sites. But not only for my site, it was the same for all competitors. So we're all linked from lots and lots of hacked (legit) sites with scraped content.

I try to stick to the rule of not touching the disavow tool and since I got pretty good natural, organic links in the same period, I just ignored the bad ones. In my case, Google was smart enough to mitigate the attack. I still get them anyway, there's no stop to this. Most of the links go 404 as soon as the site realises it was a hack.

But Bing is not that smart and it simply dropped my site. I also saw some of my Google competition being delisted from Bing as well. After sending the usual email to Bing support and got no reply, I went to the Bing console and starting disavowing links. I had links from around 1500 domains and I selected only 10-15 links which I recognised and simply disavow the rest.

So far, it seems to work as the site slowly rebuilds in Bing:
[i.imgur.com...]

Since the disavow it took around 2 weeks to see the changes (yeah, I moved slowly because sometimes I forget about Bing existing in the first place) :)

And to not forget, I also updated my sitemaps with the new ones.

Hope this helps! Good luck!

[PizzaMatt]

Having this same issue, but the disavow tool seems to be missing. Did bing get rid of it? Tried on a couple of different bing webmaster tools accounts and it isn’t there in any of them.

[phranque]

welcome to WebmasterWorld [webmasterworld.com], PizzaMatt!

according to the Webmaster Tools doc it is still available in the Backlinks Tool [bing.com]

[MrOnTheRopes]

My backlink report doesn't show anything any more (just No Data Available). The disavow tool was available in there when it was showing some data. I can still see backlink counts to various pages in Site Explorer.

[Amrita]

This is the current disavow link for Bing - [bing.com...]

Also for anybody who gets flagged for malware or spam I suggest analyzing your domain using this DNS validation / health checkup tool to see if you may be getting DNS poisoned/hijacked - [dnschecker.org...] .

Along with the DNS health checkup you will want to be sure that you are setting outgoing mail rules with a TXT record to prevent other people mailing out using your domain as the sender. Below is the SPF record that I use for domains that do not send out e-mails, and so by setting this as a TXT record with '@' as the host no outbound e-mail can be delivered ( or you can allow certain hosts, etc., tools like this can help - [spf-record.com...] ):

TXT record to disable all e-mail relaying for a domain:

v=spf1 a -all

[robzilla]

While it's good advice generally to configure SPF for your domains, it's irrelevant to any issues reported by Site Explorer or the Webmaster Guidelines. E-mail spam is separate from web spam, and an SPF record is not going to be looked at by Bing.

(Note that an SPF record does not "disable all e-mail relaying" or prevent anyone from sending e-mail using your domain, it just decreases the chance that the e-mail will reach the recipient's mailbox.)

[Amrita]

They have implemented at least some signals around it, here is one of their patents - [appft.uspto.gov...]

They say that they do not check DNSSEC, DKIM Or SPF records in calculating rankings but they don't have to, they have Gmail to sample from and if people are getting inboxed spam from your domain and are flagged as spam it can pull some arbitrary score down or penalize the site.

[robzilla]

Many more patents are filed than implemented.

Penalize the site eh? Be right back; out purchasing some spam with my competitor's domain in them ;-)

Of course, if it's clear spam and also signed or validated by SPF/DKIM/etc. you might run into some trouble somewhere.

But all of that is neither here nor there with regard to the topic here.

[Andem]

In case anybody is following this, I'll give an update:

It appears that many of the pages that have been removed from bing for our site are some of the most scraped pages. Our site is squeaky clean, especially compared to the sites still listed using our content.

After further review, it appears that your site(s) did not meet the standards set by Bing the last time it was crawled.

It usually takes several weeks to months to get replies and this has been going on for around a year now.

[Frankengolfer]

Hi Andem,
thank you for the update.

After having been relisted twice, I got kicket out of the index a couple of weeks ago and once again I absolutely have no idea why. I'll just try to contect Bing one more time, let's see what happens this time.

As usual, there are no errors, malwares, etc. on my site - it is german travel website, genuine content and online since +20years. Let's see what happens this time.

Regards,
Thomas

[robzilla]

After having been relisted twice, I got kicket out of the index a couple of weeks ago and once again I absolutely have no idea why.

How's Google treating you? A site: search returning normal results?