Reluctant bing

almost no bots and no results

[dstiles]

Only noticed this yesterday and cannot be sure how far back it goes; lack of bots certainly longer than a month. Used to get lots of visits from bingbot and relevant results in serps but now less than half a dozen a day across a dozen sites. Looking for those sites in bing results, none of them are there, even for a domain name.

Nothing changed on sites nor server in several years.

Anyone else seeing this? Is it anything to do with their chat-results\?

The lack of serps also applies to duckduckgo, which reputedly gets its results from bing.

Google has at least some of the sites, though their serps are atrociously presented. Mojeek and Brave serps are fine.

[SumGuy]

I've seen no reduction in bingbot hits from their what, 3 or 4 different IP ranges? Quite a lot of hits, as usual.

Are you signed up with their Bing Webmaster tools?

Do you have a sitelist.xml? Bing likes to read the sitelist file.

[dstiles]

Not signed up with any SE,

Siemap.xml AND sitemap.xml.gz, generated about four months ago.

I've just tried a few online SEO tools and those that work tell me the domain does not exist. I KNOW it does because he site is working and returns good Qualys esults - A+. I'm assuming they're upset because I block a lot of unuseful bots. Actually some of it may be an iptables IP block. I'll have to look into that. It's not so for bing, though, I checked this morning.

[SumGuy]

If you sign up with the bing webmaster tools, you can force their bot to scan your site. Might be a useful starting point.

I personally never use bing. But I did, just now. I tried a couple of key words and my company's site came up on the first page results. I tried just my company name, and got the full deal, with map location, address, etc. This is a domain I've had since about 1998.

But ya know, funny thing, and maybe I'll do a scan for this tommorrow, but I don't recall ever seeing a web hit with bing being the referrer.

Edit: And I block a crap-load of IPv4 IP space, in the router. MS IP's, Google IP's, AWS IP's.

Edit: My cert comes from Lets Encrypt, which I have to renew every 90 days. I always do an ssl Labs (which I guess is Qualys?) test and it always passes after renewing, and I always see this:

Trusted Yes
Mozilla Apple Android Java Windows

Not bad for a site running on a Win-NT4 server eh?

[dstiles]

My wife uses bing and DDG rather than G. Until recently our sites cm up reasonable well in most SEs and still do for non-bing-related ones. These are mostly domains registered in the 90's.

I also use LE for cets buy only rarely check with Qualys.

NT4 - gods, I remember using that a couple of hundred years ago! :) Been using a windows server until a couple of years ago, when I rebuilt all the sites to use linux/apache/php.

[SumGuy]

For HTTPS logs going back to Dec 2020 to the present, I have 75 log lines that have bing somewhere on the line (all microsoft IP's already removed so these are not bing-bot). I can't easily search a specific field - I'm using grep.

47 of these 75 are HEAD's (boils down to only 21 unique IP's). The referrer for all of those is simply [bing.com....] I picked 3 of those and re-scanned the logs to see if they actually requested any files (as a human browser would). They did not. The HEAD was the only thing they did. So those are garbage.

For the remaining 28 (22 unique IP's) most referrers are www.bing.com, one is cn.bing.com, and the rest are www.bing.com/search?q=semalt.com, I chose one of those IP's (an academic institution in Japan) and went down a bit of a rabbit hole with that IP (about 1/2 dozen hits this year, not a bot). I see google, bing, and search.yahoo.co.jp as referrers for these hits (and some are direct, no referrer).

I haven't scanned for hits that would have started as http with bing being the referrer, presumably the referrer would be preserved when the redirect to https is followed.

I could repeat this log scan - looking for google being the referrer, just to get some stats out of this. But I think clearly getting referrals from bing is very rare (for me), I see quite a lot from google.

[dstiles]

Pretty similar on our web server. Lots of G, few B.

I reviewed the server's robots.txt funcionality yesterday. Ever since I set up the server bingbot has been hitting robots.txt on port 80 as well as 443. In theory port 80 is redirected to 443, but bingbot has never followed it, Even with the dearth of bingbot hits of the moment, about half of the robots.txt ones are still to port 80.

Given the lack of bing referers I'm not worried about our ratings in bing, but I am concerned that DDG has lost us.

[SumGuy]

Market share of leading desktop search engines worldwide from January 2015 to March 2023

[statista.com...]

As of March 2023, online search engine Bing accounted for 8.23 percent of the global desktop search market, while market leader Google had a share of around 85.53 percent. Meanwhile, Yahoo's market share was 2.44 percent.
-----------------
[digitalgyd.com...]

Google drives 96% of Mobile Search traffic, followed by Yahoo at 2% and Bing with 1%.
----------------

Another thing I think I'll look for is any referrals from applebot. I see a low level but consistent traffic from applebot. To me, Apple continues to be a big question mark in the search engine world.