Well done Bing!

webmaster error fixed, index updated in 3 days

[caribguy]

I made a redirection mistake on one of my sites that caused http://example.com/robots.txt to serve a default robots file with the 'Disallow /' directive rather than 301 redirecting to http://www.example.com/robots.txt as intended.

The redirect worked fine for any other file except robots.txt but the result was that most pages were ranking for http://example.com/ rather than http://www.example.com/ - None of those pages would have a description, just the URL.

The redirect was fixed last Saturday, and today I'm already starting to see some pages rank properly. Impressive!

[mack]

Bing is getting quick! Even when it comes to new sites/domains. I'm glad its working out for you.

Mack.

[tangor]

I've been very pleased with Bing indexing, and it seems to be getting quicker!

[TheMadScientist]

Yeah, I think they're doing a great job too.

I read in the Google forum someone thought they were not treating the noindex tag correctly, and don't know if they were for a time or not, but it's been working fine for me... They aren't quite as quick as G yet, but it looks like they're getting there.

[weeks]

Good for you. I have two sites where Bing has me number one in the category-rightfully so, G agrees--with eight sublinks on the primary listing. We completely re-did those pages with new links and google handled it within three weeks. Bing? Still waiting, three months-plus...

Still, percent of traffic from Bing to the home page--where 87 percent want to go anyway continues to rise. Overall traffic is up, so it's tough to say if they are getting any share from Google--not that my sites would be any kind of real test.

[abhoy]

The competition is the actual boss here. Bing is in direct competition with the search engine giants. It is bound to improve its performance in order to get more business. Feeling good for Caribguy.

[caribguy]

I'm happy of course, but the reality is that G sends close to 60% of traffic to this site, and Bing a mere 4.3%

Overall keyword positions are very similar between the two.

Noticed 'something' was wrong back in November when some of the descriptions went missing, but I couldn't be bothered to find out and fix it until last week.

[JS_Harris]

Not buying it.

I launched a new site 5 weeks ago and I submitted it to both the Google and Microsoft webmaster centers. I also submitted sitemaps to both.

Google - within 1 day the index and each main category was indexed. 3 days later inner pages began to appear.

Bing - still waiting, nothing indexed.

"Faster" is relative and Bing is still as slow as molasses when compared to G.

edit: I'm still seeing images in Bing being linked to sites that hotlinked them from another site, it SHOULD be a simple matter to ONLY link an image to the site it is HOSTED on ... no?

[LostOne]

Sure would be nice if they followed NOODP robots tag instead of using that outdated ODP title and description tag of mine. It's actually embarrassing every time I see it. The editor used the url instead of a description in the title.

I have widgetsubject.com as the title seen in the SERPS...that's it! Description shows I sell certain products. Shut down sales two years ago. Now it's a "general widget guide"

Hey are those guys at ODP still around? Maybe I can get it changed?

#2 or 3 position for main key phrases...few clicks.

[Silvery]

I've also seen some really nice improvements in Bing's spidering over the past 8 months. Also, some really nice increases in referral traffic.

[JS_Harris]

Is Bing ranking sites with extensive Google services any differently than neutral or Bing related sites? (ie:Google search box, adsense ads etc)

[caribguy]

On this one: Analytics, Maps and Adsense... What made you suspect G services could be a factor?

[kartiksh]

Bing Spidering and Updating SERPs Much Faster.....

and also noticed URLs in search result which are specifically disallowed in robots.txt - although just URLs not the content snippet or title etc. but this too is alarming for us as we do not want anyone to see url path too. So in this case it is displaying a URL domain.com/directory/page.htm where directory and all content under it is disallowed for all spiders in robots.txt

Is this for me only or do we have others here with similar experinece.

[mack]

kartiksh, that can happen with Google as well, but it required a visible link to your pages that are protected by robots.txt

In this case the page will be listed on the serp, but will use link anchor as the title on the results page.

Are you sure there are no pages linking to your excluded pages, even log files that may be spiderable?

Mack.

[blend27]

I have several Bot Trap pages listed in BING, Clearly disallowed in Robots.txt, each page has <meta name="robots" content="noindex, noarchive">. Bing like zis for the past 3 years or zo. PITA when normal users click on them.

[pawas]

Bing has a very tiny index when compared to Google and Yahoo. Just do your own test on a few websites including yours and I bet you wouldn't find 1 website that have more pages indexed in Bing than Google & Yahoo.

Bing is slowest of all in indexing and crawling.

btw, do not trust the numbers Bing display (found xx results), they are mostly fake, go to the last page and you will see actual number of pages indexed.

[rise2it]

"btw, do not trust the numbers Bing display (found xx results), they are mostly fake, go to the last page and you will see actual number of pages indexed. "

...and even though Google only shows the top 1000 results, we're supposed to trust THEIR numbers without question, right?

[TheMadScientist]

Clearly disallowed in Robots.txt, each page has <meta name="robots" content="noindex, noarchive">

When you disallow a page in the robots.txt it will be indexed as URL only because the bot can't access the page to see your noindex,noarchive tag. This is Standard Procedure for search engines, not only Bing...

It's not Bing's fault you disallowed a page in the robots.txt (which they obviously obeyed) and didn't see your noindex tag. They handled it exactly as they should. They did not spider the page, which is what a disallow in the robots.txt tells them to do. Disallow does not mean 'noindex'. They're two different things and one cannot be used in conjunction with, or as a replacement for, the other...

Disallow Mean:
Do not open the URL. Do not access the URL. The URL is 'off-limits'. The more links there are pointing to the Disallowed URL the more likely it is to be shown as a URL only result, because the bot cannot access the page to see if it's a content rich page visitors expect to see in the results or garbage, so they have to rely completely on links to the disallowed page to make any type of determination.

NoIndex Means:
Make no reference to the URL in the results people see.

[blend27]

which they obviously obeyed

NOpe, the bot still hits that page at least 2 times a month, they get a 403 along with <meta name="robots" content="noindex, noarchive">. It's a 1px image & link that is linked from several pages on the site. They disobey Robots.txt. So does the SLURP/3.0, but about twice a week. The URL is NOT shown in Y! SERP. I've never seen this from GoogleBot are ASK Spiders.


In my opinion if the URL is disallowed in robots.txt, the robots should simply stay away from it all together.

[ByronM]

btw, do not trust the numbers Bing display (found xx results), they are mostly fake, go to the last page and you will see actual number of pages indexed.

Not true at all, they're just only publishing the first valid pages and not necessarily exposing all results. All search engines use their algorithms to do this.

[subhankar ray]

Just did a test by adding fresh content page to an established site, Bing beats Google crawler by a few micro-seconds (almost tie..)

2010/02/21 16:59:13¦65.52.22.222¦ bing
2010/02/21 16:59:13¦66.249.71.246¦ google
2010/02/21 16:59:14¦216.24.142.47¦ oneriot ?
2010/02/21 16:59:16¦70.37.70.228¦ bing

[Pass the Dutchie]

Google.de is horribly slow at indexing new pages. Bing listed and ranked pages within a matter of days. Still waiting for Google to list pages uploaded 6 days ago! Not even picking up the unique text on the pages.

The issue might be to do with the fact that while working on the new pages we had used no noindex,nofollow robot directives. Also, did not link from site to new pages until we were ready to go live. The tags were removed and sitemap was submitted 6 days ago to WMT.

Still nothing.....

[onepointone]

Similar situation, disallowed a new url/site for a few days to begin. Google still hasn't come back or indexed. It's in Bing tho.

[Pass the Dutchie]

@onepointone

how long have you been waiting?

[Pass the Dutchie]

and have you also submitted sitemap.xml

[Pass the Dutchie]

ok, the pages in question are now indexed by Google.

[AndyA]

I'm finding new pages on my established site appear in Bing much faster than Google. Google is taking forever, and Googlebot is on that site every day, throughout the day. Weeks go by and still...I wait.

Bing and Yahoo are much fresher for the things I search for. And since I started using Bing, I've discovered some really wonderful sites I never found in Google, when I used it exclusively.

People who use Google search without exception are missing out as far as I'm concerned.

[rise2it]

"And since I started using Bing, I've discovered some really wonderful sites I never found in Google, when I used it exclusively."

LOL....so in other words, you see results other than Amazon, eBay and Wikipedia, which seem to be at the top of most Google results.