homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

bing indexing without crawling
non-spidered pages in serps

 8:19 pm on Jun 3, 2009 (gmt 0)

For a particular site that only allows GoogleBot to spider, I am seeing Bing has thousands of pages in its index, but the pages have no "blurb" content. (Yahoo has only a handful of pages indexed for this site.)

The pages even appear high in the SERPS for appropriate specific search phrases.

At first glance I thought bing was ignoring robots.txt, but this does not seem to be the case. They must be indexing these pages based on external links.

Has anyone else seen this?



 3:01 pm on Jun 8, 2009 (gmt 0)

Most ranking factors are off page anyhow. Try using the noindex tags on your site. Just set up a script that shows the noindex meta tag when a user is not googlebot.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved