homepage Welcome to WebmasterWorld Guest from 54.204.215.209
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Microsoft / Bing Search Engine News
Forum Library, Charter, Moderators: mack

Bing Search Engine News Forum

    
bing indexing without crawling
non-spidered pages in serps
timster




msg:3925922
 8:19 pm on Jun 3, 2009 (gmt 0)

For a particular site that only allows GoogleBot to spider, I am seeing Bing has thousands of pages in its index, but the pages have no "blurb" content. (Yahoo has only a handful of pages indexed for this site.)

The pages even appear high in the SERPS for appropriate specific search phrases.

At first glance I thought bing was ignoring robots.txt, but this does not seem to be the case. They must be indexing these pages based on external links.

Has anyone else seen this?

 

ogletree




msg:3928668
 3:01 pm on Jun 8, 2009 (gmt 0)

Most ranking factors are off page anyhow. Try using the noindex tags on your site. Just set up a script that shows the noindex meta tag when a user is not googlebot.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Microsoft / Bing Search Engine News
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved