Welcome to WebmasterWorld Guest from

Forum Moderators: mack

Message Too Old, No Replies

Number of indexed pages changes all the time

site:www.domain.com fluctuate widely from day to day

10:27 pm on Mar 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 25, 2004
votes: 0

This has been our situation = MSN had been hitting the site aggressively prior to their search engine launch. When the launch happened the site had about 8,000 URLs in the index. While this seems like a lot - they actually spidered a lot more URLs than that (our site has 50,000 URLs). From then on the number of indexed pages changes quite frequently both UP and DOWN constantly. Looks like MSN does not have any 'memory' of the URLs it spiders and must re-spider pages constantly to have a representation of a site in its index.
10:54 pm on Mar 22, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 22, 2002
votes: 0

That could be a good idea for a rapidly changing web, everytime you respider drop all previously cached pages, on a site by site basis rather than a database by database. I've seen some evidence of this on some sites I used to monitor, everytime MSN respidered by old pages were vanishing from the index. (Superficial testing only I hasten to add).

Stops all the clutter of old, orphaned, dead or vanished pages oi suppose!


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members