Forum Moderators: open

Message Too Old, No Replies

Does Yahoo spider pages so slowly?

It's been almost 3 months.

         

Nick Jachelson

5:35 am on Jan 11, 2006 (gmt 0)

10+ Year Member



When I submitted my site to Google about 3 months ago, they indexed 50,000 pages within a week. Sandbox or now, that's still about 90% of my site indexed.

Yahoo, on the other hand, so far indexed only ~700. It seems like they are adding about 10 every day.

Is this normal? At this rate it will be 16 years until all the pages are indexed. Surely that can't be right. I was even linked to from several high-profile, high-ranking sites, which Yahoo shows in the backlinks, but that still hasn't sped up the indexing.

martinibuster

5:42 am on Jan 11, 2006 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Is this a brand new 50,000 page website?

stu_uk

10:22 am on Jan 11, 2006 (gmt 0)

10+ Year Member



slurp is pretty slow at indexing my stuff as well, its also by far the stupidest form of artificial 'intelligence' i have ever witnessed :)

Nick Jachelson

1:09 pm on Jan 11, 2006 (gmt 0)

10+ Year Member



Well, technically it's not new, just under a new domain (the old domain was sold to somebody).

But I guess Yahoo assumes it is and there's not much I can do about it.

BillyS

2:35 pm on Jan 11, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Slurp crawls enough pages to index my site about 7 times over each month. My URLs end with a slash, but Slurp likes to remove it - even though I 301 it back to the slashed version.

Yahoo seems very slow to add new pages. Seems to me they've chosen to add them in batch. I've seen big jumps during the last two announced updates. Other than that, slow and steady.