Welcome to WebmasterWorld Guest from 184.72.148.222

Forum Moderators: martinibuster

Message Too Old, No Replies

When will Yahoo spider all my site?

8 months.

     
8:53 am on May 8, 2004 (gmt 0)

Full Member

10+ Year Member

joined:Jan 27, 2004
posts:230
votes: 0


Yahoo has only indexed my homepage, which was created in October 2003. I have approx 70 pages of reasonable content and about 50 incoming links - all to my homepage.
When will Yahoo index my other pages, or do I need to submit them all?
Any guidance is appreciated.
10:55 pm on May 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 12, 2000
posts:1808
votes: 0


What kind of inbound links do you have for your site? If there are no problems with your site, and you have good inbound links, Slurp should be spidering you frequently.
11:34 pm on May 10, 2004 (gmt 0)

New User

10+ Year Member

joined:Nov 27, 2003
posts:24
votes: 0


I have almost the same problem. Yahoo reports 154 links to my site, of which about 125 are from duplicates of Zeal and ODP. The rest of my links are legitimate links from other sites, some reciprocal, some not. No link farms.

Yahoo has included 5 of my pages so far. It has been adding them one at a time every couple of weeks. Is there anything other than pay-for-inclusion I can do to make Yahoo spider more of my pages?

Thanks!

Christian

11:50 pm on May 10, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 26, 2001
posts:1422
votes: 0


What seems to work for me is to get external inbound links pointing to my interior pages.

Also take a look at the link structure within your site to make sure that the pages you want spidered are not to far off the root.

12:10 am on May 11, 2004 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1379
votes: 0


Thanks for the advice pmac, I'm finding slurp isn't too thirsty these days either...
8:33 am on May 11, 2004 (gmt 0)

Full Member

10+ Year Member

joined:Jan 27, 2004
posts:230
votes: 0


I had been searching for my Yahoo indexed pages with
mysite.com
which only showed my homepage

I now realise that the correct sytax should be

site:http://www.mysite.com

This shows 50+ of my pages and so I am already indexed.
Thanks.

12:13 pm on May 11, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 27, 2002
posts:927
votes: 0


I've had almost 3000 pages on my site since January and Yahoo still indexes just 139 of them. Hundreds of incoming links, Google PR7. Slurp just doesn't do crawling very well.
12:50 pm on May 11, 2004 (gmt 0)

New User

10+ Year Member

joined:Apr 3, 2004
posts:28
votes: 0


i have the same problem 200 000+ pages and slurp still get my robots.txt and index.html page 20 times/day and never goo deeply.. all the pages are in the root directory and are linked correctly.. i haven't hidden text or link, no popup..

i'm going crazy

2:43 pm on May 11, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:June 8, 2003
posts:177
votes: 0


you have 200,000 files just in 1 directory?

I guess you are the same guy I heard about before that had 20,000 emails in his inbox.

3:09 pm on May 11, 2004 (gmt 0)

New User

10+ Year Member

joined:Apr 3, 2004
posts:28
votes: 0


no i'm not the same guy, but yes i have 200 000 files in root directory.. i have my own server and i'm programmer i can do this and more :)
9:46 pm on May 11, 2004 (gmt 0)

New User

10+ Year Member

joined:Jan 9, 2004
posts:23
votes: 0


Can anybody tell me what this means? Every other day Yahoo! Slurp visits my site and this is what shows up in the logs.

2004-05-05 10:36:14 66.196.65.38 - W3SVC33 VEGA 10.0.1.72 80 GET /robots.txt - 200 0 421 197 31 HTTP/1.0 www.widgets.com
Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp) - -
2004-05-05 10:36:14 66.196.73.49 - W3SVC33 VEGA 10.0.1.72 80 GET /index.html - 200 0 542 187 32 HTTP/1.0 www.widgets.com
Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp) - -

4:52 am on May 17, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 24, 2002
posts:43
votes: 0


I'm having a similar problem getting all my pages in the index. For about two months now Yahoo seems to have all my pages in, but only displays about 3/4 of them. Whenever I hit "more pages from this site" it says the correct number are there, but looking at the results only yields 3/4 or less. What gives? When will all the pages show up?
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members