Welcome to WebmasterWorld Guest from 54.144.243.34

Forum Moderators: martinibuster

When will Yahoo spider all my site?

8 months.

   
8:53 am on May 8, 2004 (gmt 0)

10+ Year Member



Yahoo has only indexed my homepage, which was created in October 2003. I have approx 70 pages of reasonable content and about 50 incoming links - all to my homepage.
When will Yahoo index my other pages, or do I need to submit them all?
Any guidance is appreciated.
10:55 pm on May 10, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What kind of inbound links do you have for your site? If there are no problems with your site, and you have good inbound links, Slurp should be spidering you frequently.
11:34 pm on May 10, 2004 (gmt 0)

10+ Year Member



I have almost the same problem. Yahoo reports 154 links to my site, of which about 125 are from duplicates of Zeal and ODP. The rest of my links are legitimate links from other sites, some reciprocal, some not. No link farms.

Yahoo has included 5 of my pages so far. It has been adding them one at a time every couple of weeks. Is there anything other than pay-for-inclusion I can do to make Yahoo spider more of my pages?

Thanks!

Christian

11:50 pm on May 10, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What seems to work for me is to get external inbound links pointing to my interior pages.

Also take a look at the link structure within your site to make sure that the pages you want spidered are not to far off the root.

12:10 am on May 11, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for the advice pmac, I'm finding slurp isn't too thirsty these days either...
8:33 am on May 11, 2004 (gmt 0)

10+ Year Member



I had been searching for my Yahoo indexed pages with
mysite.com
which only showed my homepage

I now realise that the correct sytax should be

site:http://www.mysite.com

This shows 50+ of my pages and so I am already indexed.
Thanks.

12:13 pm on May 11, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've had almost 3000 pages on my site since January and Yahoo still indexes just 139 of them. Hundreds of incoming links, Google PR7. Slurp just doesn't do crawling very well.
12:50 pm on May 11, 2004 (gmt 0)

10+ Year Member



i have the same problem 200 000+ pages and slurp still get my robots.txt and index.html page 20 times/day and never goo deeply.. all the pages are in the root directory and are linked correctly.. i haven't hidden text or link, no popup..

i'm going crazy

2:43 pm on May 11, 2004 (gmt 0)

10+ Year Member



you have 200,000 files just in 1 directory?

I guess you are the same guy I heard about before that had 20,000 emails in his inbox.

3:09 pm on May 11, 2004 (gmt 0)

10+ Year Member



no i'm not the same guy, but yes i have 200 000 files in root directory.. i have my own server and i'm programmer i can do this and more :)
9:46 pm on May 11, 2004 (gmt 0)

10+ Year Member



Can anybody tell me what this means? Every other day Yahoo! Slurp visits my site and this is what shows up in the logs.

2004-05-05 10:36:14 66.196.65.38 - W3SVC33 VEGA 10.0.1.72 80 GET /robots.txt - 200 0 421 197 31 HTTP/1.0 www.widgets.com
Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp) - -
2004-05-05 10:36:14 66.196.73.49 - W3SVC33 VEGA 10.0.1.72 80 GET /index.html - 200 0 542 187 32 HTTP/1.0 www.widgets.com
Mozilla/5.0+(compatible;+Yahoo!+Slurp;+http://help.yahoo.com/help/us/ysearch/slurp) - -

4:52 am on May 17, 2004 (gmt 0)

10+ Year Member



I'm having a similar problem getting all my pages in the index. For about two months now Yahoo seems to have all my pages in, but only displays about 3/4 of them. Whenever I hit "more pages from this site" it says the correct number are there, but looking at the results only yields 3/4 or less. What gives? When will all the pages show up?
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month