homepage Welcome to WebmasterWorld Guest from 54.211.34.105
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
Forum Library, Charter, Moderators: martinibuster

Yahoo Search Engine and Directory Forum

    
SiteExplorer Data
siteexplorer yahoo
dgrand

10+ Year Member



 
Msg#: 3880841 posted 7:31 pm on Mar 28, 2009 (gmt 0)

Hi,

I added my site to siteexplorer, here are the stats.

Crawled Pages: 1,701
All Known Pages: 78,982
Known Hosts on This Site: 4
Hosts linking to Site: 2,955
Domains linking to Site: 2,634

What could be the problem here? Traffic from Yahoo is negligible, but we get 200,000 unique visitors monthly, tons of backlinks.

Yahoo slurp rarely visits the site, site:mysite.com shows 1300 pages in the index. All known pages and crawled pages numbers differ greatly. On another site I have these numbers
102,800 /143,876 which seems a resonable ratio.

DG

 

martinibuster

WebmasterWorld Administrator martinibuster us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3880841 posted 8:07 pm on Mar 28, 2009 (gmt 0)

How many deep links does the site have?

Do many of your backlinks come from:
Recips?
Directories?
Article submissions?
Blogs?
Forums?

dgrand

10+ Year Member



 
Msg#: 3880841 posted 3:32 am on Mar 29, 2009 (gmt 0)

Site has about 1700 deep links, 700 links to the front page, all from good sources, it has been around for 6 years, no recip linking, no links from article directories, some from blogs forums, but just as a reference mentioned by the posters. All links naturally "grown"

martinibuster

WebmasterWorld Administrator martinibuster us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3880841 posted 6:31 am on Mar 29, 2009 (gmt 0)

It's strange that slurp doesn't visit, particularly with the amount of links you have. However it might be something else.

How much 301'ing are you doing? Are there redirects employed within the site from old URLs to new URLs?

dgrand

10+ Year Member



 
Msg#: 3880841 posted 4:30 am on Mar 30, 2009 (gmt 0)

no 301 redirects whatsoever. All pages are static. Sitemap has been submitted to Yahoo siteexplorer with about 30000 pages (as seed to discover about 600,000 pages) Number of Slurp visits for today ~200

My main concern is that less than 1% of the site is indexed by Yahoo. "site:mysite.com shows 1300 pages in the index"

martinibuster

WebmasterWorld Administrator martinibuster us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3880841 posted 5:01 am on Mar 30, 2009 (gmt 0)

Ah, here's one that's a fairly common reason. Unique content. I've seen cases where content aggregated from other sources (RSS, product feed, YouTube, etc.) was keeping the site from performing on Yahoo even though Google was loving it like it's firstborn. Even if it's not aggregated from another source, the content should in general consist of more than links (like a directory page) and should have a couple hundred original words on every page.

Also saw this post [webmasterworld.com]. Inbound links should typically come from outside your own network for best effect.

Longshot, but have you checked in archive.org to see if it was previously owned and maybe abused?

dgrand

10+ Year Member



 
Msg#: 3880841 posted 4:53 pm on Mar 30, 2009 (gmt 0)

Unique content, this seems to be the issue then. The content is mostly aggregated and mixed with the original. Regarding the old post this is not the issue any more, supporting domains where dropped (hey this is 5 year old thread :) We are the first owneres of the domain.

okror

5+ Year Member



 
Msg#: 3880841 posted 6:11 pm on Apr 3, 2009 (gmt 0)

hmm..I have a very similar issue but all my content is unique. Following is my data from Y!SE:

Crawled Pages: 665
All Known Pages: 1,114
Known Hosts on This Site: 4
Hosts linking to Site: 601
Domains linking to Site: 485
Hosts Outlinked from Site: 1
Domains Outlinked from Site: 1

Google shows I have 56,000 page indexed. And Y! itself told me I have over 4,000 inlinks to the site in Y!SE.

Yahoo crawls about 1500 pages everyday since Feb so it should get most of my pages crawled. Why just show so little here? I registered this domain 3 years ago...

dgrand

10+ Year Member



 
Msg#: 3880841 posted 8:33 am on Apr 5, 2009 (gmt 0)

okror, have you ever submitted yahoo sitemap for your site? It ssems that for us when we submitted a bunch of URLs acouple of years ago then the "halfway" ban was introduced.

okror

5+ Year Member



 
Msg#: 3880841 posted 4:43 am on Apr 6, 2009 (gmt 0)

Thanks Dgrand. Yep, I just did so after I posted my question here. It makes a HUGE difference. Now Y!SE has:
Crawled Pages: 8,074
All Known Pages: 16,558

compare to what I have before submitting the sitemap:

Crawled Pages: 665
All Known Pages: 1,11

Wow.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Yahoo Search Engine and Directory
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved