SiteExplorer Data

siteexplorer yahoo

[dgrand]

Hi,

I added my site to siteexplorer, here are the stats.

Crawled Pages: 1,701
All Known Pages: 78,982
Known Hosts on This Site: 4
Hosts linking to Site: 2,955
Domains linking to Site: 2,634

What could be the problem here? Traffic from Yahoo is negligible, but we get 200,000 unique visitors monthly, tons of backlinks.

Yahoo slurp rarely visits the site, site:mysite.com shows 1300 pages in the index. All known pages and crawled pages numbers differ greatly. On another site I have these numbers
102,800 /143,876 which seems a resonable ratio.

DG

[martinibuster]

How many deep links does the site have?

Do many of your backlinks come from:
Recips?
Directories?
Article submissions?
Blogs?
Forums?

[dgrand]

Site has about 1700 deep links, 700 links to the front page, all from good sources, it has been around for 6 years, no recip linking, no links from article directories, some from blogs forums, but just as a reference mentioned by the posters. All links naturally "grown"

[martinibuster]

It's strange that slurp doesn't visit, particularly with the amount of links you have. However it might be something else.

How much 301'ing are you doing? Are there redirects employed within the site from old URLs to new URLs?

[dgrand]

no 301 redirects whatsoever. All pages are static. Sitemap has been submitted to Yahoo siteexplorer with about 30000 pages (as seed to discover about 600,000 pages) Number of Slurp visits for today ~200

My main concern is that less than 1% of the site is indexed by Yahoo. "site:mysite.com shows 1300 pages in the index"

[martinibuster]

Ah, here's one that's a fairly common reason. Unique content. I've seen cases where content aggregated from other sources (RSS, product feed, YouTube, etc.) was keeping the site from performing on Yahoo even though Google was loving it like it's firstborn. Even if it's not aggregated from another source, the content should in general consist of more than links (like a directory page) and should have a couple hundred original words on every page.

Also saw this post [webmasterworld.com]. Inbound links should typically come from outside your own network for best effect.

Longshot, but have you checked in archive.org to see if it was previously owned and maybe abused?

[dgrand]

Unique content, this seems to be the issue then. The content is mostly aggregated and mixed with the original. Regarding the old post this is not the issue any more, supporting domains where dropped (hey this is 5 year old thread :) We are the first owneres of the domain.

[okror]

hmm..I have a very similar issue but all my content is unique. Following is my data from Y!SE:

Crawled Pages: 665
All Known Pages: 1,114
Known Hosts on This Site: 4
Hosts linking to Site: 601
Domains linking to Site: 485
Hosts Outlinked from Site: 1
Domains Outlinked from Site: 1

Google shows I have 56,000 page indexed. And Y! itself told me I have over 4,000 inlinks to the site in Y!SE.

Yahoo crawls about 1500 pages everyday since Feb so it should get most of my pages crawled. Why just show so little here? I registered this domain 3 years ago...

[dgrand]

okror, have you ever submitted yahoo sitemap for your site? It ssems that for us when we submitted a bunch of URLs acouple of years ago then the "halfway" ban was introduced.

[okror]

Thanks Dgrand. Yep, I just did so after I posted my question here. It makes a HUGE difference. Now Y!SE has:
Crawled Pages: 8,074
All Known Pages: 16,558

compare to what I have before submitting the sitemap:

Crawled Pages: 665
All Known Pages: 1,11

Wow.